Publishing events from an Indigo server to another server

Hi all

First post here. The plan is to hook up some motion sensors and to dispatch the events they create to a HTTP endpoint on another server. An application running on this server will send SMS and tweets depending on what type of event is received from the sensors. The application will probably be written in Ruby/Rails or Node.js.

So far I've been advised on the following set up:

- Z-Wave Aeon Labs Multisensor, which talks to...
- Z-Wave Aeon Labs Series 2 USB Controller, which talks to...
- Indigo running on Mac

What's the best way to receive or intercept the commands that will be routed through Multisensor-to-Controller-to-Indigo? For example:

* Can the Indigo server forward all received commands on to a defined HTTP endpoint (a bit like a Webhook)? If not, perhaps I can add this functionality in a Python plugin.
* Or can the USB commands be interpreted directly from the USB Controller?

Thanks in advance

Neil

[jay (support)]

Lots of different ways to do this actually. Probably the easiest would be to create device state changed triggers for each of the devices you want to monitor. Watch for any change and then the action could execute a little Python script that got the new state value and posted it to the appropriate URL.

If you really wanted all commands then you could use the Z-Wave Command event under the Z-Wave Event type. However, you'd potentially get duplicated data. For instance, the motion sensor wakes up periodically to report the sensor values (temp, humidity, light) which it always reports regardless of whether it's the same as the last report. Using the device state changed triggers will only report changes while watching for the raw commands will give you everything.

Is this some existing server that you're already running? Because you can send SMSs directly from Indigo (via the Send Email action and your cell provider's email-to-sms bridge). There are also python libraries to talk to twitter so you could write a script (or better yet a plugin that could easily be shared) to post data to twitter. Just a thought...

Thanks Jay

- Would it be possible to bind to a global/catch-all for any 'device state changed' triggers occurring across all devices connected to Indigo? Or does a new config option need to be created on a more granular, per-device level?

- The Python script doesn't sound too tricky; receive new state value, parse to JSON, make HTTP POST to URL with JSON params. Although I'll need to wrap my head around how the state value is made available in a var which is local to the script. Maybe there's some Indigo plug-ins on Github which can be used as a starting point.

- Do the Z-Wave Events include a parameter to differentiate between the periodic reports and on-demand triggers? Maybe something like 'event.type'?

The original plan was to publish JSON to separate servers/apps which are already set up to send SMS or tweets...

One comment here... It sounds like you are an experienced programmer, and as Jay asked, probably have other existing systems you wish to interface to. I'd suggest taking a small step back and spending a couple of hours (or days) to get the flavor of Indigo. It is actually much more than it seems at first glance. If effect it is a multi-languare programming environment in witch you string together devices, and their states, triggers, actions, conditions with the support of persistent 'global' variables.

But, to answer your last question... You can access an Indigo device's states(s) from within a Python script, and for many devices, within AppleScripts as well. There is no need to move a state value from the device to a variable to use it in a script. However, you can certainly store a value in a variable to use at some later point - perhaps to measure a delta.

So, as Jay suggested, you can setup a trigger to watch for a specific event and then in the action of that trigger embed a short Python script that will read the desired state(s) and do whatever you want with them.

It is possible through some plugin extension to create triggers based on multiple triggering events. But, I suspect for the type of application you are describing you would be better off with a one-to-one model. Think of it as a case statement.

[jay (support)]

Thanks berkinet - I was going to suggest the same thing. I think spending some time using Indigo to get familiar with it's concepts would go a long way in helping you to understand where the appropriate hooks are for your solution. For instance, just grabbing raw commands and trying to process them yourself is duplicating a lot of functionality that's already in Indigo. No need to reinvent the wheel unless there's really a need.

- Would it be possible to bind to a global/catch-all for any 'device state changed' triggers occurring across all devices connected to Indigo? Or does a new config option need to be created on a more granular, per-device level?

We don't yet expose a way to sniff all Z-Wave traffic like we do for INSTEON and x10 so you'll need to do it device by device. I suspect however that once you get more familiar with Indigo you'll find that filtering all raw traffic is probably overkill.

- The Python script doesn't sound too tricky; receive new state value, parse to JSON, make HTTP POST to URL with JSON params. Although I'll need to wrap my head around how the state value is made available in a var which is local to the script. Maybe there's some Indigo plug-ins on Github which can be used as a starting point.

As berkinet points out you can access device states directly from the Python IOM that we expose - check out the various technical documents on the documentation page.

- Do the Z-Wave Events include a parameter to differentiate between the periodic reports and on-demand triggers? Maybe something like 'event.type'?

Again, I think you'll want to spend some time with Indigo and I think it'll become ore clear.

Thanks berkinet, jay. I'll spend some time getting to grips with Indigo and post back.

I'd be interested in a HowTo on this as well. I'm trying to see if I can get Applescript/Python to sense when there is an Indigo device state change, and push it up to a server, hopefully via in a json format. Sometimes this will be a lot of data, but often it's not that much.

I'm not trying to duplicate Indigo's feature set (that would be a little insane), but instead prevent code I am writing from polling Indigo over and over when it's not needed.

Example: When the bedroom lights turn on, tell the remote server this happen. Don't make the remote server, ask every 10 seconds "did the bedroom lights change?" via the cURL APIs.

I am pretty sure this can by done through a "on receive insteon event" in Applescript, but I want to make sure I am not going down a bad path here. Something tells me there is a better way in Python to do this.

Any ideas?

security for applescript between 2 MACs has been increased in 10.9.
applescript events do not go through anymore, or lat least things that worked before stop working due to "no rights" error messages..

...Any ideas?

Well, I'm not sure this would meet your needs, but I think the RESTful API could do a lot of what you want. If the goal is to link two Indigo instances, then for a simple example create some real device on server A, then create a dummy (virtual) device on server B. Any change in the state of either device triggers a RESTful call to the other server. You could use curl in a shell script (or AppleScript) to generate the packet. For now, you'd have to have one trigger for each device and state. But, when Indigo supports argument passing, you'd really need only 1 trigger. Of course, you have to trap the race condition so that when server A changes the state of the matched device on server B that doesn't generate a return response. But, there are lots of ways to do that.

Another option might be a simple client/server plugin that subscribed to device state changes and fired off packets to the other end, and received packets from the other end. Basing this on a plugin would allow you to contain all the logic in one manageable place.

If the second server is not Indigo, well the same basic mechanisms would work, but you'd have to design your own remote service.

Thanks for the thoughts, berkinet.

And yes, this remote server in this case isn't an Indigo server, but one I have created from scratch myself. I am toying with creating a service that augments Indigo with graphing and monitoring features. To do this well, I'd need Indigo to push up changes to its devices and variables when they change.

The rest APIs work fine when polling, but it's not a scalable solution. For instance, it takes me about 11 seconds to poll the status of 66 Indigo devices in my house. While this does have internet latency buried in it, I think it's too small to matter. So to get semi-real-time accuracy of events, 1/6 second per device just isn't scalable, especially when most devices don't change attributes every few seconds.

The trick here is going to be code that runs on Indigo that pushes up changes to Indigo's devices and variables to a remote server.

Sounds like I'm going to have to code this one myself, unless you know of any code that listens and then does a push. Unfortunately, I'm a PHP coder, and not Python coder, so I have not attempted anything like a plugin yet.

[matt (support)]

Hi Dave,

Take a look at the SQL Logger plugin included with Indigo (its python source code is accessible from the plugin itself by right-clicking on the plugin package file). It logs all Indigo device state and variable value changes to SQL. What you want is somewhat similar -- instead of writing to SQL you'll make remote calls with the changed data. It should provide a good example as how to catch device and variable changes as they occur from an Indigo plugin.

Take a look at the SQL Logger plugin included with Indigo

Ooooo - that might be just what I'm looking for - thanks!

Plugin question: I know they are just finder packages, but if I make a change to a file inside the package, do I need to restart Indigo, or will it automatically pick up the changed source code in the package as soon as I save it?

Thanks,

Dave

...The trick here is going to be code that runs on Indigo that pushes up changes to Indigo's devices and variables to a remote server.

Actually, I was thinking about using the RESTFul API to push "events" triggered by a device change in Indigo to another Indigo server instance. But, if the other "server" doesn't have a RESTful API already and isn't even Indigo, there isn 't much point in using that. OTOH, a simple socket connection between Indigo and your remote server would allow you to push the same data using some simple text based protocol you concoct. Like, device:state:some_other_info:etc ... Or, more-or-less what Matt described as going into the SQL database.

As Matt suggests, I think a plugin is the perfect place to do this since the plugin can subscribe to Indigo events. You just need to filter which events you need and "push" them over your socket connection to your server. There are lots of examples on the web for simple socket client/server implementations in Python. You can also take a look at the apcupsd plugin which uses a very basic server to get events from the apc daemon. Though, in your case, you might want two-way communications - that would allow the server to effect changes in Indigo devices based on some logic. Also, my meta-device plugin (now pretty much obsoleted by Indigo virtual devices) shows how to subscribe and filter Indigo device changes.

Dave:

You just need to reload the plugin (from the plugins menu) after making a change. I've unfortunately done this many, many, many times while developing my plugins :-/.

One thing to note - the SQL Logger can optionally log to a Postgre SQL database; I haven't tried it but presumably that could be on another server (the one you are trying to connect/send info to). OR it could be on the Indigo box and you could connect from your remote server to the database directly (not going through Indigo).

Adam

...if I make a change to a file inside the package, do I need to restart Indigo, or will it automatically pick up the changed source code in the package as soon as I save it?

You need to reload the plugin if you update the code elements of the package. If you update file data then restarting or not would depend on how your plugin uses that data - like cache at start or read as needed.