an idea for managing "stateful" controllers

a common scenario in my house is where i have a device (typically either a lamp or a built-in light fixture) which is controlled by a insteon module (lamplinc or switchlinc respectively) and may be controlled from two or more locations. in the case of a switchlinc, it can be controlled locally or from lamplinc or keypadlinc button in what is sometimes referred to as a "virtual 3-way" arrangement. i put these in places where the electrician had originally installed regular 3-way circuits - hallways, rooms with two or more entrances, etc. i sometimes also put them in rooms where i simply want to be able to control lights from multiple locations - such as a wall-mounted keypad and a tabletop keypad.

the essence of the issue that i want to discuss here is what i would generalize as that of the "stateful controller". that means a controller of a device that has an internal state (a single LED being on or off or a bank of led's that represent the dim level) which ideally should track the state of a device that it is linked with.

so the recent additions to indigo have opened up some new capabilities in this area - specifically, it is possible now to have indigo control the state of a keypadlinc button LED, whereas that was impossible before (x10 kluges aside). i am now running into some questions and issues regarding how to best make use of these new features.

common scenario: i want to be able to control a device from within indigo and have the device and other linked stateful controllers stay in sync. for here's perhaps the simplest and most common example: i have a hallway light connected to a switchlinc and a second switchlinc configured as a "slave" switch at the other end of the hall. when i turn the light on, i want both the load to come on and the slave switch to indicate that it is on.

how do i do this currently in indigo? controlling the load is easy enough, obviously. however if i want the slave to stay in sync, then everywhere i would normally have a simple "control device" action for the load, i now need to send two commands - one to the load and another to the slave.

control pages in particular become more complicated in this scenario. controlling a single device is easy - you can simply create a "control light/ appliance" type control. what if you want to send commands to multiple devices? indigo allows you to essentially add additional devices to be controlled with a single control page item. (i actually had not known that this was possible until i started writing this, but this is extremely useful.) so this means that in the example scenario above, i can make the click action for the control page item corresponding to my hallway light have an action consisting of toggling the load and toggling the slave. so this is good.

however, if we now consider a variant of the above example where one module controls the load and the "slave" switch is a keypadlinc button instead of another switchlinc, then this doesn't work so smoothly. a keypadlinc led is not controlled by a normal insteon command, but instead by a group command. so although indigo now has the ability to control the state of the led, these don't interface smoothly with "control light/appliance" type of controls on control pages, since there are no toggle commands for groups like there are for regular insteon commands. thus, you can turn an LED on. you can turn it off. but you can't include one command that will toggle the current state.

so this seems to imply that keypad led's don't really behave the same way as regular insteon modules. indigo would have to track the state of keypad buttons in order to support a meaningful "toggle" command. this seems like it should not be too hard.

implementing this might have some other useful benefits. suppose i want to use a keypad button to represent a bit of state information, i.e. (my current "home" vs. "away" status). in indigo currently, i need to use a variable to represent this state, and i need to use triggers or other types of "glue" to make the remote LED track the state of the indigo variable and vice versa. if indigo could report on the state of the LED directly, then i wouldn't need the variable at all. i could write scripts something like:

Code: Select all

if on state of button1 of device "foyer keypad"
... do stuff...
end


obviously, the above would entail a more complex representation of devices like keypadlincs - to both track their internal state and expose it via applescript properties.

generally speaking, it is my opinion that indigo's new link management capabilities have exposed the need for perhaps another level of abstraction in indigo - namely that of a set of stateful controllers that can be linked to a module. at first, i was thinking that this might require another top-level object type in indigo for groups of devices that need to stay in sync (in addition to devices, trigger actions, time/date actions, and action groups) so that they could be controlled together with a single command. ...but then it occurred to me that indigo already has the relevant information in its link database.

so i would like to see a new option for indigo's existing "control light/appliance" action type. i envision a new checkbox called "also control linked devices" or "simulate local control" (or something...). if this were checked, then instead of merely sending a single insteon command to directly control a device, indigo would instead modify other devices to keep their status indicators in sync with the device being changed. in other words, i could tell indigo to do the same thing that would have happened if i had pushed the switch/button manually.

this could be implemented in two ways (and thus might merit another sub-option): a) indigo could interpret the link database and send a series of direct commands b) indigo could send a single insteon group command. i think the two options are useful since there are only a total 256 (?) group commands that can be issued by the controller, so for situations where simultaneity of actions is not critical, you could just let indigo "simulate" a group command instead of actually defining one.

having this capability would keep actions, triggers, and even control pages and the applescript interface almost the same, but would add hugely to the ease of scripting and programming in indigo for insteon deployments with lots of linked devices (like my house).

although i've spent a lot of words trying to describe this, i think from a new user perspective, this is actually a great simplification for indigo overall. as a new insteon user, it takes a long time to understand what is possible and why (currently) what happens if you push the ON button on an insteon switch is in general very different from what happens when you send the switch an ON command from indigo.

thoughts, anyone?

I too use KeypadLinc LEDs to control and indicate the status of remote devices. The simplest solution I have found to keep my LEDs in synch is to create an Indigo scene for each KeypadLinc LED. Then, by simply issuing a scene ON or scene OFF command, you can control the LED. If you also have virtual 3-way switches, then include them in the scene.

Next, create two trigger actions that monitor the status of the MAIN LOAD switch (not the LED or virtual 3-way switches). One trigger action looks for the status to be changed to ON, then it issues the LED scene ON command. The other trigger action looks for the status to be changed to OFF, and it issues the LED scene OFF command. That's it.

By doing it this way, Indigo control pages can be used to turn switches on or off in the normal manner, then the trigger action will take care of synchronizing your status LEDs and virtual 3-way switches.

I forgot to mention that things are simple for KeypadLinc LEDs, but a bit more complicated for virtual 3-way dimmers. For dimmers, trigger actions that are tripped by a change in the state of the main load should run a script which reads the brightness of the main load, then sets the brightness of the secondary switch to match.

madscientist, when you say "create an indigo scene for each keypadlinc LED", i assume you are referring to creating an insteon group by using the new beta link management features in indigo, is this correct?

in that case, yes, i am aware that you can achieve control of the keypadlinc led's. in some cases this is a workable solution, but in some other cases, it's clumsy/awkward to do, for example, when creating control pages to control a device. if i want to make an item on a control page such that one button will toggle the state of multiple insteon devices, i can do this easily by simply making the click action for the item be a compound action with multiple toggle commands for multiple modules.

however, if i want to make a control for an insteon module (the load) and one or more keypadlinc buttons (sync'd stateful controllers), i can't do this (afaik) because there is currently no way to toggle keypadlinc buttons - they can only be turned on or off.

in thinking about how to do what i wanted to do it dawned on me that indigo already had essentially all the information that it needed to do the "smart" thing. currently, to do what i want to do requires multiple triggers (separate ones for ON and OFF, which itself is a pain) and it requires me to do a lot of scripting where the scripts essentially redundantly encode the state of the links between my insteon devices. in other words, i have to enter information twice - first to set up links, then to set up a bunch of scripts to make the devices work in indigo the way they do in the real world when humans use them. this seems way overly complex to me.

dduf619 wrote:

i assume you are referring to creating an insteon group by using the new beta link management features in indigo, is this correct?

Yes, that's correct.

It's actually not very complex at all. All you have to do is create two trigger actions for each main load. It takes a just minute, and it works great.

You need not make any changes to your existing control pages. They will continue to work, and the trigger actions will keep the LEDs in sync with the main load by switching the LEDs on or off about a second after the device status changes.

madscientist, i hear what you are saying. i agree that this can be done with enough work. but my post was not about what was possible but more about things taking more complexity to implement AND about violations of basic design principles such as requiring me to enter essentially the same information through multiple means in multiple places in the system.

to summarize some specific objections:

- using your approach, i need to take all of the buttons on keypads and all the "slave" switches" throughout my house that are linked to other modules and i need to encode this information in a bunch of trigger actions. i also need to set up the links between the devices. my point is that when the links are setup, indigo now (with newest beta versions) ALREADY KNOWS how to "do the right thing". it seems to me that this in itself is a very strong argument for a new feature in indigo.

- having to enter two different triggers for ON and OFF to do state tracking is a pain. this could be fixed in indigo by extending the list of possible actions for a trigger driven off of a state change so that the action can set the state of another device to the state of the device which triggered the action.

this could be an extra menu choice in the "Action" menu called "Mirror state of trigger device" or something. alternatively, you could use the "set brightness" command and instead of specifying a fixed level there could be another choice called "brightness of triggering device" or something. obviously, this is just two different possible UI's for essentially the same new indigo feature. you could instead use a single "state changed" trigger and an applescript action that says "set the brightness of device 'a' to the brightness of device 'b' " or whatever. at least this cuts the number of triggers needed in half.

you could also create a single "master" trigger that runs whenever any insteon device changes state - it would look up the changed device in a table to see what other devices need to be updated. at least this would centralize all the messiness in one place. offhand, i don't know how in applescript to access the device that trigger a "state changed" event, but i suspect that there is a way to do this via some global property or something.

[matt (support)]

I think this is a very interesting idea. I agree that keeping device states in-sync is a useful feature, and currently is cumbersome to accomplish. I need to think about which approach is best for this solution, especially given the complexity of how KeypadLinc buttons require the use of PowerLinc groups (I sure wish they didn't!).

Thanks for the comments and discussion on possible solutions.

Regards,
Matt

I think this is a very interesting idea. I agree that keeping device states in-sync is a useful feature, and currently is cumbersome to accomplish. I need to think about which approach is best for this solution, especially given the complexity of how KeypadLinc buttons require the use of PowerLinc groups (I sure wish they didn't!).

Wow, again! I just posted about this in another thread.

http://www.perceptiveautomation.com/php ... 9&start=60

Seems like my kludgy solution there, or some variation of it, is the norm.

This issue is virtually rampant throughout your forums, Matt. I know your eventual solution will be as elegant as the rest of Indigo!

But please finish suport for my new [pre-ordered] thermostat first. My feet are freezing right now but I don't want to get up to turn on the heat!!

[matt (support)]

This is a great topic and discussion.

First, I want to bring this topic back to this thread instead of continuing it on the Link Management thread. That thread is already long and covers several different topics, so let's discuss stateful syncing here.

Dduff617 is correct on how Indigo updates and maintains the states of responder devices. Indigo has always tracked the internal links (at least in most devices) and uses an internal link database to calculate how device states should adjust based on controller group messages. So when KeypadLinc button #3 group message is sent out, Indigo looks up in its internal link table and knows that SwitchLinc B responds to that message by going to 75% and thus adjusts its internal brightness state for device. I'm not sure if any other applications do this. I think some periodically poll all the modules to get their current states, but Indigo tries to update everything in real time based on its understanding of all the INSTEON links in the network. The new Link Management feature allows for editing of these links as well as creating new links.

I like the UI simplicity of DDuff617's solution ("simulate local control" checkbox in the Actions panel), but Mark is correct that this won't cover all the cases (ex: ApplianceLinc is only a responder and thus won't have a controlling link back to a KeypadLinc LED button).

From an implementation standpoint I'm concerned about trying to maintain the KeypadLinc LED buttons as a state. Because there are no direct INSTEON commands to control the LED button states, PowerLinc link groups have to be used. I'm concerned about the usability and maintainability of having Indigo automatically create such links for a few reasons. I think it can probably be done at some level, but I'm not sure how much it should be abstracted. For example, if Indigo automatically creates those links, then does it also delete them if they are no longer needed? What happens after a re-sync and Indigo notices that links, and thus the logic of the stateful controlling map, has changed? Indigo might have to immediately re-sync again. I'm sure there are other issues as well, but the point is that this level of abstraction isn't going to be simple.

If we were to punt and just use direct commands to maintain stateful syncing, then it becomes much easier. But unfortunately, this leaves out the most common/needed case, which is tracking the state of respondrs in the KeypadLinc buttons. So only doing the feature at this level (direct messages only), isn't going to be much of a solution except for 3-way SwitchLinc configurations.

Note that this won't be making it into 3.0, however. There is a certain thermostat adapter that is needing Indigo support.

Let's keep the discussion going, in particular about how to present to the user the option to sync states across devices. Thanks again for the great feedback and discussion on this.

Regards,
Matt

I like the UI simplicity of DDuff617's solution ("simulate local control" checkbox in the Actions panel), but Mark is correct that this won't cover all the cases (ex: ApplianceLinc is only a responder and thus won't have a controlling link back to a KeypadLinc LED button).

Let's keep the discussion going, in particular about how to present to the user the option to sync states across devices.

OK, if you don't like the option in the Manage INSTEON Device Links dialog, how about this:



Or "Mimic:" or "Match:" or "Mirror the On Level to/of:" or something like that.

The Mirror popup would offer whatever devices Indigo was able to mirror the state of. For 4-way setups (or more), you'd just set the Mirror option of all the slaves to the same master. I realize this is more complicated for the user than dduff617's idea to just let Indigo manage the responders, but as Matt and I pointed out, there are scenario's where the user will want to mirror devices that don't have the INSTEON controller/responder relationship necessary. Maybe for KeypacLincs the popup would list each button?

I still think the option should be available for devices though, instead of actions. And here's just one reason: if a user puts a simple button on a Control page that calls the Popup UI Controls to control his hallway light with a 3-way SwitchLinc setup, what action would be involved for that? See? He'd want the master/slave syncing, but there'd be no action group to set the option in. Or did I misunderstand dduff617's proposal?

Matt- you might start out with just SwitchLinc pairs and just do direct commands at first, adding support for other devices, like KeypadLincs, when you figure out the best way to do that. And since you'd be using direct commands, couldn't you include ApplianceLincs mirroring ApplianceLincs, and other "pairs" that Insteon linking doesn't directly support?

The Mirror popup would offer whatever devices Indigo was able to mirror the state of. For 4-way setups (or more), you'd just set the Mirror option of all the slaves to the same master. I realize this is more complicated for the user than dduff617's idea to just let Indigo manage the responders, but as Matt and I pointed out, there are scenario's where the user will want to mirror devices that don't have the INSTEON controller/responder relationship necessary. Maybe for KeypacLincs the popup would list each button?

I still think the option should be available for devices though, instead of actions. And here's just one reason: if a user puts a simple button on a Control page that calls the Popup UI Controls to control his hallway light with a 3-way SwitchLinc setup, what action would be involved for that? See? He'd want the master/slave syncing, but there'd be no action group to set the option in. Or did I misunderstand dduff617's proposal?

Matt- you might start out with just SwitchLinc pairs and just do direct commands at first, adding support for other devices, like KeypadLincs, when you figure out the best way to do that. And since you'd be using direct commands, couldn't you include ApplianceLincs mirroring ApplianceLincs, and other "pairs" that Insteon linking doesn't directly support?

i can definitely see some good points in this proposal. the "simulate local control" approach leaves some problems for indigo that are hard to solve (perhaps too hard) and that this method puts a bit more control back in the users' hands.

so in this view, the "mirror" setting is on the "follower" device and it specifies another device that it should follow.

a few issues:

1. keypadlinc button as follower: setup

presumably in order to setup a kpl button as follower, there would need to be some richer interface for follower devices that have more than one group id associated with them (like keypadlinc's) so that you could set separate "follow" attributes for each group id associated with a device, e.g., for each button. in other words, this won't be as simple (for keypadlincs) as a single follow attribute for the entire keypadlinc device.

2. keypadlinc button as follower: indigo control

it seems that making a keypadlinc button be a follower would necessitate dedicating a group to it, since there's no way for indigo to control a keypadlinc button with a "direct" insteon command. would you expect indigo to automatically allocate a groupID to any device/group which is set up to be a follower? this adds complexity, as matt commented earlier, and creates issues of what happens if you turn off the follow attribute - does the group automatically get deleted? these are the same issues that arise with "simulate local control".

alternatively, you could avoid having indigo handle this by making it possible to set the follow attribute for either a device or a group. so to make a keypadlinc button be a follower, the user would create the group containing the button(s) first and then would set the group's follow attribute. that probably makes more sense. on the plus side, this side-steps having to build a lot of complexity into indigo. the downside, is that it creates extra configuration work for the user. if you went this route, you'd need to modify the UI for setting up groups with the same (optional) pulldown menu for selecting a device/groupID to follow.

3. following a keypadlinc button: setup

presumably the "list of devices" to follow would need to include all controllable items in the system, so all the simple devices plus the module/groupId pairs for all the multi-group modules (like keypadlincs).

4. triggering of actions via follow relationships

would the "follow" functionality to apply only for indigo-initiated changes or to all state changes? presumably, insteon links can be used (entered either externally or internally) to achieve the desired linking for actions initiated outside the system (i.e., button presses). so you could make the follow-related actions apply only to indigo changes. also, the normal trigger action interface would be available if you did want to add linking behavior for external signals.

5. transitivity of the follow relationship

if a follows b and b follows c, and you turn on c, should a get turned on? or do follow-links behave link regular insteon links? in insteon, you can (and do quite often) create cross-linked devices. would symmetric cross-linking be allowed for follow links - i.e. a follows b and b follows a? that would make it so that you could pick any member of a group of linked devices to control (e.g., "hall light @ south end" or "hall light @ north end") and the result would be the same. if not, then you always need to know which is the "master" in the relationship when you are programming indigo to control your devices.

6. bidirectional query ui

it would be good to be able to see for a given device (or more generally, for a given device/groupID) which other device/groupID's are following it. this would be useful for troubleshooting, maintenance, etc.

so in this view, the "mirror" setting is on the "follower" device and it specifies another device that it should follow.

No, not necessarily. That's why I called it "Mirror:" or even ""Mirror the On Level to/of:". Note the "to/of" (though, that needs some wordsmithing). You're still thinking in terms of responders (hence, your "followers" term), where I'm thinking more like "twins." I'd like to pull away from the master/slave analogy, and think in terms other than those of Insteon's. In my model, once the relationship is established, then whatever one device does (either device), it's mate(s) mimic(s), no matter how or from where the request originates.

As you point out, and I agree, my interface idea is weak in that it doesn't allow for the setup, or review, of a greater-than-two device setup. But I was trying there to make my case for the option to be at the device level.

Rather than a popup (which is all I had time to draw), what is really needed is some sort of list where the user can add and delete mirrored devices. In fact, once you add device B to device A's list, device B's list would automatically show device A. That's why I had first pictured this happening in the Manage Insteon Device Links interface (MIDL), because that model is very close to what is needed. Perhaps instead of using the existing Link to Responder Device list or Link to Controller Device list in the MIDL dialog, there could be a third list that deals with these Mirror relationships. And just as the existing two lists show and hide, based on the capabilities of the device being edited, so would the Mirror list. Which is close to what I had pictured in the first place, remember? Now that might make the MIDL interface unruly, and that's why I had first proposed to just add the mirror attribute in a new column of either of the two existing lists. Either way, it still makes sense to me to use the MIDL interface, after all, it does deal with Links, and for new users that would be a logical place to look for the option.

1. and 2. keypadLincs
And of course, the freakin' KeypadLinc throws a wrench in the works, both in inplementation and UI. I'll have to think about that for a while...

3. setup
Yes, the user would be able to select any and all devices appropriate to mirror to, including components of multi-group modules, but would not be able to mate devices when that wouldn't make sense.

4. triggering
Any thing that causes a "twined" device to change will cause its siblings to change in the exact same way.

5. transitivity
Forget about master/slave. No matter how many devices are "siblings" together, changing any one of them changes all of them identically. Now this brings up an issue of a device belonging to more than one "family." I think this would have to be disallowed (and I can't think of why you'd want to do this anyway). So any number of devices can be in a mirror family (hey, I like this new analogy), but no device can be in more than one family.

6. ui
Addressed above.

Something like this:



Not sure about the bottom section, I didn't think that all the way through. But I think there may be a solution to part of the KeypacLinc dilemma in there somewhere!

[Addresses blocked to keep Matt from hacking my system and making me pay for causing him all this grief!!]

I just ordered a slew of switches and dimmers for my house. There are several places where 3 switches will control a single light. I would love the idea for those to be 'permanently' linked to each other as far as status goes. However, I'd love to have the ability to still control them so that the buttons can do different things.

Eg, turning off the lights at the bottom of the stairs also turns off the lights in the upper hallway, but from the top of the stairs it should only turn off the stairs.

O.K, now that I've got things set up, I've learned a lot more.

I am able to set the switches to control each other's status. One one switch I have a load. On the 2nd slave switch, I set that switch to control the main load switch. On the main load switch, I set it to control the slave switch. In one room, I have 4 switches and I paired them all up so any control switch controls all of the indicator lights. Beautiful.

The difficulty comes when I try to control the master from Indigo. It controls the light just dandy, but doesn't keep the other lights in sync. Sending a command to any of the slaves only controls the indicator as there is no load on them.

What would be super cool is the ability for Indigo to duplicate the commands sent to any master switch to be sent to all of the slaves. It can be done manually, but this should be automatic.

Now, any command received from the switch shouldn't be mirrored, at least by default, as different actions may need to be programmed depending on which switch you operate.

Very glad that this thread was started as the set up I'm planning includes multiple KPLs (each connected to a single load) that will all "overlap" in terms of groups of other loads (linked to SLs and KPLs) that are assigned to each button. I'll be setting up virtual 4 and 5 way switches in this way. A streamlined way of controlling these and keeping the KPL LEDs mindful of their "states" would be extremely useful.