Just today I performed a simple driver update by going to the Driver Code page then downloading the latest LZW31-SN official Inovelli driver. I had already been using that driver since first jumping onto the Inovelli bandwagon a few weeks ago. Now something very strange has happened. One of my dimmers (and conceivably all of my LZW31-SN dimmers) has vanished from all scenes, rules, groups, etc. Worse yet I cannot get it back. No amount of reboots nor cache empty nor even switching browsers gets me the device back. Worse than that if I open a rule or scene to edit it, the device is removed from that scene completely and I cannot get it back since it does not appear on the lists within the drop downs to add a device. This is corrupting rules I have spent weeks building and adjusting so I am in a bit of panic.
Ok this is a lot worse than I thought. ALL my LZW31 dimmers no longer are available to rule, scene, etc creation throughout Hubitat and editing any rule, scene, etc. drops all mention of those devices from existing scenes. This is bad. I’ve lost a large majority of my home’s programming in the last few moments that I’ve worked weeks refining. What can I do?
Are you certain you used the driver for the Red Dimmer and not he Black.
If so what is the date shown in Hubitat would appear and update what made earlier today and maybe that cause some issues. My hub is using a version from 3-27 and have not upgrade yet…
It is for sure. I used the GitHub import URL that already was present on the driver page from when I originally installed the Inovelli “official” drivers. So that much I know is correct. Dated 5/5 so the latest at the time of this post. It might be something from today’s update that caused this.
All I can say is that I’m in a world of hurt right now and the spouse isn’t going to be too pleased when it comes time to push the “sleep time” button in a few minutes. Not a great turn of events…
@Eric_Inovelli any suggestions? Perhaps something is up with this latest driver update? I updated from the version published to GitHub on 3/27 per the code (though my last update was imported on 4/25). As of this evening I see 5/5 and that’s when all the trouble started.
Great minds… I was trying exactly that when you were typing. So it seems that alone was insufficient to fix the problem. I had to restore to a previous hub backup that I made earlier today. I lost a few rule tweaks but I now can see the LZW31 devices again throughout Hubitat’s UI. That is more important than timings or LED brightness settings.
Something for sure is up. This cannot be normal behavior. I have little experience switching drivers but in the few times I have done this in the past nothing like this occurred. It’s not even a “big” switch. Switching from the Hubitat-included Inovelli driver to the Inovelli official driver in my first trip around this process caused no issues like I just saw.
My loss of devices was not caused by a firmware update but instead by a Hubitat driver code update. That is shocking but the unacceptable part was that the device loss was silent. No errors. No log entries. Nothing. Simply missing devices and only after/during editing rules or scenes in which those devices appear.
Going through an exclude-include cycle would force me to completely rebuild all my rules, dashboards, scenes, groups, etc. It could be “from memory” if I have no way to access the contents of the backups in a way that is readable. That’s a massive undertaking which I don’t have the time to do even now when I’m home during the pandemic shutdown.
I shudder to think how this would go if I were busy. Driver updates, firmware updates, etc — any of these sorts of things shouldn’t break most of the rules you’ve spent hours building or days/weeks/months tweaking. Besides the Hubitat side of this, I can say with certainty now that updating the Inovelli driver (again to be clear not firmware) caused this. I repeated the update steps again and it led to the same situation. Restoring to my known good backup from earlier today gets me going again but of course this is unsustainable going forward.
I really hope that Inovelli and the Hubitat community here can sort this out. I’m digging myself but I’m still in the early stages of learning.
I’m going to tag @EricM_Inovelli here as he’s more fluent with Hubitat and writes the drivers.
I did notice some quirks recently from the ticketing system with Hubitat users, so I’m not sure if it’s something with the driver or Hubitat in general, but it just happened recently (like within the last couple of days).
Not helpful with your current situation however but I always download a Hubitat backup right before and right after any significant undertaking or firmware backup. Settings/restore & backup / downloat
Also go to the Hubitat forum and tag chuck he can likely get your latest backup restored.
It sounds like the capabilites of the device were lost, but I can’t say I know how that happened. If you replaced the Red Series Driver with the Black Series you would lose the Button capability and the Energy capabilities, but you said that didn’t happen.
When you do the import can you copy and paste the code into pastebin or something? When I do an import everything seems fine.
Edit: Also, when you import it and save it which capabilities are shown on the drivers page?
First thanks to all in this community who are helping! As much as this is a panic it is satisfying to see this thread trying to solve the problem.
@JohnRob — I was able to restore from a fairly recent backup. I am practiced at it much in the way you suggest. In this particular moment confidence was high (that has a name… overconfidence ) At least I am back to where I was and can continue editing rules, etc.
I performed the test as you requested. I can confirm that the problem repeated exactly the same way as it did last night. I repeated the same steps twice to prove it to myself. No amount of copy-paste code rollback works. I have to restore from backup to get my devices back. Screenshots below…
Like I described, after the update, I can click the device to open its device page and directly control it in any way I want. But it never will populate within the scenes, apps, dashboard, groups, rules… lists no matter what I do. If I edit a scene or rule with any device that used this driver the device is silently removed from the scene, rule, etc. without error and without any evidence the device used to be referenced (and I did confirm by restoring from backup).
Edit: Missed that you asked for OS/Browser…
MacOS 10.15.4, Safari 13.1, Hubitat 2.2.0.128 on Hardware C-4
So strange, things don’t look out of the ordinary. You even said that rolling back to a previous version of the driver didn’t fix it right?
It sounds exactly like the capabilities are getting lost, but they are showing up on your driver screen. Like, if I remove a couple capabilities from the driver and go into an app that has the device selected, the device disappears from the selection. I also can no longer select the device of course.
Strange is the word for it! I’m glad I’m not the only one surprised by this. I feel much better knowing that a restore from backup is a reliable means restore functionality. A driver rollback comes as a result of the restore from backup. A direct driver rollback did not resolve the issue. My hunch is that once the rule/dashboard/other app members are confused and spoiled the damage is done. So even if a rollback would have worked what’s done is done and irrevocably ruined. Something is up with this as the failure occurs 100% of the time. Were you able to repeat my steps to create the same failure mode?
What else can we do? Is it possible that this is a Hubitat bug/issue? Do you have a direct connect to the Hubitat team to investigate this issue on their end? Can you try repeating my steps with a different device type and driver? (an LZW30 for example, which I do not have)
So I did try it on my other hub. I hit the import button on the LZW31-SN driver, clicked ok, and clicked save. I then went back into my some of my rules and everything seemed fine upon opening them up.
Then, as a test, I went to the driver and deleted a couple of the capabilities (button, energy, & power). I then went back to the rules and the devices with the modified driver were no longer there. The device selection box was red and it said to choose a device. If I clicked on that box I could no longer choose the device with the modified driver.
I wonder if there is some kind of database corruption where the driver is being saved. Just a guess because I have no idea how they manage data on their hub. If you create a new driver with the github code and switch a device over to it I wonder if it will do the same thing.
Switching drivers alone definitely shouldn’t cause this problem. Here’s my guess: if Hubitat detects a “bad” database at boot, it restores the last known-good copy. OK, that part is fact, not a guess, so the guess is that this might have happened to you. This is rare but more likely to happen if you rebooted/shut down by pulling power instead of using the Settings menu (or if you unexpectedly lost power and the same happened; some people keep their hub on a UPS to avoid this). If this does happen, you normally see a “notification” (a badge in the chat-looking bubble in the upper right) afterwards to let you know.
If you didn’t reboot, I suppose it is possible for corruption to have happened at any time or for you to be now experiencing the results of existing/previous corruption as Eric guesses above. Again, we don’t really have the tools to see what Hubitat is doing underneath the hood here, but this is consistent with what we’ve been told. A “soft reset” is usually a pretty safe option and might help if you have a “good” database backup you can restore from (don’t do a full reset unless you feel like starting all over and re-pairing all devices—a backup won’t help there–though it would certainly be one way to get rid of the problem too if it’s only software/database-related).
I see you just posted on the Hubitat forum and tagged Support, so they should see it and hopefully get you sorted out. If they don’t respond, you might want to reach out officially (support at hubitat.com) so it generates a ticket, perhaps pointing them to the forum posts for more information.
@BertABCD1234 indeed and as you said I did post on the Hubitat forum tagging @support. We’ll see what develops.
@EricM_Inovelli in the meantime while waiting for other new developments, I performed a different test. I edited the driver code by simply adding some “TEST” comment text. My goal was to determine if the issue is with my hub, something more universal to Hubitat, or something with this code. In the screenshot you’ll see the change I made is obvious.
I can confirm that this test does not cause the missing device problem. Importing the updated Inovelli driver (using the import button on the Driver Code page) from GitHub causes the same failure I described originally. Copy-pasting the driver code from GitHub also causes the same failure.
So it seems the driver code as posted to GitHub has some sort of issue? Or perhaps the problem is caused by a complete change to driver code instead of a small edit?
If the database or filesystem(not sure which they use to store drivers) is corrupt, then the amount of data modified could impact whether the corruption matters. In the filesystem case, if the block you modify is already correctly mapped and it is only updated, the problem might not present itself. However, if new blocks are created or the mappings have to change, that might actually cause an issue.
If you want to try and narrow it down more, you could create a new driver that is just a copy/paste of the old driver. Then change one of your switches to the new driver. If that doesn’t work it would definitely point toward corruption. If it does work, well that doesn’t really mean anything unfortunately.
@AnotherUser I doubt this is database corruption in any grand scale. However I did perform exactly the test you suggested. I had been wanting to do something like that for the last few days but just didn’t have the time.
So I renamed old driver “no updates” and added something to that effect at the top of the driver in the comments. Saving this caused no issues with any existing device/app nor did it prevent me from creating a group with any LZW31-SN devices present. That’s all to be expected and solid.
Next I created a new “blank” driver, copy-pasted the driver code from the raw GitHub link as of the last update, and saved it. I then changed the driver for one LZW31-SN device in my system to use the new driver and low and behold it did not work, the exact same sort of problems appeared per my original post.
Any device that I switched to use that updated driver would vanish from all places in all apps and corrupt existing apps in which it appeared. I still could control the device but I could not see it in any apps. If I swap back to the “no-updates” driver the device reappears in all the apps list.
So @EricM_Inovelli that driver update as of May 5th is a major problem for my Hub. What else can we do? It seems that all signs point to this driver. Perhaps the cause is something specific about it that I am using? Custom Commands/Actions perhaps? Notifications? I have no child devices at all so this should be fairly simple to determine.
@rpulivella , Sorry, I probably wasn’t clear in what I was suggesting.
Open old “working” driver
Select and copy all the code
Create a new blank driver
Paste the old code
Save the driver as “old code test driver” or something to designate it as the newly created driver
Switch a device to use the newly created “old code test driver”
Check if it the devices disappear
If you still see the same issue, then that highly points to a database/filesystem issue. If you don’t then it really does point to the new driver code instead.
) At least I am back to where I was and can continue editing rules, etc.
This is rare but more likely to happen if you rebooted/shut down by pulling power instead of using the Settings menu (or if you unexpectedly lost power and the same happened; some people keep their hub on a UPS to avoid this). If this does happen, you normally see a “notification” (a badge in the chat-looking bubble in the upper right) afterwards to let you know.