Help, all Blue series switches stopped working w/Inovelli's Edge Driver but work ok with ST's ZIgbee driver

I’ve been replacing my Jasco Zigbee switches with Blue’s (2.08 and 2.15 firmware releases) and all was going great until roughly an hour ago. All of a sudden none, and I mean none of my Blue series switches (33 of them) are controllable via the app, voice, routines, nothing at all. All my other Zigbee devices like sensors, locks, and Jasco switches work great and perform super quick. It’s just all my Blue series switches that stopped working.

Any ideas why all of a sudden all these Blues stopped responding? Was there an Edge driver update? The driver version is 2023-08-17T22:28:06.820642294, so I doubt that could be it.

EDIT : hub reboot did nothing to bring the Blue switches back. The switches are not showing offline in the mobile app or advanced api site. This is super odd and I have never ever run across anything like this.

EDIT : It looks like a driver issue because when I switch over to ST’s Zigbee Switch driver the switches work again, but then if I go back to Inovelli’s edge driver it broken again. @EricM_Inovelli any ideas? I did submit a ticket but i didn’t write down the # cause I thought I would get an email.

EDIT : It’s repeatable. If I switch over to ST’s driver the Blue series switches start working again. I just switched over 10 and they now work using ST’s driver.

So when you use the app and attempt to turn on a light that is off, does the app indicate the light is on even though it isn’t, or does it indicate off even though you pressed the button to turn it on? I’m not presently where my Blues are, but they seem to be functioning fine in terms of the app working properly to turn on and off, and SharpTools reporting the on/offs properly as well. I just can’t tell if the bulbs are actually on, at present.

I’m using the 8/17 driver as well, so I believe it’s the current one.

The driver has been working fine, but it’s possible ST broke something that conflicts with the current driver.

Check your settings to make sure remote and local access haven’t been disabled.

Other than that, I’d let them sit for a while without much tinkering. Every once in a while the weirdness just seems to work itself out.

Hi @Bry , no what happens is that when attempting to turn on a switch the on/off indicator just spins and then I get the “a network error has occurred” message and then it’s back in an off state.

I’ll check those settings as you mentioned, but it’s too odd that all 36 switches just stopped working with Inovelli’s driver. I just switched over to ST’s driver for my most used switches hoping that something on the back end broke and will be magically fixed…

Yeah, it’s definitely systemic, at least for you. I just checked with someone that has quite a number of them on ST w/2.15, and they’re all working fine. If there is some sort of widespread issue, others should be chiming in. You could be on a different shard than everyone else, but what are the odds?

1 Like

Yikes, can you share a screenshot of one of the devices for me (instructions below)?

Can you follow those directions and let me know what it says by Manufacturer Code and Model?

Just for one, I want to see what it reports there.

1 Like

Yeah something is really weird.

I moved all switches to ST’s driver and got them working again. But…

I took a couple and just put them back on Inovelli’s driver, resent the settings and now those are working using that driver.

I only moved back two switches, so now I’m going to chill for a bit and have a beer or two and mess with this crap tomorrow.

yup, here you go:

Still on ST’s driver:

One I moved back to Inovelli’s:

Quick update before tackling this tomorrow.

I added a brand new Blue switch and it joined and configured perfectly fine.

I have no idea what was up earlier this evening, but is was a little bit scary to see that many switches just stop working with a particular driver. I’m betting something on ST’s end happened… Been there, done that with them over the years…

2 Likes

I have to think that there was a blip in the ST system. They may have targeted a certain function that our driver uses, but maybe others don’t. If it happens again it might be helpful to get some of the driver logs by using the smartthings command line tool.

Releases · SmartThingsCommunity/smartthings-cli (github.com)

This is the command that I use to connect to one of my hubs:

smartthings edge:drivers:logcat --hub-address 10.37.20.95

Oh I’m pretty sure it was. It’s the love-hate relationship I’ve had with ST since their kickstarter days.

Things this AM look great and I just have 4 more switches to move back to the Inovelli driver. I also use that CLI command quite often, but not this time around since I was more in a rush to stabilize things before my wife returned and gave me heck about ST again.

FYI @EricM_Inovelli and @Bry ,

The problem where all the switches stop responding that are using the Inovelli driver came back this morning as I was adding back the last 4 switches. I have a total of 35 Blue 2-1’s on my ST hub (v2) right now, 31 running the Inovelli driver, 4 running ST’s Zigbee Switch driver.

I switched one of the four over to the Inovelli driver, and that worked fine. That switch and all the other 31 switches on the Inovelli driver worked. I literally walked around to every one and verified.

I’m now running 32 switches on the Inovelli driver, and 3 running ST’s Zigbee Switch driver.

As soon as I switched over one of the now three remaining, ALL the Blue’s running the Inovelli driver (33) stopped responding when using the mobile app or the advanced API.

When I switched that one back to the ST driver, still NONE of the remaining Blue switches (32) worked. I had to revert another one of the switches back to the ST driver. As soon as I did that, the 31 switches running Inovelli’s drive started working again.

So, now I’m back to 31 switches running the Inovelli driver, 4 running ST’s Zigbee Switch driver.

I can repeat this problem over and over again. I have time today to do more debugging, so I’ll update this post. I did not spin up the CLI this morning during all this (I was on a Chromebook), but I will during my follow up debugging sessions on my Windows PC. I have a couple meetings to get through this morning, so as soon as those are over I’ll get to debugging this.

What version of the ST hub do you have? V2 or V3?

V2, and I just got done running the CLI while moving over a switch to the Inovelli driver. Holy cow tons and tons of messages, so much so I could not keep up. I’m looking through what I captured, but I did see warnings, but I’m also seeing what looks like a loop of some sorts. Still looking right now before my first meeting.

EDIT : FYI, 55 pages in a matter of a minute or two. I’m preparing to share a google doc with you.

Hi @Eric_Inovelli @EricM_Inovelli you should be seeing a PM with a Google Docs link with what I captured.

Thanks, I am looking over the logs.

Post what you find, I’m curious as well. I’m expecting an out of memory condition with the driver restarting over and over to try and recover.

1 Like