Lag / Slow Switch or Bulb? We Did A Speed Test - SmartThings vs Hubitat vs IQ Panel 2

Digital Lounge
#1

This pains me to post this as I’m a SmartThings user, but dang, we’ve been getting flooded with support tickets asking why our devices are slow to activate in SmartThings where, “my GE switch works just fine”.

I also have been experiencing the same lag, failure to update status, wrong status, etc in my own setup and so we decided to put some hub/gateway’s to the test to see if it was really a device issue or if it was something to do with the hub/gateway.

In this example, we used the following Hub/Gateway’s:

  • SmartThings v3
  • Hubitat (the OG one)
  • IQ Panel 2 (Alarm.com)

The products used were:

  • Ilumin RGBW A19 Bulbs (Z-Wave)
  • Red Series Dimmer Switch (Z-Wave)

Video:

Results:
Hubitat & IQ Panel turned on/off immediately (slight edge went to Hubitat when the app was used for IQ Panel 2) whereas SmartThings took 8 seconds to turn on the bulbs and 6 seconds to turn off the bulbs. Same story for the switches, except SmartThings ended up throwing a network error and timed out.

Why Did You Do This?
The purpose here isn’t to bash one hub or the other, it’s just meant to stick up for us a bit. We’re not in the blame-game business, but it’s hard sometimes to portray the message of, “your switch is only as smart as your hub” without coming off as jerks, so we wanted to create a video to show what we really want to say.

FAQ’s:
Q: My GE switches don’t do this on ST – why do yours?
A: Oftentimes the GE switches use ST’s built in device handlers, which run locally and are limited to basic functionality. If you want the advanced functionality of the Inovelli switches, you, unfortunately have to use the device handler to, “expose” them. Third party device handlers run in the cloud.

Happy to answer any other questions about this. Again, this pains me as I’ve been a long time ST user and I know the engineers over there who are working non-stop to improve the platform and I love those guys… but we also have to stick up for ourselves as we also strive for excellence and to create the best products on the market.

11 Likes
Black series dimmer not holding status
#2

Next, a Home Assistant test with the new Zwave integration :wink:

1 Like
#3

Lol, I’d love that actually – just bought a couple RPi 4’s, but I’m too afraid to figure out how to use them :slight_smile:

#4

I will see if I can get a video of the switch in action with HA tonight. Don’t have the bulbs but some are on their way and should be here Saturday.

1 Like
#5

Done - Home Assistant with the current standard Inovelli z-wave install method (not the new Zwave integration because I don’t have the guts to switch over yet).

Pretty fast! The only lag I have seen has been due to LED bulbs not actually turning on until 20% power is applied which is overcome by properly setting the min and max levels on the switch to match the bulbs.

Video:

3 Likes
#6

tired of smartthings!

why not an inovelli hub for the (not-so-distant) future :stuck_out_tongue:

1 Like
#7

That was fast too! I love smartthings. It was my first girlfriend and it’s hard to let go but the time has come.

4 Likes
#8

Wise choice.

2 Likes
#9

Damn, mine was Wink and moved on to ST. Apparently, I just don’t know how to pick them lol

6 Likes
#10

Third times the charm! :wink:

1 Like
#11

I have been using ST for several years and this delay seems to have become much worse recently. There was always some delay but usually no more than a couple seconds worst case and I guess I just accepted that due to being processed in the cloud not local. But it does seem to be much slower lately (more than a couple seconds) and I also get the “network or server error” message a lot more often now and I don’t recall ever seeing that before a few months ago

So it seems to me like something has changed over at the ST Cloud processing center.

5 Likes
#12

I feel the same way, but not sure which way to go. Are you moving to HA or Hubitat or other?

#13

I haven’t decided yet but i’ll probably use Hubitat. I’ve never used Home Assistant before and I don’t know if I can get my wife on board with that learning curve. ha ha.

1 Like
#14

Hi All. I’m going to preface this post with admitting i’m 100% a SmartThings fanboi. Hell, I run a twitter account about it. With that said, I feel like i’m pretty realistic with my expectations of SmartThings.

For this speed test, I would NOT expect the SmartThings app control to be faster than the Hubitat app control. The Hubitat communication path, assuming you’re on your local LAN, is Mobile App -> Hub -> Device. The SmartThings communication path, even when on local LAN, is Mobile App -> ISP-> ST Cloud -> ISP -> Hub -> Device. (i’m not including the path back to the app to update status here). So those 3 extra steps with your ISP and the ST cloud are inherently going to add some latency.

My experience with the Inovelli dimmer on SmartThings is much different than what’s shown, though. Here’s a video I took. Some background on my ST environment, 190+ devices (165 hub connected), using the Inovelli device handler, the Inovelli switch is connecting directly to the hub

So I think there is something wrong with the SmartThings environment Inovelli is doing this test with. What exactly? Well…hard to say. Obviously there’s more links in the chain that need trouble shooting than with Hubitat. ISP, cloud, etc. SmartThings also doesn’t offer much in the way of z-wave troubleshooting other than z-wave repairs and looking at the route in the Groovy IDE. I’m sure Inovelli has some direct contacts with SmartThings devs they can use to investigate, though.

#15

@Eric_Inovelli @anon14959390 Happy to help you sort through this. I have six red dimmers and two of the Ilumin bulbs on the V3 here at home.

I don’t see lag like this on my setup even with the custom DTH. Typically seeing times of 400ms- 1 second worst case.

Questions

Are you using the switch has a hardwired controller for the bulb or button controller? It’s much faster in my case (even with cloud processing) if he red switch is controlling a mains switch.

#16

This is definitely encouraging! Love seeing it work as intended :slight_smile:

Correct, we didn’t either for this exact reason – but what was perplexing to us was how fast the IQ Panel 2 was which also runs on the cloud.

That’s lightning quick brotha – idk how you did it, but glad to see that! Would love to know more about your optimization techniques and/or best practices. I think it would be super helpful for the community too – heck we’ll even plug your twitter account if you want!

Yeah, not sure either. Our ISP was working just fine for IQ Panel 2, so not sure if it was that. As for Z-Wave repairs/errors, there was only one device connected to the hub, so I’m not sure if there was a mesh issue or not.

Awesome man, thanks so much for the support and welcome to the community. If it’s the Jody from ST forum, we’re honored to have you – thanks so much for the contributions to the platform over the years – as I mentioned in the initial post, I know you guys are working hard and this post wasn’t to bash anyone, but rather open the discussion to help fix lag problems for people.

That’s great! Hopefully you guys can help a bit. Just to paint a picture, we typically see tickets daily about lag time and usually they’ve tried ST’s customer service and the blame gets sent back to us, so it will be nice to have some help from you and whomever to help. Maybe we can create some sort of best practices guide in the ST sub-forum? Idk, just a thought.

Great question – so the initial speed test was just with detached bulbs on their own inside lamps. They were plugged in with an extension cord.

The switches were hardwired into a box that we built for testing. It was wired in a 3-Way to an auxiliary switch (GE) and the bulb was unscrewed because if it lit up, you wouldn’t really be able to tell the LED bar was turning on/off.

Hope this helps!

Again, thanks guys for coming over here and posting the positives with ST. As mentioned, ST was actually the place where Eric M and I met (forum) and now three of us have it powering our own homes.

1 Like
#17

Yep. Jody from ST.

2 Likes
#18

@Eric_Inovelli I switched the DTH back to the latest one on my bulb (was using stock RGBW before) and now the bulb won’t even turn on

groovy.lang.MissingPropertyException: No such property: bulbMemory for class: script_dth_xxxxxxxx @line 147 (configure)

#19

Dimmer works instantly, but using the button fails.

#20

Awesome, welcome!

@EricM_Inovelli – do you know what this means?

EDIT: Also wanted to add that we are using the stock ST driver as the bulbs are WWST so we weren’t using a custom handler.

Button in the app or at the switch? Are you turning it on/off? The biggest problem we see (I see it all the time on my own as well) is that you press on/off and the app hangs (exactly what happens in the last part of the video). I usually don’t get a network error like we did at my house, but it definitely has issues with the spinning wheel and oftentimes does not report the correct status.

I’d say I have a solid mesh network (at least 4-5 Z-Wave devices in every room) and the hub is in the most centralized location. Internet speeds are fast and I have a fairly new router, so not sure.