Large Zigbee Network Experiences

Curious why the switch from z2m to zha?

My coordinators have all been EZSP firmware. With Z2M, when getting to about 70 devices, I had to quickly power the device off right after joining to get it to stay on the network. That has completely gone away with ZHA. I prefer the current Z2M capabilities with binding/groups/OTA/MQTT information etc…but at this size and complexity, the joining process had to be closer to flawless. With EZSP, ZHA was much better for me. I have worked through the binding/grouping “features” of ZHA and they are very early. The OTA was a nightmare, as I did not know they had stopped it from working because of Tuya devices and spent countless hours trying to fix something that I could not without a very specific YAML entry. Of course, there is a PR for an almost complete rework of the OTA for Zigpy. I would expect that to drop into HA shortly as well and might resolve a bunch of OTA issues. You can even use the JSON from Z2M to feed the update process.

This is almost impossible to zoom in on and use, but it looks “cool”.

2 Likes

In case you’re not familiar with it, zha-toolkit does give you a bit more flexibility with the binding options, etc.

2 Likes

Thanks. I have used zha-toolkit before, but I had not thought about the group/binding options. That helps a lot. Thank you!

This looks like the most dense mesh I’ve ever seen.

Perhaps…I wonder what the largest operational residential Zigbee network size is. Should be over 300 devices by end of the weekend. Had to adjust my adaptive lighting routines and make multiples of the same settings and then have different time frames for adjustment checks. Like 567, 605, 633, 667, etc… so that chances are low that of multiple requests. Otherwise it would overload the network with broadcast messages changing 15 light groups with a single adaptive light setting.

Continue to be happy with the performance of the network. It takes a moment after a HA/ZHA restart for everything to be happy, but it is a LOT of devices to check in on. Just setup a Proxmox cluster and high availability for the VM to try to keep everything up and happy and be fault tolerant.

Still solid and have hit 312 devices. Should be starting the outside lighting soon, which will also be Hue lighting part of this Zigbee network. Things are still fast and joining actions are perfect. Construction workers tripped a circuit on Saturday that took down power to 1/3 of the house. Network adjusted perfectly and when the power came back on it, everything was perfectly happy.

1 Like

@cscott,

Could you elaborate a little more on you adaptive lighting setup?
Are you controlling lux and CCT or just CCT?
How are you setting up groups with the blue switches in ZHA?
Any groups with smart lights and multiple smart (blue 2-1) switches?
(Are you just making one group or did you make multiple groups with one switch in each group?)

Sorry for all the questions, you’re forging a trail (ok, a superhighway), that I’m going to be following albeit with only 2/3rds the number of devices.

Thanks so much,
Jonathan

I am a huge fan of Zigbee binding so that lights will still work if the Zigbee network is down for some reason…coordinator/HA/network/etc…

So I create Zigbee groups in ZHA and add the lights to them. Now I would not typically add the switch to the group, and it is not necessary to do that, but since I also control the lights via HA and HomeKit/Siri, I want the switch to reflect the status of the lights as they are used. I will use the latest example to likely cover all that you are looking for:

Install Adaptive Lighting (via HACS) - using this for brightness and temperature of the lights based upon sun position. They have taken up a request of mine to add in an offset for large networks to help prevent broadcasts overloading the network

In ZHA, go to the groups in the Configure page and create a new group
Add in the lights you want as part of the group
Add in the switch(es) you want as part of the group
(side note: if you are one switch to one light, I bind those directly without a group)
For this example, we have light_1, light_2, light_3, light_4 and switch_a, switch_b, switch_c
So four lights and with three entrances into the room all three switches are controls for the lights.
switch_a is the switch that powers the lights and is put in Smart-bulb mode
switch_b and switch_c just have constant hot and are configured in the 3way mode
Add all the lights and all the switches to the group.
Go to each individual switch and bind On/Off and LevelCtrl to the group
Also bind each switch to the other two switches

This allows for any switch to control the other switches and lights and always display the status. If I use the light switch, everything is bound and in sync. If I send a command via Adaptive Lighting, the brightness and on/off state is also reflected in the switch. If I use Siri to change the lights, that command goes to the ZHA group which changes the lights and switches at the same time.

In essence it is like I have a 4 way switch configured for the lights and all works perfectly. I am running this exact scenario in one of my rooms.

I have a few other tweaks that I am running in the ZIGPY configurations that I am just verifying before I detail out all the things. Devices being used, configurations, lessons learned etc…but I am feeling confident of this working for over 400 devices when I am done.

Good luck on your design!

Awesome write-up.
I’m doing “3-way, 4-way” the exact same way and have had zero issues. Huge fan of the group binding for all the reasons you listed, the lower amount of network traffic, and most importantly, it avoids any popcorning (downlights coming on over 5-10 seconds in a room with 18).

I have not delved into custom ZIGPYs, plan to in the future, but haven’t had a need yet.

Glad to hear your using HACS Community Adaptive Lighting and not something custom created that would be out of my reach :slight_smile: .

Thanks for taking the time to respond. You need to set up a “buy me a coffee, or buy me a beer” link for sharing your wisdom!
-JD

When I am done, I will consider the Buy me a Coffee link as part of the final solution message. I was not an Adaptive Lighting user until last week. Before that I had a custom Node-Red function driving MQTT commands for Z2M and some other custom Node-Red features. All into HomeKit and was almost completely out of Home Assistant. Then moved to ZHA for the EZSP chip support.

Now fully embracing HA/ZHA/HACS, but still putting it into HomeKit. Also have some NSPanels that I plan to add into the house, and had the electrician setup locations for them. Going to use custom firmware on those and make them completely local. Perhaps https://nspanelmanager.com/ but there are other options as well.

One thing I am working on still. Every once in awhile the initial color change for a group misses a couple of lights. Trying to find the sweet spot for Adaptive Lighting on this one.

For those of you that have been following this journey to create an extremely large Zigbee network, I wanted to share some of the key details with you on how this has been possible.

I want to thank @bgreet for introducing me to @tube and the TubesZB product line. After discussion, I convinced Tube to make me a MGM24 based coordinator for this experiment and he sent me two of them based upon the 7.4.0.0 firmware for EZSP. Moving to ZHA, and these devices have been stellar for my network and this project. These devices have just gotten into stock at the tubeszb.com website and I highly recommend.

EFR32 - MGM24 POE

In addition to that coordinator, I have a tweak to the zigpy configuration options in ZHA that can be made in configuration.yaml:

This should allow for the device to store all the neede source routes for the network and this new device has more memory than the MGM21 devices allowing for more freedom. In addition to this setting, I tend to disable entities of Zigbee items that I am not using, so disable LQI, RSSI, and the power monitoring items of the Inovelli switches. It is data that is not important to my use case and I am doing my best to limit traffic to the devices.

I love adaptive lighting, but you still need to be careful about broadcast messages and overloading the network. The adaptive lighting group should have an update to help make sure to not overload Zigbee networks, but you can do some manual work in how you group lights to overcome it currently.

I will add additional small details as we finish up over the next few weeks and hope to add a final visualization with over 400 devices on the Zigbee mesh. But I will say, if you get the MGM24 coordinator and some good devices, you should be great to 250 without any tweaking. Happy to provide my feedback/suggestions as time permits if you want to share here at all.

If you have found this post valuable, and wanted to Buy Me a Coffee, that would be stellar, but I am just happy to share my experiences with a community that looks to grow capabilities and the love of proper home automation and control.

8 Likes

One more setting that I find helping reduce Zigbee traffic is to turn off polling entities for updates:

Oh damn, I just received my CC2652 P7-based Tube ZB yesterday. I hadn’t realized a potentially better model was just around the corner.

Though I’m probably topping out at fewer than 100 devices for now, so maybe I wouldn’t notice the update. The big difference is the size of the routing table? (and it looks like the new one doesn’t yet fully support Z2M, so maybe it wouldn’t be right for me regardless)

Z2M is not really ready for EZSP IMHO. When I hit 70ish devices the pairing process with EZSP adapters was rough and I had to cut power to them to keep them from unjoining and rejoining in a loop. For your size, I think you are all set and would not change it. When you think about getting into something large, then consider some of the newer EZSP modules and maybe Z2M will be there by then.

Update for April:

As Z2M has built support for the latest firmware updates for EZSP devices, via the new “ember” adapter setting, I have migrated the entire network over to Z2M.

Why?
I was having issues when HA and ZHA were down/offline and device bindings still working properly. This is a no go for me as if the computers are down the lights still needed to turn on and off. So I moved to Z2M and device binding is working great again!

So first, I have been able to run a 300+ device network in both ZHA and Z2M with the TubesZB MGM24 adapter. Second, both networks are very stable and work well. Third, source routing appeared to be great in ZHA but bindings broke when the system was down so I moved to Z2M. In Z2M, the bindings are perfect, but I have one issue that it can only maintain about 254 routes stored and must ask the network if an older route has fallen out of device memory. This has been fine except for one situation. When I bind a single swtich to a single light that sometimes breaks. Working on a solution with some of the devs for Z2M and hope that fixes it.

If it does not fix it, I will move to two adapters to maintain ALL routes in the coordinator memory. So if you are under 250ish devices you can easily use one adapter and it seems perfect. Above that and you should probably split to two adapters for now.

Next update as things change.

2 Likes

Is there a theory or reason the bindings were not working in ZHA when HA was down? If all the configurations live on the coordinator, I’m having trouble wrapping my head around why this would occur.

Great question. I am thinking it was because of the source routing configuration I was running on ZHA perhaps. It was very strange at the least, but again it might be because the network is so large and source routing made it flow differently than what we expect for bindings.