Problem upgrading Blue 2-1 firmware

I filed a support ticket for this since I hadn’t received a response from anyone here. But I’ll reply here with any updates in case it helps anyone else.

I am on HA 2024.1.

To turn on the OTA updater, I added a zha section to my configuration.yaml like so -

zha:
  zigpy_config:
    ota:
      otau_directory: /config/zigpy_ota
      inovelli_provider: false

I tried it without otau_directory and inovelli_provider: true, to pull updates directly from the internet first. That had the same effect. (Although it was only trying to pull down 2.08 since that seems to be the latest reported in that repo.)

I did find that manually triggering the update on a single switch seems to work, per the instructions here - ZHA Firmware Update Guide - #10 by jtbandes

However, the automatic attempts from HA continue to fail every time.

I’m wondering if ZHA is just trying to update too many at once, overwhelming the network? I did turn on debug logging and grabbed what I think are some relevant lines from one of the failures. Maybe someone who understands the protocol can decipher what is happening here -

2024-01-09 11:11:27.528 DEBUG (MainThread) [bellows.ezsp.protocol] Application frame received incomingMessageHandler: [<EmberIncomingMessageType.INCOMING_UNICAST: 0>, EmberApsFrame(profileId=260, clusterId=25, sourceEndpoint=1, destinationEndpoint=1, options=<EmberApsOption.APS_OPTION_RETRY: 64>, groupId=0, sequence=214), 255, -75, 0xd235, 255, 255, b'\x11T\x06\x95/\x12\x01\x01\x0f\x02\x02\x01']
2024-01-09 11:11:27.528 DEBUG (MainThread) [bellows.zigbee.application] Received incomingMessageHandler frame with [<EmberIncomingMessageType.INCOMING_UNICAST: 0>, EmberApsFrame(profileId=260, clusterId=25, sourceEndpoint=1, destinationEndpoint=1, options=<EmberApsOption.APS_OPTION_RETRY: 64>, groupId=0, sequence=214), 255, -75, 0xd235, 255, 255, b'\x11T\x06\x95/\x12\x01\x01\x0f\x02\x02\x01']
2024-01-09 11:11:27.528 DEBUG (MainThread) [zigpy.application] Received a packet: ZigbeePacket(timestamp=datetime.datetime(2024, 1, 9, 16, 11, 27, 528748, tzinfo=datetime.timezone.utc), src=AddrModeAddress(addr_mode=<AddrMode.NWK: 2>, address=0xD235), src_ep=1, dst=AddrModeAddress(addr_mode=<AddrMode.NWK: 2>, address=0x0000), dst_ep=1, source_route=None, extended_timeout=False, tsn=214, profile_id=260, cluster_id=25, data=Serialized[b'\x11T\x06\x95/\x12\x01\x01\x0f\x02\x02\x01'], tx_options=<TransmitOptions.NONE: 0>, radius=0, non_member_radius=0, lqi=255, rssi=-75)
2024-01-09 11:11:27.529 DEBUG (MainThread) [zigpy.zcl] [0xD235:1:0x0019] Received ZCL frame: b'\x11T\x06\x95/\x12\x01\x01\x0f\x02\x02\x01'
2024-01-09 11:11:27.529 DEBUG (MainThread) [zigpy.zcl] [0xD235:1:0x0019] Decoded ZCL frame header: ZCLHeader(frame_control=FrameControl(frame_type=<FrameType.CLUSTER_COMMAND: 1>, is_manufacturer_specific=0, direction=<Direction.Server_to_Client: 0>, disable_default_response=1, reserved=0, *is_cluster=True, *is_general=False), tsn=84, command_id=6, *direction=<Direction.Server_to_Client: 0>)
2024-01-09 11:11:27.530 DEBUG (MainThread) [zigpy.zcl] [0xD235:1:0x0019] Decoded ZCL frame: Ota:upgrade_end(status=<Status.ABORT: 149>, manufacturer_code=4655, image_type=257, file_version=16908815)
2024-01-09 11:11:27.530 DEBUG (MainThread) [zigpy.zcl] [0xD235:1:0x0019] Received command 0x06 (TSN 84): upgrade_end(status=<Status.ABORT: 149>, manufacturer_code=4655, image_type=257, file_version=16908815)
2024-01-09 11:11:27.531 DEBUG (MainThread) [zigpy.zcl] [0xD235:1:0x0019] OTA upgrade_end handler for 'Inovelli VZM31-SN': status=Status.ABORT, manufacturer_id=4655, image_type=257, file_version=16908815
2024-01-09 11:11:27.532 DEBUG (MainThread) [zigpy.zcl] [0xD235:1:0x0019] Sending reply header: ZCLHeader(frame_control=FrameControl(frame_type=<FrameType.CLUSTER_COMMAND: 1>, is_manufacturer_specific=False, direction=<Direction.Client_to_Server: 1>, disable_default_response=1, reserved=0, *is_cluster=True, *is_general=False), tsn=84, command_id=7, *direction=<Direction.Client_to_Server: 1>)
2024-01-09 11:11:27.532 DEBUG (MainThread) [zigpy.zcl] [0xD235:1:0x0019] Sending reply: upgrade_end_response(manufacturer_code=4655, image_type=257, file_version=16908815, current_time=0, upgrade_time=0)
2024-01-09 11:11:27.533 DEBUG (MainThread) [bellows.zigbee.application] Sending packet ZigbeePacket(timestamp=datetime.datetime(2024, 1, 9, 16, 11, 27, 533088, tzinfo=datetime.timezone.utc), src=AddrModeAddress(addr_mode=<AddrMode.NWK: 2>, address=0x0000), src_ep=1, dst=AddrModeAddress(addr_mode=<AddrMode.NWK: 2>, address=0xD235), dst_ep=1, source_route=None, extended_timeout=False, tsn=84, profile_id=260, cluster_id=25, data=Serialized[b'\x19T\x07/\x12\x01\x01\x0f\x02\x02\x01\x00\x00\x00\x00\x00\x00\x00\x00'], tx_options=<TransmitOptions.ACK: 1>, radius=0, non_member_radius=0, lqi=None, rssi=None)
2024-01-09 11:11:27.533 DEBUG (MainThread) [zigpy.application] Max concurrency (8) reached, delaying request (94 enqueued)