Ehhhhh technically the relay isn’t in line. The relay is for the dumb switch traveler connection. Everything is going through the dimming mosfet and the sine wave is being clipped, so that is most likely causing the issue.
Not to say you don’t have a defective switch, but just stating the switch was not approved for motoring loads.
More information in: