What @fatherdoctor said. If you want to keep from zapping yourself, use a proximity tester like the one you have. But if you want to measure voltage, you need to use a meter.
Plus, it doesn’t matter if the load wire is connected or not. If a switch is switched to put out voltage, that terminal will be hot whether or not you have a conductor connected to it.
As to why it’s hot if you think it’s off, remember that it’s a dimmer. Just because it’s dimmed down doesn’t necessarily mean there won’t be voltage present. Dimmers work by clipping the AC waveform.
If you pull the airgap, do you still detect voltage?