How do I keep some of my devices separate from voice control?

If you want to disable voice control for a device or devices so that they don't respond to voice commands, go to the Settings page for that particular device and enable Disable App / Voice Control.