Personally I would only do it when the power is off. The reason - contact bounce on the switches. It's possible to get bounce when switching over. Regards the CPLD switching over between games based on the switch settings, I suspect from that angle it would be OK though.