"For round holes i use various stepping drill bits and for square holes i have had good results using a combination of a dremel tool and handfiles. That is of course when working with metal. I haven't really done much in plastic"
I do the exact same thing except the plastic. A small edge comes up since it's so hot I think, but it's nothing that can't be smoothly removed with a exacto knife. Of course it's not going to be exactly perfect, but that's why you make it a very very very little bit smaller than you need it and then use a hand file to clean it up. I use jewelry files because they're small and work really cleanly, as well as come in a variety of shapes.