Drilling rivets is ok...

Sometimes its better to just knock them out with a good sharp chisel!