It may stress him out a little, but it's better than him breathing in paint fumes. I had a neighbor that forgot about the fumes, and painted a room with his cockatiel in it. The poor bird didn't survive, and he was absolutely heartbroken over it.
One more thing, although I'm sure you already know this, take him out of the cage before you move it. It'll be less scary to take him out, move his cage, and then put him back in.