From what I understand, when you place them in a new "territory" they have no idea whether they have been placed in someone else's turf or in a turf all their own. Therefore they require some time to assure themselves that they are the only cham living in that territory, and that some bully is not going to come along and beat them up.
When you enter the room or approach, at first all they see is the movement (they don't wait to determine whether you are friend or foe), and so they do the intelligent thing, they run and hide. Just in case. Even a friendly, personable, cham might do this in an unfamiliar enclosure. He won't wait to see if he recognizes you, he'll just quickly run and hide. It's actually the smart thing to do, looking at it from his perspective.
The fact that he is still eating, etc., tells you that he is just being cautious, not really totally strung out. After a period of time, he will realize it is his territory and make it his own. Then he will regain some self confidence and level out.
If he lets you know he's not in the mood for company, and you respond by backing off, this will bolster his self confidence, which is important for these male veileds. It gives them a sense that they have some control, which is a comfort to them. Of course, there are times when we have to feed them, etc., and can't just back away. But, you know what I mean.