Stress
Chameleons react to stressful situations differently by altering their colors and patterns they display. They also darken their colors to absorb heat. So I would say he gets nervous/stressed when you take him out of his enclosure. When you put him back in he darkens to reach an optimal temp.