How often do you handle him? When you handle him, do you "play with him" or simply let him hold onto your fingers?
One of my many mistakes as a new chameleon owner, was handling them. I had a 4 month old Sambava, and I took him out every few days just to hold and look at. When I did handle him, I would twist my hand just a little so I could feel his strength, and view his acrobatic abilities.
What I did, is a bad thing.