I recommend only handling them when you need to. They are naturally a solitary animal so any contact with another living creature is stressful. Especially when that creature is as big as we are to them.
That being said, it is good to build trust between you and your cham to lessen the stress-fullness. When you get you cham, give it a few weeks to adjust to its new home and the sights and sounds of the environment. When you notice the cham not trying to hide or put distance between itself and you you can try to start hand feeding. Start by holding a cup with the feeders in it and hold it out for the cham to eat from. This takes time so be patient. When they no longer hesitate to eat from the cup in your hand try holding a feeder in your fingers or on your palm. If the cham doesnt take the bait after 10-15 minutes, use the cup and try again the next day. Eventually the cham will hopefully associate you with food and let you handle it if it knows a treat is coming.
Keep in mind that no two chameleons have the same personality or temperament. Chameleon 'A' may start hand feeding right away but 'B' may never be comfortable with it. Chameleon 'C' may be very defensive and territorial and not want you near his cage while 'D' might see you as another tree to climb and will happily come out and sit on your head. Chameleon 'E' might hate you but like your friend/sibling/parent/ect...
My male veiled, for example, with go out of his way to try and bite me when I open his cage for maintenance. But if my girlfriend opens his cage, he will gladly come to her if she puts her hand out and climbs to her shoulder. Even lets her hand feed him.... Jerk....
Key thing to remember is patience. Chameleons do everything in their own time. Its hard to build trust between you and them but it is easy to lose that trust too.