I guess there is not really a right answer to that question. All chameleons have different personallities and different levels of tolerance. I will just say what I do currently.
The female I have hates being touched. Since she has become full grown I have handled her twice. Once after she layed eggs to put her in the shower. Once while she was asleep to move her cage around a little. Any more then that would be bad for the both of us. I have seen her rip the bark completely off a stick in a temper tantrum. She is however very friendly when I clean her cage or have to do something in the cage. She usually just watches me.
The little male that I have I handle once a day. I hand feed him at least one food item a day. He comes out on to my hand and just chills out for a minute or two. When ever he seems like he is over it I let him go back in his cage. He is very friendly and I do not think I will push it much more then that.
The level at which they start stressing is probably when you need to put them back in there cage. This does not take long with most chameleons. Coloration changes, posture or frantically trying to escape from your hand. You are also a an animal that is 98 degrees at all time which some could find not so appealing for an extended period of time.