No you are not wrong, it depends on how much effort and stress you want to put on the cham. For example, my two Chams, Bruce and Arda are both extreamly calm chameleons. They do not mind being handled and seem to enjoy it at times. Now, they were like that when I got them, and though I don't handle them too often, I do handle them on a regular basis (at least twice a week) to keep up this bond, as well as hand feeding.
On the other hand-or rather off-at the store I work at there is a veiled chameleon female who hates people, or did at least, with careful work my co worker and I have gently coaxed her to accept being handled for a short period of time. She used to bite and snap at everything, and over the weeks she has calmed and now lets us hold her for short periods of time. But it took the two of us a long time and we probably wont ever have her as gentle as my chams. I think it is important to be able to handle the cham if there is ever an emergancy, or you need to clean something that an angry cham wont let you near-like a cut or something. But there was a cham at chromachameleons, at the reptile show, who wouldn't be handled without leather gloves. It just goes to show that each one is slightly different. You can try to teach, but in the end it will come down to the chams personality.