Here's what I did to get my newest panthers used to me a little, which has made getting them out for vet visits, giving medication, cleaning cages, etc. much easier on both of us. They obviously don't love me, but they haven't associated me with anything bad yet (except the one who I had to give oral medication to, so he still doesn't trust me to not stick a syringe in his mouth. Subsequently, he's learned not to open his mouth to gape/bite.)
I think that actually grabbing them/coaxing them out of the cage is way too scary. I think that right there gets everything started on a negative foot. If you don't have your cham yet, when you do get him try to make the process of getting to the cage as slowly and calmly as can be. Mine were shipped, so I opened up their bags and let them come out of the bad and onto a branch in their cage on their own without. So automatically they saw me, but didn't see me as a big scary thing right off the bat.
After a week or two of settling in with no contact, I began trying to get them more comfortable with me. Chameleons freeze up a lot if they're scared of you, so step one was to get them comfortable just seeing me at a small distance, as well as getting them comfortable with coming out of the cage by themselves. I bought a 6' fake ficus and put it in front of the open cage, and I sat on the floor a couple yards away (to be really much lower than them) watching TV on my laptop. One of my chameleons took 4 full episodes of Dexter to come out onto the tree! lol And after climbing to the top for 15 minutes, he then out of his own free will went back to his cage. Then I closed the cage and left him alone for the day. I kept doing this every other day or so for a few weeks until he would quickly come out and explore his fake tree calmly even if I was in the same room moving around.
Once he was comfortable with me in the room while he was outside his cage, I started putting my hand in front of him on the branch he was on to get him to walk onto it. I'd let him walk on my hands a minute or two before he showed any signs of wanting off and put him back on his tree and walked away, so the handling didn't end on a stressful note.
And this has worked really well for me, trying to apply what I learned about working with other exotics to have a better "relationship" with my chameleons. And the chameleons I got after trying this technique have been much calmer chameleons around me. Which, like I said, makes moving them for cleaning, meds, vet visits, moving, etc. much easier.
This all being said, I still don't hold them very much. But I think you understand that, you just want to be able to avoid the biting and fighting, like I still have to do with my oldest panthers. Those I have to handle with gloves or sticks, and it's super stressful for them. So I'm with you, if you can get your chameleon a little comfortable, it's a lot less stressful later. It won't necessarily work for ALL chameleons, I've just had luck with my last couple. But you can try and see how it goes. Remember to respect the signals he sends you and back off if it's just freaking him out. But best of luck!
Edit to add - Oh, and don't forget to try hand feeding. It also helps associate you with good things, so soon you're that person that means food and free time on a tree or something.