How long have you had him? It's normal for them to be skittish when you first get them. They have to get used to their new surroundings. Never grab them. Once he gets more comfortable, you can try holding your hand out to him and see if he climbs on you. My biggest fear was mine biting me. He will walk right onto my hand if he wants, but there are days where he just turns around and walks away. They are solitary animals and usually if there isn't something in it for them, they don't want to be bothered.
so if i give him more time he will be more friendly? I mean my chameleon will hand feed no problem but when ever i try to hold him he run away and when i do grab him he trys to get off my hand and will often fall
a cham is not really a handling animal...if you think that one day he will be your best friend and sin on your shoulder like a dragon or a iguana your mistaken...try sticking a clean branch in there and lifting him out on the branch that would make him more comfortable. and lure him with a meaty horn worm.