Chameleons, for the most part, are solitary reptiles that prefer to be left alone with no to minimal handling.
When you are purchasing or adopting a chameleon, consider what you want from your pet first. Do you want someone to just watch and admire? If that's the case, then pick out one that likes to be left alone.
To me, it seems that you are looking for a friend that you can handle and bond with. One thing about chameleons is that early socialization is best for the ideal friend who will climb on you and stuff. You adopted an adult, who has already lived a good portion of his life and is just now meeting you. With some slow introductory methods like many have listed above, you may be able to make him more friendly towards you but keep in mind that there's a good chance he may also continue to remain solitary.
For the future, take the time to find a chameleon that is best suited for you. Find a young baby, and begin to slowly introduce yourself into his or her life. Your chameleon will be growing up along side you, and therefore with proper socialization should be friendly and trusting enough of you to allow you to handle them. Just take it slow.