If his mouth is healed, and the only issue you are currently treating is worms, then the treatment shouldn't be more often then once a week, in most cases.
My advice is to not do ANY handling, outside of the medications, leave the chameleon alone and offer enticing food, in a bowl, underneath his perch, in clear sight, every 3 days.
If he doesn't eat, remove the food and try again, 3 days later. If he hasn't eaten at all, then when you are handling him for medications, then you can feed the chameleon with whole feeder insects. It may take several attempts before he is willing to swallow the feeder, but once it does, maintain a line or one feeder after the other, until he's been given a good amount. Stick with high calcium and substantial items, such as silkworms, hornworms, medium sized feeder roaches, butter worms, and crickets. You can use forceps to drop several butterworms or a few crickets in the animals open mouth at a time. I'd feed up to 4-5 larger prey items and up to 10 smaller ones. Wait 3 days and offer bowl feeding again. Keep up this routine, until the chameleon eats on its own. Don't forget to dust the feeders on the usual schedule of calcium and vitamins. I'd discuss this program with your vet and avoid feeding him the vet vitamins, as it's more handling and more risk. The chameleon appears to have good body condition, so it can survive once a week supplementation and feeding, until it gets going, in my opinion.