I'm no fan of traveling it either. Leaving it to the ground will causes some vibration. Plus, the temperature is lower on the floor than higher.
Did you know that chameleons can live a few weeks without eating and over a week without drinking? Of course I am not telling you to do it, but what i mean is it's not your three/four days that will ruin his life if you leave him home, put 7-8 big silkworm in his enclosure and have your misting system run while you're gone. The chamsitter cold just come in and check it every day to tell you he's Ok.
To be honest, i am more worried by your behavior and thinking and their impact on you and your chameleon than by what could happen to your pet while you're away from it. You know, they can be your family, but you have to leave your family at home sometimes... Overprotection does as much harm as underprotection in my mind.
Of course i have read stories of people who left their chameleons for weeks to friends, and ended up having a chameleon with serious illness, but this won't happen in 4 days. If you took care of your pet already, it's not a week without dusting that will cause MBD, especially if you gutloaded your insects before leaving! And you aren't leaving for a week!
At the end, the decision is yours, but since you are on this forum, i assumed i could share my thoughts. And if you decide to still bring it in car with you, try to avoid vibrations, by puting some towels on the bottom of his cage and also under his cage. They will help reducing it.
I suggest watching this video about vibrations. It might help you out:
http://www.youtube.com/watch?v=bVL3b1wKZQU