It does SEEM like it's a good thing to do "once in a while,"but if you look at it, from pretty much any angle, it's never ever worth it.
I had to deal with my brother doing this. The little turd would - behind my back - feed our female veiled full size bullfrog tadpoles, and my male (who was only 13-13" long) fuzzies. Never did we observe any unusual stools, or any weird digestive problems, so his digesting it was not - in our case, anyway - an issue.
It's fat and calories. It's so easy to overfeed chameleons in captivity, I find it a challange to keep them lean. Most of the most convienient feeders are fatty larvae. Vertebrates have a different balance of nutrition than arthropods. People think a pinkie is like a "shot in the arm" of calcium, protein and nutrients. In reality, the chameleons digestive system will probably extract more nutrients out of a dollar's worth of gutloaded crickets, than one single pinkie. Factor in the milk in it's gut content, and you may be giving it indigestion - reptiles and dairy do not usually mix.
Insects have more protein, per pound, and per dollar, than mice. Even more importantly, they are optimized in the digestion of insects and insect proteins, not mammals and animal proteins.
The only real reason anyone oculd have to justiy feeding mice or fish or frogs to a chameleon is "cause they think it's cool to watch."
If that's the case, then give your chameleon to someone else and buy a snake!
They really dont' benifit from eating vertebrates - at all - when insects are available.