What follows is sort of a stream of consciousness about this and sharing of some articles and ideas.
I appreciate you bringing this up, as it's been floating around but has little evidence or studies supporting its application to captive husbandry to my knowledge. I think it has mostly been argued by one particular well-known chameleon enthusiast.
I might be accidentally strawmanning this, but the argument as I know it is that the fruits we eat are so full of sugar (fructose) that they are not a natural food item anymore because they're so far removed from what exists in the wild, especially from the natural habitats calyptratus inhabits. Furthermore, there is an argument that chameleons cannot digest the "sugar" in fruits (although sugars in fruits are variable and the term is broad). I take issue with this, as we are not talking about feeding exclusively these fruits in isolation. Sugars are present in many foods and prey items consumed by chameleons (as you nicely point out) and we are talking about feeding a diet comprised of many things. Further, as someone who has published on the impact of diet on immune system and liver physiology, I can tell you it is not about a single ingredient most of the time, but the overall composition of nutrients in the diet. So I can see not exclusively feeding fruit, or not using it for gutload for every single insect, but completely avoiding the ingredients for the above reasons doesn't stack up to me.
I'm not advocating for people to feed their chameleons fruit constantly, but I don't think the reasons why fruits should be avoided (as food, as gutload) are totally solid. I don't bother with fruits except for gutloading on occasion, but I doubt they do much harm. There's also an argument that citrus should be avoided because it causes "breakdown of homeostasis", I have some concerns about that and I don't think it's accurate nor are there any studies that I'm aware of that look into this.
Here's a couple papers that are worth discussing, but do not really support the whole "chameleons can't digest sugar" thing. I unfortunately don't think this has been tested rigorously in chameleons specifically, but here's some stuff in other reptiles that might be able to be extrapolated to chameleons.
https://journals.physiology.org/doi/epdf/10.1152/ajpgi.1985.249.2.G271
One is demonstrating in turtles, chuckawallas and iguanas, that rates of glucose uptake in the intestines are less than mammalian uptake but this is because of differences in overall anatomy and decreased surface area of intestinal mucosa rather than something intrinsic to the biochemistry of the system. However, there is still plenty of glucose uptake in reptiles. Maybe this was taken out of context to argue that chameleons can't digest sugar. Obviously the study was not done in chameleons but there is some reason to believe the findings are similar.
https://journals.physiology.org/doi/abs/10.1152/ajpregu.1996.271.5.R1447
This one just demonstrates that some lizards have ability to run similar metabolic pathways we do, one of which utilizes a metabolite of fructose (F26BP), suggesting that there might be mechanisms to modulate dietary fructose uptake, conversion etc. in reptiles. This does not necessarily mean reptiles (chameleons) uptake fructose, but it at least suggests that the fructose molecule is not totally foreign to their systems.
https://www.ncbi.nlm.nih.gov/gene/6518/ortholog/?scope=8504
Using NCBI blast, one can identify that corn snakes express the gene encoding the enzyme that leads to uptake of fructose (so digestion), which is a bit surprising.
Now, here's where things get a bit interesting.
I am not able to find a homologous glut5 (SLC2A5) gene (encoding the main enzyme that uptakes fructose) in the veiled chameleon genome. This doesn't necessarily mean they don't uptake fructose, but if they do, it might be through a different mechanism. So maybe this is the source of the idea that they can't digest sugar-they don't have a similar gene as the typical fructose transporter. Again, this doesn't mean there's no way for them to take up the fructose, just that it might be a different mechanism.
So, the last bit is the only thing that supports any inability to digest fructose, but it is far from conclusive.
It's an interesting topic. However, even if we were to determine that chameleons cannot uptake any fructose at all and it's sitting idly in their gut (I think this is very unlikely), nobody has demonstrated that this is a bad thing. Further, other species of chameleons have been documented eating fruits in the wild. A lot of steps are necessary to really get at why feeding/gutloading with fruits is a bad idea. I think it's worth considering all of this though and keeping things in moderation. Probably not worth feeding chameleons fruit regularly regardless, as it doesn't seem like a major part of the diet in the wild, IMO.