The risks associated with dirt are that the cham may ingest something poisonous to it (like pesticides as Dylan mentioned, or fertilizers, or those perlite balls), or it may ingest a solid particle that it cannot digest properly which may lead to impaction in the cham if it is too large for the cham to pass through its system whole.
Provided the dirt it ate was safe and he didn't eat any large particles of stone or dirt, your cham should be just fine.
As Dylan mentioned, it seems that some chams naturally seek out dirt to eat - this could be due to some mineral/nutrient that they are trying to obtain.
Just keep a watchful eye on your chams eating habits and also make sure that it is producing healthy feces regularly. Any change in these habits could indicate that it has picked up a digestive tract issue.