My vet said that they may be attracted to some minerals that may be in the dirt or sand.
My male veiled was obsessed with the sand in the female's laying bin and did his damnedest to get a tongueful every chance he could (when I was cleaning it). Doc told me that a little bit on rare occasions doesn't hurt as long as there is nothing too big and chunky or too much of it.
So, every month or so I did allow a bit of cleaned and baked dry sand. I would put just a tiny bit in a feeder cup and show it to him and he would zero in on it immediately. Then I would move it away and make him go over and get it for himself. Didn't matter how sick he was, it might take him awhile to get there but he would work until he did and then get huffy that I took it away even though the cup was practically empty.