I get little gnats in my keeper, and occasionally I see 1 on Gary's cage. I've never worried about it because I live by a hay field. I actually don't let a lot of things bother me because of my living location. Location is a big factor. If you live somewhere that is odd, get a new supplier. I order mine from georgiacrickets.com. They'll ship them anywhere, they're a Premium Crickets franchise. Check out the sponsor's list on this site. You may find a better one that's closer. I just like those guys.
I've never had a problem feeding the crickets to my lizards after finding them in there. If you clean out the keeper after you're done, the problem may stop. But some flies are small enough to get through the screens and then you may need to get a carnivorous plant or something.
Just take where you live into consideration and maybe figure out ways to catch them. I wouldn't throw the crickets out, but I also wouldn't buy them weekly, I'd order them online.