So not to underplay this, as I wasn't aware of the severity, but there are a lot of woods here. Removing it I feel wouldn't do anything, as they're all around. There'a also a lot of people around here that live in the woods. I read something that there's only been 4 cases of the virus in PA since its discovery and they seemed to be away from here. Maybe there is some new info on that though? While I don't want to ignore this being a risk, I really don't have the time or money right now to trash the cage if it's unnecessary. If they were all around and there's so many people here too, wouldn't somebody have come down with it by now?
Well it's like I said, it's rare because they usually don't come out of the woods. It's unusual for them to do so.
When they do, it's not the live mouse that's an issue, it's their Fecals and urates that are the problem or a dead mouse.
Even then, it's only when its disturbed. If the mouse poops and no one touches it, then it will dry up and I think the disease dies off, over a period of time.
Usually infection is caused by cleaning, or disturbing the poop. Such as vaccuming it up, or sweeping it up. Especially when inside of a non ventilated area. Outdoors, I reckon it's harder to breathe in, but still possible.
I wasn't saying scrap the cage, I would switch the mesh to keep them out of the cage though. I would scrap all the soil, and clean the dirt below it. And I would wear a respritrator to do so.
Field mice, do not normally have a different color belly at all. Your first pic, shows what appears on my phone to be a white belly.
Another way to tell, is field mice tails are hairless and skin color, where deer mouse tails have short hair and thus are darker, with a light color to the underside of the tail, like in the belly.
That mouse in the photo does not have the pink tail a field mouse or rat will have. Like the furless cat from Austin Powers

.
It's tail, at least for the Pic, is dark.