Since stink bugs really do live up to their name, I personally would avoid them. The "musk" they release when threatened is their natural defense against predation, so they are probably quite distasteful. Not sure about their toxicity.
I live at the edge of a couple hundred acres of woods, so no shortage of bugs and critters here! I do avoid large grasshoppers, redworms, lightning bugs (highly toxic), the known toxic caterpillars, the "pedes" centi and milli, ladybugs..and probably others I can't rememember right now! LOL
If you keep him outside this summer, be wary of red/fire ants if you have them there like we have here.