Generally that rule is not applied to soft-bodied worms. A cham can usually handle worms that are much larger than the size of crickets that are appropriate for them.
But with young chams, I'd be a bit cautious, and still wouldn't feed them anything ridiculously large. I've noticed that younger chams tend to eat more rapidly, and often don't chew their food properly. With a really large worm, there could be a chance of choking.