I also think it is selfish to be honest. I'm not a germ-phobe either - I just don't relish the idea of my child having to feel like shit and no doubt pass it on to the whole household. Only my partner currently works so that I can be at home to look after our daughter, and we get by ok, but he is self-employed and paid-per-job, so if and when he gets ill he basically has to battle on through to avoid losing money.
Personally, and people may well slam me for this, I think that sitting at home and caring for your sick kid through the sleeplessness and misery is one of the things you sign up for when you decide to have a baby. It's rubbish, but you suck it up and you deal with it. If you absolutely have to get out, then choose something appropriate!
There's a difference between taking your poorly baby out for a walk/to the supermarket, and taking him or her along to a class packed full of other young babies or newborns with immature immune systems.
If my baby (when she WAS still a baby) had contracted a virus from another child in such a setting, which then developed, say, into life-threatening pneumonia, I think it would be hard not to feel blameful towards the mother.