I could have written this post myself. My first two children were Irish twins... not by choice but because I fell pregnant while on the pill with the second and thankfully he stuck. At first I was shocked because we were then undecided whether we wanted another one or not but then I came around and actually looked forward to having him. My third would have been an Irish twin too but then I had three unexplained back-to-back losses. Finally I got my rainbow and she was born in August of this year.
We want a fourth baby but, like you, I'm worried about people judging me. When I posted on FB my pregnancy with my rainbow baby, I had rude and hurtful comments from people. I think if I do get pregnant I'm going to keep it off FB but even that won't completely help me. I have people commenting on the size of my family all the time. Really I want to tell them off because it truly is none of their business. My theory is that some people are able to handle more kids than others and at the same time some kids are easier to raise than others so just because someone has had a negative experience in parenthood doesn't mean that they should judge others that are able to cope better. Sometimes I think it is jealousy over anything else.
As a side note I almost made a lady faint once during my last pregnancy. She was drawing my blood for my first trimester bloodwork and asked me if it was my first baby. When I replied "No it is my third" she gave me the typical grunt of disapproval and then went on to say "Well then you should have a fourth to even out the genders". I think she said that to be smart and when I replied "Yes, we already plan to" she just about fell off her feet. Luckily she didn't because she still was holding the needle in my arm but she didn't say anything else the rest of the time she was drawing my blood.