I do think a lot of it is either people only seeing or remembering the tough stuff, like that one day 4 months ago when you were really exhausted and overwhelmed, or just them projecting their own wishes on you. When I told my mum we were pregnant with our daughter (she's 3.5 now), her response was, "oh my god, are you okay? did you plan this?!?"
I should add that I was at the time a 31 year old, married, well educated, financially and emotionally secure professional, like we couldn't have been in a more ideal situation to have a baby, who was very much planned. She loves her granddaughter but routinely says things like, "I hope you aren't thinking of having another!". Again, stable, happily married, financially secure, I have a PhD, just got a great new job, we're really solid, capable, confident parents with really flexible schedules, and god, I'm about to turn 36. Still she's like, "you know, you really shouldn't be thinking about having another anytime soon!".
I think some people are just like that sometimes. I think in her case, it's her projecting her own insecurities as a parent on me as she struggled a lot, had a crappy marriage to my dad, and definitely didn't want another after she had me (I'm an only child). People should think more about what they say, but at the same time, just take it with a grain of salt. It's more about them than you, even though it's crap.