I was going to say don't let anybody judge you just because of the area you live in.
I've been there, done that and got the t-shirt, I was brought up on a typically bad estate and I'm not just saying that I mean it was littered with druggies and alcoholics.
As I got older and we did move houses after 12 years of being there, people couldnt believe I lived there but if you keep yourself to yourself no one bothers you and you've just got to bat away all the comments and frowns. I was brought up in a very loving household and thats all that matters and that all that should matter for you and LO.