I was wondering the same thing yesterday actually. I was reading online about it, they say it's just to remove the chemicals from the clothes that manufacturers put on them to make them wrinkle-free and nice looking in the store.
I read a lot of people say they never did and their baby turned out fine, I think it's really just your personal preference/comfort I think we will just because baby's skin is going to be so sensitive, I don't want to put clothes on that might irritate her.
This is why I'll be washing all of baby's stuff - for my own level of comfort and because you never know how sensitive their skin's going to be. Better safe than sorry.
I'll be using Arm and Hammer Free and no fabric softener, as it seems to me that would defeat the purpose of using a scent-free detergent. I'll be seeing if using vinegar really does work as a softener, though.