Might as well. Most babies aren't so sensitive that it would do much harm if they weren't...but think about how much better you sleep after your bedding has been washed a few times. Smells nice and clean, it's nice and soft...
So if I have the time (which I CLEARLY do right now...) I go ahead and wash it all. But I'm also not afraid to put something on LO straight from the store if I'm in a hurry, or get caught out without an extra outfit or something.