This happened when I got pregnant with my son. I had what I thought was a period, so I took a round of clomid, and it turns out I was already pregnant. I didn't find out until they did a dating scan at the end of the month and we realized I was 3-4 weeks further along than I thought. I was FURIOUS that the OB didn't recommend a pregnancy test before I started!
I spoke to quite a few specialists and the consensus was that it was early enough in the pregnancy that it would have an "all or nothing" effect, ie. either I'd miscarry very early or there would be no effect...something to do with things not being developed enough yet for any harm to be done I think? I don't remember the details, but I had no complications and have a perfectly healthy 13 month old now. They did extra ultrasounds throughout my pregnancy just to make sure everything was developing correctly, but they did not expect to find anything other than a possible early miscarriage. I was told there would be NO increased chance of chromosomal abnormalities (like down's syndrome, etc) as a result of the clomid. Sorry I can't remember all the details of WHY this is the case, but it all turned out fine.
Most of the research on clomid that points to negative effects on the fetus was done on animals, rather than observation of human populations (since no doctor would prescribe clomid to a pregnant woman, they can really only get data from the small number of mistakes like ours) and while animal studies are illuminating, animals ARE different from people. Just throwing that out there.