Hmm, I would definitely check with a phone nurse, as someone else mentioned. I was always told by my OBs to treat my fevers during pregnancy so that my body temperature didn't get too high.
If it provides any comfort, when pregnant with DS I had a bad cold around 6 weeks (fever of ~101 for two days) and was terrified that I was going to miscarry. My OB wasn't worried one bit. He just said to take tylenol and drink plenty of fluids. Then around 34 weeks, both DD and I got very sick (fever of 103+ for at least three days) with a respiratory virus that she brought home from daycare. For me, it ended up turning into severe tonsillitis. The original virus was not the flu, but the fever was high and quite hard to control. With medication, the lowest I could get the fever down to was 100.5. I was very, very concerned but was told by my OB that DS would be fine as long as I kept taking the tylenol and drinking lots of fluids. DS was very healthy at birth and still is, and my fever was nearly 104 a couple times. My tonsils were massive and white, and I was on antibiotics and painkillers too.
I think the flu's fever is potentially dangerous when uncontrolled and high, but I think the bigger concern is when the fever occurs during the first tri, or when complications of the flu such as pneumonia or inflammation of the heart occur. I think a call to your nurse or speaking to an on-call midwife or doctor might give you some guidance and put your mind at ease. And I hope my experiences provide a little reassurance x