That is so frustrating. Unfortunately, this is par for the course when dealing with doctors. You really have to take more of role in getting the care you want, because if you leave it up to the docs things like this can happen. This is the thing I'd wished I'd know before I visited an RE. Doctors can give conflicting advice, and two different doctors in the same office can have incredibly different opinions on the same situation.
One lesson I will take from this journey is that it is important for you, as the patient, to do as much research as you can and push them to give you the care that you need. Especially dealing with infertility, there's a lot that doctors don't know so you should not hesitate to question the doctor along the way to make sure they are crossing all the t's and dotting all the i's. Doctors have so many patients they really don't have time to give each patient the time they deserve.
From what I understand, a lot of docs don't like to perform hysteroscopies after O because of the chance of pregnancy, but others will do it and just take a pregnancy test before performing. For your own peace of mind I'd try to get in touch with a nurse to get the official word on how they handle these things.
On your hubby's sperm analysis, it's a good thing your doc brought this up, because my doc did not and led me to believe my hubby's sperm was just fine when a lot of docs would have disagreed with that advice. It still could be that the polyp is causing your infertility, so it's good that it's being removed. But now you have valuable information about another potential cause and you can work on that at the same time. For me, I had been seeing my RE for months before I got a second opinion elsewhere who felt that it was very unlikely my hubby could get me pregnant based on the results of his SA. The RE I had been seeing felt that my hubby's sperm was borderline, but still felt that it was good enough to get me pregnant and so we didn't take any steps to improve it. Soooo much wasted time and it's going to take several more months before we can expect to see any improvement.
I know firsthand how soul deflating it is to get such contradictory advice from a doctor, but don't let it get you down. You're already getting the polyp taken care of, and now you can start researching articles on sperm. The good news is that many men are able to make dramatic improvements in their sperm quality and counts with diet, lifestyle changes, and supplements.