I’ll put a disclaimer on this and say that you obviously know your hubby and relationship best - but seriously? It’s not fair of him to toy with your emotions like that. Say yes to trying in June then saying let’s not try until... when?? YOU are his wife now, and what you want matters. Is having another baby really going to disrupt his relationship with his children to the point where he can’t be there for them anymore? No. And oh my word his ex should butt right out. How rude.
Try talking to him, explain that this is really important to you and see if you guys can come up with a game plan so that his kids will still be very involved even with a new baby. And I would try and get across the point that he needs to discuss these things with you before making a life decision with his ex about your family life.
Let us know how it goes, sorry this has been such a rollercoaster for you xx
(Also I’m 23, DH is nearly 29. You’re not a child and you’re not too young to be wanting to start a family. His ex probably feels threatened by you which is why she would say something hurtful and demeaning like that)