I have to be honest, if I was your age and had only 4 years left to pay off mortgage, I would do it too, if it was going to revert back to 20 years if I had a baby now. It's an amazing position to be in, an you would never have to worry about finances again as you would always have equity. Failing that, if you can pay off 50% in two years and your monthly amount will drop, that's bloody good too. Babies can wait, what an amazing position to be in!