Yes, I have had this in both my pregnancies (so far). I am average weight, but just didn't gain much. With #1, I lost 14lbs in the first trimester (I wasn't really that sick, just seemed to lose) then gained it back slowly. I was only 7lbs above my pre pregnancy weight when I gave birth.
This time I lost 4-5lbs in the first trimester and haven't gained anything back yet at 16 weeks. Last time I wasn't weighed at all by the midwife and was only weighed at booking this time. My daughter weighed 8lb 14oz at birth and this one is growing well so far so I am not concerned.