Heather! it is so so good that you two guys are stating to talk about your feelings!
he REALLY does sound like he's in love with you and cares A LOT about you, at least to me.
He was brave enough to tell you he feels attracted to you despite being under pressure. He DIDN'T say he DOES NOT feel attracted to you, but said how the external pressure makes him feel that this is WRONG. I also don't think he kissed you because he feels bad. I think he did it because he has feelings for you and it was the most normal and natural thing to do in that moment and he just did it. it is also very good that you let your feelings go and that stress melt out a bit.
and the thing about getting married and all: i don't think what he said is the worst thing ever. i think he's just trying to word his feelings somehow, a feeling of being surprised how much he actually WANTS the baby and a family life with you himself. I think he's surprised by his own reaction and his own feelings and their strength, i think any 23-year-old would be. I am sure having a kid now never crossed his mind as something he may actually be happy and excited about before it happened, but now that it is here and with YOU, he is actually excited, he's attracted to you, he's thinking of future with you and all.
maybe he feels/fears that asking you to be in a committed relationship/marry him, would mean he'd force you into motherhood and into being a housewife and not taking your chance at college and whatnot (since he mentioned that if you were older - and already had your experiences and clear ideas regarding your place in your daughters' life - he'd ask you to marry him, but because you don't, he doesn't wanna force you).
i totally understand why you shrugged your shoulders when he asked you if you wanna be together, i get your fear... but you know you said just a part of your true feelings with it... and i think he'd be different to you if you told him how you feel.
if you ask me - without wanting to push you into anything - but i think you two should just start hanging out more often, and get closer again. and open up about your feelings gradually. i'm sure this guys cares a lot about you heather.
the worst thing that can happen is that you two grow closer before your baby comes, and share a part of your pregnancy together. (by the way, did he get to feel your baby kick??)
as the previous poster said, children put strain even on most solid relationships, which is true, but you guys never took your full chance so far so... getting to know each other more before your baby is here can't do you harm, and may actually help you get at least a little bit ready for when the baby actually comes.
ps. it's also great that he told you how his family feels about you and the baby and how excited they are
it's so good that his parents are also supportive like yours. you kids come from two good families and are really lucky!
and i already see your daughter will have two grandmas to spoil her rotten
(i mean this in a GOOD way!)