Take your time. We all deal with this a little differently.
When I was waiting to miscarry, I actually had more sex with my husband because I needed the closeness and physical contact. It was an emotionally fulfilling thing for me rather than a physical arousal thing. It wasn't like we were just so horny we couldn't keep our hands off of each other, more like words just weren't as effective as contact.
But I think that not wanting sex sounds very natural and like an important part of the recovery for you. If it's the pressure of TTC that gets to you, then by all means take a break from that. You don't have to rush in and pick up right where you left off. It's okay to take it slow for a while. If it's the physical side of the intimacy that's hard, then that's okay, too. Your body has been through a lot, both physically and hormonally, and I am sure that your partner will understand that it's a difficult transition for you. Keep him in the loop about it so that he doesn't feel that you've just cooled off because you don't love him anymore. Help him to understand what you're feeling.
When you do start taking steps back to a more active sex life, take it slow. Try to allow yourself to let go of your loss during intimate times and just feel the love and closeness with your partner. It's all part of the healing process, and there's nothing bad or wrong about how you feel, it's just a difficult transition.
Good luck.