Well for a start, whether he is on probation or not, any kind of disturbing behaviour from him that you feel is threatening should be reported. The more info you have against him, the less chance he stands of gaining access. Personally, if I were in your shoes, I would do the following;
1. Have an open and frank discussion with him about his rights. State that because of his behaviour and being in prison etc he may NEVER, if at all, get to be with his child alone and that is where you stand, period. He chose the path of being a burgular and he is behaving agressively, tell him this is not an environment you wish your child to be exposed to. Give him hope for the future though and say that, maybe, just maybe you may consider after professional evaluation and if he has changed completely to give him more access, until then he has not earned the right to it at all.
2. Write down and record ALL dealings with him. If this lands him back in prison, tough. he cant be bullying and aggressive towards a child, you have a baby to think of, that is your no1 priority.
Lastly hun, he doesnt stand a cats chance in hell of being alone with your child, period. If he threatens you, threaten him back and say if he carries on like this, you will and can call the police /probation officer on him.
I cant see that any court in the land would grant unsupervised access to a man that has been in prison, threatens the mother of his child and also doesnt pay a penny. Stop worrying about this and perhaps speak to a Health Advisor /professional about procedures /steps to take.
I know its unfair on men sometimes who want to see their children but unfortunately just because you father a child does not make you a proper 'father'. You earn that right by supporting the baby, loving it, nurturing it and doing what is best for baby. If he had any sense, he would approach this with kid gloves or hes back in jail again. He's a bit of an idiot really.
Hope that helps, be strong and record everything and don't be afraid to report him hun, just dont be afraid of protecting your LO.