Sometimes people imagine the bad things that could happen or that they might do as a way to protect themselves and stop the thing from becoming reality.
I read an article in a magazine where a woman said she was holding her baby up to the window to watch the snow fall and started to imagine what would happen if she lost her grip and the baby fell, even though the window was closed.
She felt terrible because she thought something was wrong with her for imagining such things but a psychologist said it was a natural way of trying to stop the event happening. A kind of superstition, imagine the worst thing that could happen and by thinking of it you are prepared for it and can stop it happening.
Some people do find it very hard to cope with a baby who cries for hours on end but honestly when it's your baby and you love it you find you have patience reserves you never suspected. I don't mean that you turn into a saint and nothing bothers you but you do cope. And if you truly do need to get away for a minute either pass the baby to somebody else or place it safely in it's cot or pram and go into another room for a short break.
As long as your baby is safe while you do it taking a minute or two away so you can calm yourself down can help you.
But if you are worried, as others have said, talk your fears out with someone you trust.