Kay-leb
I know a couple of people called Caleb and it's always pronounced like that. ( I think the ending is stronger than the ending of Jacob, hence the stronger actual 'leb' rather than a softened version.)
You can, of course, pronounce it how you like. It just depends on how bothered you'll be when people say Kay-leb.