Lch, you are absolutely not a bad mommy! BF difficulties happen, and you are concerned about his weight loss and whether he is thriving, which makes you a good mommy.
5 days would be on the longer side for your milk not to have come in, but not unheard of. However, like you, I am concerned that it means he is not feeding properly.
Do you have any access to a lactation consultant, or a LLL group or other breastfeeding support group? Your pediatrician's office might have a LC you can meet with. I would strongly recommend seeing a professional SOON for help with his latch, because the first few weeks of BF are important for building up your supply.
Weight loss is pretty normal (depending on how much it was, I think they say up to 10% is ok?) and acting fussy and frustrated is normal too. But if you are concerned, you could really benefit from speaking to an expert in person.
Also if you can find any time to pump, that might help bring your milk in faster, which will give him more of a reward for trying to BF and thus encourage him to keep at it. And it will keep up your supply in the meantime while he figures it out.
Hang in there hon, you will both be ok