It's totally normal at this stage not to feel movement every day. Baby is still pretty small, and if he/she switches positions (facing your back, for example), you may not be able to feel those little kicks and rolls. And it's definitely possible that baby is very busy in there, but you're distracted by other things and just don't notice. Then when you sit down to concentrate on the movements, baby is asleep or just feeling lazy. They don't really care about our schedules at this point... or ever...
Most doctors/midwives don't recommend doing kick counts until after 26 weeks because you're more likely to scare yourself/drive yourself crazy if you don't feel movement. But take a deep breath and stop freaking out. You're both fine