Don't add in formula. Babies sleep well on formula because it's hard for them to digest and takes longer. It could also backfire on you if he has tummy troubles on formula and is up even more because of stomach upset.
It honestly sounds like he has a little bit of a sleeping issue. At 6 months breastfed babies should easily go 8-10 hrs at night without a feed. He maybe nursing out of habit or just want to be with you as there is no reason why he can't go through the night without a feed (there's no physical reason why). Of course he may be going through a growth spurt, in which case it'd be ok to feed at these times, but if this is lasting more than a couple weeks it's likely a habit.
Everyone has a different opinion on the subject, but I recommend sleep training, especially at this age. I sleep trained my LO two weeks ago and it's one of the best things I've done so far. She was waking up at midnight, 2am, and again at 4am to "snack." I came to the conclusion she was habitually waking up. I used the Ferber method after reading this blog: https://noobmommy.com/2008/11/to-ferberize-or-not-to-ferberize.html. She now is asleep by 6:30pm, I dreamfeed at 11pm, and she doesn't fuss until she wakes for the day at 6:15am when I nurse her again.
Some people are against sleep training, which is fine. I've read that somewhere around a year or older babies that once woke throughout the night sort things out themselves and stop waking. This is not from experience, though, so I'm not sure if it's true.
Also, when you night wean your LO will just take more milk in during the day. They'll always get enough milk that they need sometime within a 24 hr period. If you night wean just make sure you nurse him longer during the day or give him a little more milk in his bottles. I also agree solids won't do much to help him sleep, especially so if he's habitually waking.