I would just treat it as play and let him play, but don't let it encroach into non-play activities. Mine is 4 and certainly she does play being a baby sometimes and gets a blanket and asks to be held like a baby. She doesn't have access really to her old baby things as they've been put away, but she certainly will find a cup or a water bottle and play that it's a bottle sometimes (there's nothing in it though). I think that's fine as part of play. I wouldn't let it go so far that you are putting milk in the bottles and actually feeding him, or that all your meals become you spoon feeding him either. But I think as part of pretend or occasionally if it makes him happy does no harm. Mine occasionally asks to be fed when she's feeling silly. She was never spoon fed as a baby anyway, but it's cute and I don't mind playing along, but it wouldn't be at every or even most meals. I otherwise emphasise that she needs to eat like a big girl, so it's more just when she's feeling silly. She has found a dummy (pacifier) recently and was giving it to one of her toys, which was fine, but I wouldn't let her put it in her mouth. She doesn't need it and it's like 4 years old and she found it behind her toy cupboard, so it's sort of gross. I think it's fine to indulge it as part of play, but just draw some boundaries so it doesn't become part of a new routine or anything (like he's wanting to be fed at every meal or have a bottle before bed at night). You could even encourage him to play babies with his toys, so they become the babies and he can play with all his old baby things, but without always needing to be the baby himself.
