I warm it slightly. We only do milk now before bedtime, so I think it's just nice to have warmed rather than freezing cold milk just before bed. She has it at nursery for snack time though and I'm pretty sure there it's cold. If you are just making the change now, I'd warm it.