yes definitely. I've been offering cooled boiled water the past couple of days now it's been warmer, in between feeds. She sometimes has a few sips.. other times takes up to an ounce. They'll only drink what they need. As long as it's not near feeding time so you don't fill them up with water. Altho you'll prob find if you're baby is hungry they'll spit out the water and continue crying for a feed. x