You can’t fully immerse your baby in water until their umbilical cord stump has fallen off, which won’t be for the next week or so. You can spot clean your baby now, as frequently as you see fit.