Using water alone can dry out your baby’s scalp, especially if you are using hard water. It is recommended to use a mild shampoo, such as Johnson’s baby shampoo, which does not irritate your baby’s eyes.