Yes, when an umbilical hernia develops in babies, it goes away on its own by the time baby turns 2-3 years old. It might look weird, but the baby isn't in pain. However, if it gets worse, you will notice physical symptoms in your baby, and then it's time to see your baby's pediatrician.