It's not always a sweet newborn smell. Babies and bad breath can go hand in hand. That's because babies, can suffer from sinus infections and colds, which create bad breath-causing bacteria.