No, your diaper bag won’t count as carry-on baggage as long as it contains articles for the infant travelling with you.