Japan Rail state that you need a child Rail Pass for children aged 6-11 years only.
Children under 6 years do not need a Rail Pass, but without a Rail Pass they may only occupy a seat if it is not required for a fare-paying passenger.
If you want your infant aged under 6 years be able to occupy their own seat, you should buy a child Rail Pass for them. This is particularly recommendable at busy times.
Please note that at busy times, you should always try to reserve seats in advance in the particular trains on which you wish to travel. With the Japan Rail Pass, seat reservations are free of charge, so it's always worth doing if you can plan ahead enough to know which trains you are going to take.