As long as your child has a Social Security number, you can open a bank account for them at any age—even when they're a baby.