It is a long-standing Christian custom, but there are genuine Christians who are not baptised, just as there are many baptised people who do not have true faith in Jesus Christ. Being a Christian means being right with God through having our sins forgiven. God offers this forgiveness because Jesus Christ died on the cross to take the punishment that each one of us should receive. Being a Christian is not mainly about being good, or being loving and generous, or following standards of morality. However, people who have been forgiven by God will want to try and please him, and so they try to live accordingly.
Baptism represents a new start.
Baptism symbolises the washing away of sin. This is another way of speaking of God's forgiveness. Baptism symbolises the death and resurrection of Christ which Christian believers share in. Baptism symbolises formally becoming a member of the church. Most Christian churches regard baptism as a condition of membership, but this aspect of baptism is not found explicitly in the Bible.