The most important rule when holding a baby is to always support their head and neck. Newborns have weak neck muscles, and proper support is crucial to prevent any potential injuries.