Food allergies may cause stomach discomfort, causing some babies to arch their backs. Check if your baby does it after consuming certain food, and speak to a pediatrician.