Yes, you can hold your newborn in an upright position. In fact, it's very important to do so to help prevent reflux and other digestive problems. When your baby is held upright, gravity will help keep the milk and food down in their stomachs. Upright holding will also help keep the airway open, ensuring that they get enough oxygen.