You should not put baking soda in your guinea pig cage. First and foremost, baking soda is a lung irritant to these rodents, likely to cause irritation to their sensitive respiratory systems. Secondly, baking soda can cause intestinal bloating and digestive upset if ingested.