Milk might provide brief relief, but it also increases stomach acid, which can worsen symptoms. Your doctor may recommend taking antacids or other drugs to reduce acid in the stomach. Talk with a health care provider if you're concerned about ongoing pain or discomfort in your stomach.