However, an allergic response can be avoided by eliminating allergy-inducing foods from your diet. Yeast allergy is a condition in which a person is allergic to foods containing yeast, mold, or another type of fungus. Symptoms include skin rashes, asthma, and other respiratory problems. See your doctor if you are diagnosed with a yeast allergy.