Certain foods may influence your mood negatively and cause anxious feelings. These are some foods you may try eliminating for anxiety: Caffeine is a stimulant that can increase feelings of restlessness and anxiety. Simple carbohydrates like sugar and bread can cause your blood sugar to rapidly spike and drop. This resulting low blood sugar can cause anxiety and irritability. Alcohol is a depressant that can cause anxious feelings, depression, and irritability. Inflammatory foods in general, like fast foods and sugar, can cause mood issues.