Too much caffeine can also cause anxiety in people with panic or anxiety disorders. For those who drink coffee, experts suggest brewing it with a paper filter, because unfiltered coffee is associated with higher rates of early death, and can contain compounds that raise levels of LDL, or “bad,” cholesterol.