But, in general, it's not recommended to drink coffee while taking Adderall. While small amounts of coffee may not worsen Adderall's side effects, combining the two can raise the risk of side effects like high blood pressure, elevated heart rate, feeling jittery, and trouble sleeping.