Anxiety is not directly linked to cancer. But anxiety and panic attacks are symptoms of cancer of the adrenal gland, pituitary gland, thyroid and pancreas.