Colorectal cancer begins in the colon or rectum, parts of the large intestine. According to the American Cancer Society, it is the third most common cancer in both men and women, other than skin cancers.