Inclusion Criteria:  The subject has a histologic or cytologic diagnosis of colorectal adenocarcinoma that is metastatic or unresectable;  The patient either did not tolerate, is refractory to or progressed (or relapsed) following treatment with a fluoropyrimidine, irinotecan, oxaliplatin, and bevacizumab; Prior epidermal growth factor inhibitor therapy is required for patients with left-sided, RAS wild-type tumors; Patients must be microsatellite stable (MSS), microsatellite-low (MSI-L) or have proficient mismatch repair (pMMR; Prior TAS-102 (Lonsurf) treatment is not required; Patients must have known extended RAS and BRAF status as per local standard of practice; Measurable disease per RECIST v.1.1 as determined by the investigator; The subject has had an assessment of all known disease sites e.g. by computerized tomography (CT) scan and/or magnetic resonance imaging (MRI) within 28 days before the first dose of cabozantinib; The subject is ≥ 18 years old on the day of consent; The subject has an Eastern Cooperative Oncology Group (ECOG) performance status of 0 or 1; Recovery to baseline or ≤ Grade 1 CTCAE v.5.0 from toxicities related to any prior treatments, unless AE(s) are clinically nonsignificant and/or stable on supportive therapy; Adequate archival tissue available from primary or metastatic site for biomarker analysis (20 unstained FFPE slides and/or tumor block with minimum of 15 slides needed for eligibility). Tissue from the primary site is preferable for analysis, but if unavailable, tissue from a metastatic site may be utilized;  The subject has organ and marrow function and laboratory values as follows within 7 days before the first dose of cabozantinib:  The ANC ≥ 1.5 x 109/Lwithout colony stimulating factor support; White blood cell count ≥ 2.5 x 109/L; Platelets ≥ 100 x 109/L without transfusion; Hemoglobin ≥ 9 g/dL; Total bilirubin ≤ 1.5 × the ULN. For subjects with known Gilbert's disease, bilirubin ≤ 3.0 mg/dL; Serum albumin ≥ 2.8 g/dl; Serum creatinine ≤ 1.5 × ULN or creatinine clearance (CrCl) ≥ 40 mL/min. For creatinine clearance estimation, the Cockcroft and Gault equation should be used:  iii. Male: CrCl (mL/min) = (140 - age) × wt (kg) / (serum creatinine × 72); iv. Female: Multiply above result by 0.85; h. ALT and AST ≤ 3.0 × ULN in patients without hepatic metastases; ALT and AST ≤ 5.0 x ULN in patients with hepatic metastases; i. Lipase < 2.0 x the upper limit of normal and no radiologic or clinical evidence of pancreatitis; j. UPCR ≤ 1 mg/mg (≤ 113.2 mg/mmol), or 24-h urine protein ≤ 1 g; k. Serum phosphorus, calcium, magnesium and potassium ≥ LLN.  The subject is capable of understanding and complying with the protocol requirements and has signed the informed consent document; Women of childbearing potential (WOCBP) must agree to follow instructions for method(s) of contraception for the duration of study treatment with nivolumab and 5 months after the last dose of study treatment (i.e., 30 days (duration of ovulatory cycle) plus the time required for the investigational drug to undergo approximately five half-lives); Males who are sexually active with WOCBP must agree to follow instructions for method(s) of contraception for the duration of study treatment with nivolumab and 7 months after the last dose of study treatment (i.e., 90 days (duration of sperm turnover) plus the time required for the investigational drug to undergo approximately five half-lives); Subjects enrolled in Part 2 of this study must have a site of disease that is amenable to biopsy and be a candidate for tumor biopsy prior to the first dose of study drug to be considered for this study; a. 20 subjects enrolled in Part 2 of the study will be required to undergo mandatory pre- and on-treatment tumor biopsies. Please see section 5.1.11;  Exclusion Criteria:  The subject has received cytotoxic chemotherapy (including investigational cytotoxic chemotherapy) or biologic agents (e.g. cytokines or antibodies) within 3 weeks, or nitrosoureas/mitomycin C within 6 weeks before the first dose of study treatment; Prior treatment with cabozantinib or other small molecule kinase inhibitors; Prior treatment with an anti-PD-1, anti-PD-L1, anti-PD-L2, anti-CTLA-4 antibody or any other antibody or drug specifically targeting T-cell co-stimulation or checkpoint pathways; Radiation therapy for bone metastasis within 2 weeks, any other external radiation therapy within 4 weeks before the first dose of study treatment. Systemic treatment with radionuclides within 6 weeks before the first dose of study treatment. Subjects with clinically relevant ongoing complications from prior radiation therapy are not eligible; The subject has received any other type of investigational agent within 28 days before the first dose of study treatment; Known brain metastases or cranial epidural disease unless adequately treated with radiotherapy and/or surgery (including radiosurgery) and stable for at least 4 weeks before the first dose of study treatment. Eligible subjects must be neurologically asymptomatic and without corticosteroid treatment at the time of the start of study treatment; The subject has prothrombin time (PT)/INR or partial thromboplastin time (PTT) test ≥ 1.3 × the laboratory ULN within 7 days before the first dose of study treatment;  Concomitant anticoagulation with coumarin agents (e.g., warfarin), direct thrombin inhibitors (e.g., dabigatran). Direct factor Xa inhibitor betrixaban, or platelet inhibitors (e.g., clopidogrel). Allowed anticoagulants are the following:  Prophylactic use of low-dose aspirin for cardio-protection (per local applicable guidelines) and low-dose low molecular weight heparins (LMWH). Therapeutic doses of LMWH or anticoagulation with direct factor Xa inhibitors rivaroxaban, edoxaban, or apixaban in subjects without known brain metastases who are on a stable dose of the anticoagulant for at least 1 week before first dose of study treatment without clinically significant hemorrhagic complications from the anticoagulation regimen or the tumor; Clinically significant hematuria, hematemesis, or hemoptysis of > 0.5 teaspoon (2.5 ml) of red blood, or other history of significant bleeding (e.g., pulmonary hemorrhage) within 12 weeks before first dose of study treatment; The subject has radiographic evidence of cavitating pulmonary lesion(s) or known endobronchial disease manifestation; The subject has tumor invading or encasing any major blood vessels; The subject has evidence of clinically significant bleeding from tumor invading the GI tract (esophagus, stomach, small or large bowel, rectum or anus), or any evidence of endotracheal or endobronchial tumor within 28 days before the first dose of cabozantinib;  The subject has uncontrolled, significant intercurrent or recent illness including, but not limited to, the following conditions:  a. Cardiovascular disorders including: i. Congestive heart failure (CHF): New York Heart Association (NYHA) Class III (moderate) or Class IV (severe) at the time of screening; ii. Concurrent uncontrolled hypertension defined as sustained blood pressure (BP) > 140 mm Hg systolic or > 90 mm Hg diastolic despite optimal antihypertensive treatment within 7 days of the first dose of study treatment; iii. Any history of congenital long QT syndrome; iv. Any of the following within 6 months before the first dose of study treatment:  unstable angina pectoris; clinically-significant cardiac arrhythmias; stroke (including transient ischemic attack (TIA), or other ischemic event); myocardial infarction;  thromboembolic event requiring therapeutic anticoagulation (Note: subjects with a venous filter (e.g. vena cava filter) are not eligible for this study).  Subjects with a diagnosis of incidental, subsegmental PE or DVT within 6 months are allowed if stable, asymptomatic, and treated with a stable dose of permitted anticoagulation (see exclusion criterion #8) for at least 1 week before first dose of study treatment.  b. GI disorders particularly those associated with a high risk of perforation or fistula formation including: i. The subject has evidence of tumor invading the GI tract, active peptic ulcer disease, inflammatory bowel disease (e.g. Crohn's disease), diverticulitis, cholecystitis, symptomatic cholangitis or appendicitis, acute pancreatitis or acute obstruction of the pancreatic duct or common bile duct, or gastric outlet obstruction ii. Abdominal fistula, GI perforation, bowel obstruction, intra-abdominal abscess within 6 months before randomization, Note: Complete healing of an intra-abdominal abscess must be confirmed prior to randomization c. Other clinically significant disorders that would preclude safe study participation: i. Serious non-healing wound/ulcer/bone fracture; ii. Uncompensated/symptomatic hypothyroidism; iii. Moderate to severe hepatic impairment (Child-Pugh B or C);  Major surgery (e.g., laparoscopic nephrectomy, GI surgery, removal or biopsy of brain metastasis) within 2 weeks before first dose of study treatment. Minor surgeries within 10 days before first dose of study treatment. Subjects must have complete wound healing from major surgery or minor surgery before first dose of study treatment. Subjects with clinically relevant ongoing complications from prior surgery are not eligible;  Corrected QT interval calculated by the Fridericia formula (QTcF) > 500 ms per electrocardiogram (ECG) within 14 days before first dose of study treatment.  a. Note: If a single ECG shows a QTcF with an absolute value > 500 ms, two additional ECGs at intervals of approximately 3 min must be performed within 30 min after the initial ECG, and the average of these three consecutive results for QTcF will be used to determine eligibility.  Pregnant or lactating females; Inability to swallow intact tablets; Previously identified allergy or hypersensitivity to components of the study treatment formulations; History of severe hypersensitivity reaction to any monoclonal antibody; Diagnosis of another malignancy within 2 years before the first dose of study treatment, except for superficial skin cancers, or localized, low grade tumors deemed cured and not treated with systemic therapy; Active, known or suspected autoimmune disease (patients with type I diabetes mellitus, hypothyroidism only requiring hormone replacement, skin disorders not requiring systemic treatment or conditions not expected to recur in the absence of an external trigger are permitted); Subjects with a condition requiring systemic treatment with either corticosteroids (> 10 mg daily prednisone or equivalent) or other immunosuppressive medications within 14 days of first dose of study treatment. Inhaled or topical steroids and adrenal replacement steroids >10 mg daily prednisone equivalent, are permitted in the absence of active autoimmune disease. Subjects with interstitial lung disease that is symptomatic or may interfere with the detection or management of suspected drug-related pulmonary toxicity; Prior stem cell transplant or solid organ transplant; Known history of positive test for human immunodeficiency virus (HIV) or known acquired immunodeficiency syndrome (AIDS). Any positive test result for hepatitis B virus (HBV) or hepatitis C virus (HCV) indicating acute or chronic infection, and/or detectable virus; Subjects that have received a live/attenuated vaccine within 30 days of first dose of study treatment.