Inclusion Criteria:  Male or female subject aged >= 18 years Histologically confirmed urothelial carcinoma with metastatic disease or with unresectable, locally advanced disease. Variant histology, including, but not limited to, neuroendocrine, sarcomatoid, and squamous differentiation are permitted to enroll  Patients must have progressed on at least one prior therapy; and  Have no further standard of care options or the available options are associated with minimal overall survival benefit; or Have no further clinically acceptable therapy; or The patient has declined standard therapy The discussion regarding the choice of standard therapy offered, if available, and patient's choice and reason(s) to decline standard therapy should be documented clearly in the research chart. Patients may have progressed on immune checkpoint inhibitor therapy Body weight > 30 kg  Malignancy harboring ARID1A loss of function (lof) genomic alterations as determined by the standard of care next-generation sequencing. Results must meet the following criteria:  Presence of a somatic alteration considered pathogenic/likely pathogenic in ARID1A gene as determined by genomic sequencing performed in a Clinical Laboratory Improvement Amendments (CLIA) laboratory Somatic alterations will include nonsense, frameshift, splice-site or missense mutations or gene deletions Subject must have measurable disease by Response Evaluation Criteria in Solid Tumors (RECIST) 1.1 criteria by computed tomography (CT) or magnetic resonance imaging (MRI) Eastern Cooperative Oncology Group (ECOG) performance status =< 1 Absolute neutrophil count (ANC) >= 1500/mm^3 Platelet count >= 100,000/mm^3 Hemoglobin >= 10 g/dL Total Bilirubin =< 1.5 x institutional upper limit of normal (ULN)  Aspartate aminotransferase (AST) (serum glutamic-oxaloacetic transaminase [SGOT])/alanine aminotransferase (ALT) (serum glutamate pyruvate transaminase [SGPT]) =< 3 x institutional ULN  Patients with liver metastases will be allowed to enroll with AST and ALT levels =< 5 x ULN Estimated creatinine clearance >= 30 mL/min by Cockcroft-Gault formula  Evidence of post-menopausal status or negative urinary or serum pregnancy test for female pre-menopausal patients. Women will be considered post-menopausal if they have been amenorrheic for 12 months without an alternative medical cause. The following age-specific requirements apply:  Women < 50 years of age would be considered post-menopausal if they have been amenorrheic for 12 months or more following cessation of exogenous hormonal treatments and if they have luteinizing hormone and follicle-stimulating hormone levels in the post-menopausal range for the institution or underwent surgical sterilization (bilateral oophorectomy or hysterectomy) Women >= 50 years of age would be considered post-menopausal if they have been amenorrheic for 12 months or more following cessation of all exogenous hormonal treatments, had radiation-induced menopause with last menses > 1 year ago, had chemotherapy-induced menopause with last menses > 1 year ago, or underwent surgical sterilization (bilateral oophorectomy, bilateral salpingectomy or hysterectomy) Highly effective contraception for both male and female subjects throughout the study and at least 4 months after last study treatment administration Recovery to baseline or =< grade 1 Common Terminology Criteria for Adverse Events (CTCAE) version (v)5.0 from toxicities related to any prior treatments, unless adverse event (AE)(s) are clinically non-significant and/or stable on supportive therapy Able to provide informed consent and willing to sign an approved consent form that conforms to federal and institutional guidelines Estimated life expectancy of at least 12 weeks  Exclusion Criteria:  Homozygous for UGT1A1*28 allele or Gilbert syndrome Subject has received systemic antineoplastic therapy (including unconjugated therapeutic antibodies and toxin immunoconjugates) or any investigational therapy =< 14 days or within 5 half-lives before starting study treatment, whichever is shorter Prior treatment with durvalumab plus tremelimumab Subject has received radiotherapy =< 14 days before the first dose of study treatment. Localized radiation therapy for the treatment of symptomatic bone metastasis is allowed during that timeframe Subjects who have undergone major surgery =< 3 weeks before starting study drug or who have not fully recovered from major surgery Diagnosis of any other malignancy within 2 years before study enrollment, except for adequately treated basal cell or squamous cell skin cancer, carcinoma in situ of the breast, bladder or of the cervix, and prostate cancer on surveillance with no plans for treatment intervention (e.g., surgery, radiation, or castration) or prostate cancer that has been adequately treated with prostatectomy or radiotherapy and currently with no evidence of disease or symptoms is allowed  Known brain metastases or cranial epidural disease  Note: Brain metastases or cranial epidural disease adequately treated with radiotherapy and/or surgery and stable for at least 4 weeks before the first dose of study treatment will be allowed on trial. Subjects must be neurologically asymptomatic and without corticosteroid treatment at the time of the first dose of study treatment  Current evidence of uncontrolled, clinically significant intercurrent illness including, but not limited to, the following conditions:  Cardiovascular disorders:  Congestive heart failure New York Heart Association Class 3 or 4, unstable angina pectoris, serious cardiac arrhythmias Stroke (including transient ischemic attack [TIA]), myocardial infarction (MI), or other ischemic events, or thromboembolic event (e.g., deep venous thrombosis, pulmonary embolism) within 3 months before the first dose  Uncontrolled hypertension defined as a sustained systolic blood pressure >= 160 mmHg or a diastolic blood pressure >= 100 mmHg despite optimal management  Note: Patients with uncontrolled hypertension who are not optimally managed may be rescreened once controlled hypertension is achieved  Patients with uncorrectable prolonged corrected QT (QTc) (Bezet formula) > 480 msec or concomitant use of medications(s) with a known risk of inducing Torsade de Pointes if such treatment cannot be discontinued or switched to a different medication before starting the study drug  Note: If a single electrocardiogram (ECG) shows a QTc with an absolute value > 480 msec, two additional ECGs approximately 2 minutes apart must be performed within 30 minutes of the initial ECG, and the average of these three consecutive results for QTc will be used Adrenal insufficiency Interstitial lung disease (ILD) Subjects with congenital long QT syndrome Patients currently on or who will require valproic acid for any medical condition during the study  Active or prior documented autoimmune or inflammatory disorders (including inflammatory bowel disease [e.g., colitis or Crohn's disease], diverticulitis [except for diverticulosis], systemic lupus erythematosus, Sarcoidosis syndrome, or Wegener syndrome [granulomatosis with polyangiitis, Graves' disease, rheumatoid arthritis, hypophysitis, uveitis, etc.]). The following are exceptions to this criterion:  Patients with vitiligo or alopecia Patients with hypothyroidism (e.g., following Hashimoto syndrome) stable on hormone replacement Any chronic skin condition that does not require systemic therapy Patients without active disease in the last 5 years may be included but only after consultation with the principal investigator Patients with celiac disease controlled by diet alone  Current or prior use of immunosuppressive medication within 14 days of cycle one day one, EXCEPT for the following permitted steroids:  Intranasal, inhaled, topical steroids, eye drops or local steroid injection (e.g., intra-articular injection); Systemic corticosteroids at physiologic doses =< 10 mg/day of prednisone or equivalent; Steroids as premedication for hypersensitivity reactions (e.g., computed tomography (CT) scan premedication) History of active primary immunodeficiency  Known human immunodeficiency virus (HIV) infection with a detectable viral load at the time of screening  Note: Patients on effective antiretroviral therapy with an undetectable viral load at the time of screening are eligible for this trial  Active infection including tuberculosis (clinical evaluation that includes clinical history, physical examination, and radiographic findings, and tuberculosis (TB) testing in line with local practice), hepatitis B (known positive hepatitis B virus [HBV] surface antigen (HBsAg) result), or hepatitis C  Note: Patients with a past or resolved HBV infection (defined as the presence of hepatitis B core antibody [anti-HBc] and absence of HBsAg) are eligible. Patients positive for hepatitis C (HCV) antibody are eligible only if polymerase chain reaction is negative for HCV ribonucleic acid (RNA) Live attenuated vaccinations within 4 weeks of cycle one day one and while on trial is prohibited Known prior severe hypersensitivity to investigational product or any component in its formulations, including known severe hypersensitivity reactions to monoclonal antibodies (National Cancer Institute [NCI] CTCAE version [v]5.0 grade >= 3) Subjects taking prohibited medications. A washout period of prohibited medications for a period of at least 5 half-lives or as clinically indicated should occur before the start of treatment