BMS-986205 and Nivolumab as First or Second Line Therapy in Treating Patients With Liver Cancer

This phase I/II trial studies the side effects and best dose of IDO1 inhibitor BMS-986205 (BMS-986205) when given together with nivolumab and how well it works as first or second line therapy in treating patients with liver cancer. BMS-986205 may stop the growth of tumor cells by blocking some of the enzymes needed for cell growth. Monoclonal antibodies, such as nivolumab, may interfere with the ability of tumor cells to grow and spread. Giving BMS-986205 and nivolumab may work better in treating patients with liver cancer.

PRIMARY OBJECTIVES:

I. To obtain the safety and tolerability of BMS-986205 in combination with nivolumab in unresectable / metastatic hepatocellular carcinoma (HCC) in the first or second line setting using Common Terminology Criteria for Adverse Events (CTCAE) version (V)5.0 criteria.

II. To determine efficacy as defined by objective response rate (ORR) of BMS-986205 in combination with nivolumab in unresectable / metastatic HCC in the first or second line setting using Response Evaluation Criteria in Solid Tumors (RECIST) (version 1.1).

SECONDARY OBJECTIVES:

I. To determine disease control rate (DCR), duration of response (DOR), progression free survival (PFS), and overall survival (OS) by RECIST 1.1 and ORR using immune RECIST (iRECIST) of BMS-986205 in combination with nivolumab in unresectable HCC. (Phase II) II. To further evaluate safety of BMS-986205 in combination with nivolumab in unresectable HCC. (Phase II)

OUTLINE: This is a phase I, dose-escalation study of IDO1 inhibitor BMS-986205 followed by a phase II study.

Patients receive IDO1 inhibitor BMS-986205 orally (PO) once daily (QD) on days 1-14 and nivolumab intravenously (IV) over 30 minutes on day 1. Courses repeat every 14 days in the absence of disease progression or unacceptable toxicity.

After completion of study treatment, patients are followed up for 100 days, and then every 3 months thereafter.

Eligibility Criteria:
Inclusion Criteria:

* Willing and able to provide written informed consent for the trial
* Life expectancy \> 12 weeks
* Histologically or imaging confirmed hepatocellular carcinoma (mixed hepatocellular/cholangiocarcinoma or fibrolamellar subtypes are excluded)
* Have disease that is not amenable for curative treatment approach
* Have measurable disease based on RECIST v1.1
* \>= 1 liver lesions accessible for core biopsy that was either not previously treated by liver-directed therapy or progressed following liver-directed therapy
* Child-Pugh score of A
* Eastern Cooperative Oncology Group (ECOG) performance status of 0 or 1
* Absolute neutrophil count (ANC) \>= 1000 cell/mm\^3
* Platelet count \>= 50,000/mm\^3
* Hemoglobin (Hgb) \>= 8 g/dL
* Aspartate aminotransferase (AST)/serum glutamic oxaloacetic transaminase (SGOT), alanine aminotransferase (ALT)/serum glutamic pyruvic transaminase (SGPT) =\< 5 x upper limit of normal (ULN)
* Total bilirubin =\< 2 ULN
* Creatinine =\< 2 x ULN
* Subjects with active hepatitis B virus (hep B) are allowed if antiviral therapy for hepatitis B has been given for \> 8 weeks and viral load is \< 100 IU/ml prior to first dose of trial treatment. Subjects with untreated hepatitis C virus (HCV) are allowed
* Willingness to undergo mandatory pre-treatment biopsy (unless there is adequate archival tumor specimen available) and mandatory on-treatment biopsy
* Female subject of child-bearing potential must have a negative urine pregnancy =\< 24 hour (hr) prior to planned treatment initiation. Women with childbearing potential and males must be willing to use adequate birth control on trial and until 5 months for women or 7 months for men after the last of study therapy
* Ability to adhere to the study visit schedule and other protocol requirements
* Participants must be able to swallow pills intact

Exclusion Criteria:

* Received more than 1 prior systemic HCC-related therapy or currently receiving HCC-related systemic treatment or participating in a clinical trial and receiving study therapy
* Known history of positive test for human immunodeficiency virus (HIV) or known acquired immunodeficiency syndrome (AIDS)
* Known diagnosis of immunodeficiency or active autoimmune disease or requiring systemic steroid equivalent of prednisone \>= 10 mg/day or any immunosuppressive therapies =\< 7 days of before the first dose of the study
* Active bacterial, viral (except hepatitis B and C), or fungal infection(s) requiring systemic therapy, defined as ongoing signs/symptoms related to the infection without improvement despite appropriate antibiotics, anti-viral therapy, anti-fungal therapy, and/or other treatment
* Active pneumonitis or history of interstitial lung disease (ILD) / pneumonitis requiring steroids
* Clinically significant ascites
* Hepatic encephalopathy
* Any significant medical condition including additional malignancies, laboratory abnormalities, or psychiatric illness that would prevent the subject from participating and adhering to study related procedures
* Live attenuated vaccine =\< 30 days before the first dose of study treatment. Examples of live vaccines include, but are not limited to, the following: measles, mumps, rubella, chicken pox, yellow fever, rabies, Bacillus Calmette-Guerin (BCG), and typhoid vaccine. Seasonal influenza vaccines for injection are generally killed virus vaccines and are allowed; however, intranasal influenza vaccines (e.g., FluMist) are live attenuated vaccines and are not allowed
* Use of strong inhibitor / inducer of CYP3A4 or CYP1A2
* Known history of surgery or medical condition that may affect drug absorption, per investigator descretion
* Participants with a history of G6PD deficiency or other congenital or autoimmune hemolytic disorders. All participants will be screened for G6PD deficiency prior to enrollment using quantitative or qualitative G6PD assay results to suggest underlying G6PD deficiency
* Participants with a personal or family (i.e., in a first-degree relative) history or presence of cytochrome b5 reductase deficiency (previously called methemoglobin reductase deficiency) or other diseases that puts them at risk of methemoglobinemia. All participants will be screened for methemoglobin levels prior to enrollment using blood methemoglobin \> ULN, assessed in an arterial or venous blood sample or by co oximetry
* Subjects with screening corrected QT (QTc) interval \> 480 ms
* Liver directed therapy =\< 4 weeks before the first dose of study
* History of esophageal or gastric variceal bleeding within 3 months of study enrollment
* Treatment with botanical preparations (e.g., herbal supplements or traditional Chinese medicines) intended for general health support or to treat the disease under study within 2 weeks prior to enrollment
* Prior history of serotonin syndrome
* Prior treatment with BMS-986205 or any other IDO1 inhibitors.
* Women who are breastfeeding
* History or presence of hypersensitivity or idiosyncratic reaction to methylene blue
* History of allergy or hypersensitivity to any study treatment components, specifically to that of BMS-986205
* Participants who have had major surgery requiring general anesthesia or significant trauma who have not recovered per physician determination for at least 14 days prior to enrollment
* Participants who have had major surgery requiring general anesthesia or significant trauma who have not recovered per physician determination for at least 14 days prior to enrollment
* Participants with uncontrolled adrenal insufficiency