Navitoclax, Venetoclax, and Decitabine for the Treatment of Relapsed or Refractory Acute Myeloid Leukemia Previously Treated With Venetoclax

This phase Ib trial is to find the side effect and best dose of navitoclax when given together with venetoclax and decitabine in treating patients with acute myeloid leukemia that has come back (relapsed) or does not respond to treatment (refractory) after previous treatment with venetoclax. Chemotherapy drugs, such as navitoclax, venetoclax, and decitabine, work in different ways to stop the growth of cancer cells, either by killing the cells, by stopping them from dividing, or by stopping them from spreading.

PRIMARY OBJECTIVES:

I. Assess safety and determine the optimal biological doses (OBD). II. Evaluate the anti-leukemic activity, as assessed by overall response rate (ORR: complete response \[CR\] + complete response with incomplete hematopoiesis \[CRi\] + partial response \[PR\]) within the first 35 days (cycle 1).

SECONDARY OBJECTIVES:

I. Characterize safety and tolerability of navitoclax/venetoclax during the first 7 days of cycle 1.

II. Obtain preliminary estimates of:

IIa. Overall response (CR+CRi+PR) duration. IIb. Progression free survival (PFS) and overall survival (OS) rates at 6-months and 1-year.

CORRELATIVE STUDY OBJECTIVES:

I. Describe the anti-leukemic activity (i.e. decrease in percent blasts) of navitoclax/venetoclax as assessed by disease in bone marrow (BM) and/or peripheral blood (PB).

II. Explore the potential relationship between changes in BH3 protein expression profiles (pre- and post-treatment) and response.

III. Profile the clonal evolution of acute myeloid leukemia (AML) in response to treatment by single cell next generation sequencing (NGS) of whole exomes and the ribonucleic acid (RNA) transcriptome.

IV. Profile the clonal evolution of AML in response to treatment by single cell NGS of the mitochondrial genome.

V. Determine mitochondrial number and heteroplasmy and changes in response to treatment.

VI. Determine the impact of treatment on leukemic stem cell (LSC) burden by examining bone marrow (BM)-derived mononuclear cells (MNC) for their ability to initiate leukemia in in vivo mouse models pre- and post-treatment.

OUTLINE: This is a dose-escalation study of navitoclax.

Patients receive venetoclax orally (PO) once daily (QD) and navitoclax PO QD on days 1-35, and decitabine intravenously (IV) over 1 hour on days 8-12 of cycle 1. Starting on cycle 2, patients receive venetoclax PO QD and navitoclax PO QD on days 1-28, and decitabine IV over 1 hour on days 1-5. Treatment repeats every 28 days for up to 12 cycles in the absence of disease progression or unacceptable toxicity.

After completion of study treatment, patients are followed up for 1 year.

Eligibility Criteria:
Inclusion Criteria:

* Documented informed consent of the participant and/or legally authorized representative
* For participants under the age of 18 years, documentation of adolescent assent by the participant and consent of both parents or guardian
* Adults aged \>= 18 years
* Adolescent patients aged \>= 16 years and \< 18 years weighing at least 45 kg who have no other standard-of-care option for treatment
* Eastern Cooperative Oncology Group (ECOG) =\< 2
* Patients with histologically confirmed AML, according to World Health Organization (WHO) criteria, with refractory/relapsed (R/R) disease following a venetoclax-containing regimen who are ineligible for therapies known to be effective for treatment of their AML.

  * Patients with extramedullary disease may be included if they also have marrow involvement
  * Patients with acute promyelocytic leukemia (APL) will not be eligible
* Fully recovered from the acute toxic effects (except alopecia) to =\< grade 1 of prior anti-cancer therapy
* Ability to swallow pills
* Absolute neutrophil count (ANC) \>= 750/mm\^3 (performed within 14 days prior to day 1 of protocol therapy unless otherwise stated)

  * NOTE: Growth factor is not permitted within 14 days of ANC assessment unless cytopenia is secondary to disease involvement
* White blood cell (WBC) =\< 25 x 10\^9/L prior to initiation of study therapy. Cytoreduction with hydroxyurea prior to treatment and/or during cycle 1 may be required (performed within 14 days prior to day 1 of protocol therapy unless otherwise stated)
* Platelets \>= 75,000/mm\^3

  * NOTE: Platelet transfusions are not permitted within 14 days of platelet assessment unless cytopenia is secondary to disease involvement
* Total bilirubin =\< 1.5 X upper limit of normal (ULN) (performed within 14 days prior to day 1 of protocol therapy unless otherwise stated)
* Aspartate aminotransferase (AST) =\< 3.0 x ULN (performed within 14 days prior to day 1 of protocol therapy unless otherwise stated)
* Alanine aminotransferase (ALT) =\< 3.0 x ULN (performed within 14 days prior to day 1 of protocol therapy unless otherwise stated)
* Creatinine clearance of \>= 45 ml/min per 24-hour urine test or the Cockcroft-Gault formula (performed within 14 days prior to day 1 of protocol therapy unless otherwise stated)
* If in the absence of anticoagulants: International normalized ratio (INR) OR prothrombin (PT) =\< 1.5 x ULN (performed within 14 days prior to day 1 of protocol therapy unless otherwise stated)
* If in the absence of anticoagulants: Activated partial thromboplastin time (aPTT) =\< 1.5 x ULN (performed within 14 days prior to day 1 of protocol therapy unless otherwise stated)
* Left ventricular ejection fraction (LVEF) \>= 50%

  * Note: To be performed within 28 days prior to day 1 of protocol therapy
* Corrected QT interval (QTc) =\< 480 ms

  * Note: To be performed within 28 days prior to day 1 of protocol therapy
* Seronegative for human immunodeficiency virus (HIV) antigen (Ag)/antibody (Ab) combo, hepatitis C virus (HCV), active hepatitis B virus (HBV) (surface antigen negative), and syphilis (RPR) (performed within 14 days prior to day 1 of protocol therapy unless otherwise stated)

  * If positive, Hepatitis C RNA quantitation must be performed
* Meets other institutional and federal requirements for infectious disease titer requirements

  * Note Infectious disease testing to be performed within 28 days prior to day 1 of protocol therapy
* Women of child-bearing potential (WOCBP): negative urine or serum pregnancy test. If the urine test is positive or cannot be confirmed as negative, a serum pregnancy test will be required (performed within 14 days prior to day 1 of protocol therapy unless otherwise stated)
* Agreement by females and males of childbearing potential to use an effective method of birth control or abstain from heterosexual activity for the course of the study through at least 3 months (males) and 6 months (females) after the last dose of protocol therapy

  * Childbearing potential defined as not being surgically sterilized (men and women) or have not been free from menses for \> 1 year (women only)

Exclusion Criteria:

* Hematopoietic stem cell transplant within 100 days prior to day 1 of protocol therapy
* Chemotherapy, radiation therapy, biological therapy, or immunotherapy within 14 days or 5 half-lives, whichever is shorter, prior to day 1 of protocol therapy with the following exceptions:

  * Subjects will be allowed to have been on venetoclax at screening and remain on it through treatment start.
  * Hydroxyurea is allowed prior to treatment and through cycle 1 for control of rapidly progressing leukemia
* Strong or moderate CYP3A4 inducers within 14 days prior to day 1 of protocol therapy
* Grapefruit, grapefruit products, Seville oranges (including marmalade containing Seville oranges) or star fruit consumed within 3 days prior to the first dose of study drug
* Immunosuppressants (steroids =\< 10 mg/day of oral prednisone or equivalent is allowed) within the last 28 days
* Hematopoietic growth factors in the last 14 days
* Must not have received or planning to receive live vaccine while being on study or 4 weeks before and after completion of treatment
* Herbal medications known to affect platelet function within 14 days of therapy initiation
* History of allergic reactions attributed to compounds of similar chemical or biologic composition to study agent
* Active graft-versus-host-disease (GVHD)
* Active central nervous system (CNS) disease
* No measurable disease in the bone marrow
* Active diarrhea
* Gastrointestinal disorder that interferes with oral drug absorption such as malabsorption syndrome
* Clinically significant cardiac morbidities (class III/IV cardiovascular disability according to the New York Heart Association classification, arrhythmia not stable on medical management, acute cardiovascular ischemic event within 6 months of enrollment, etc.)
* Clinically significant uncontrolled illness
* Active infection requiring antibiotics
* Active/uncontrolled HIV infection, acquired immunodeficiency syndrome (AIDS), or currently taking contraindicated medications for HIV control
* Diagnosis of Gilbert's disease
* Any other active malignancy at time of enrollment. Exceptions include basal/squamous cell carcinoma, in situ adequately treated breast and uterine cancer
* Females only: Pregnant or breastfeeding
* Any other condition that would, in the Investigator's judgment, contraindicate the patient's participation in the clinical study due to safety concerns with clinical study procedures
* Prospective participants who, in the opinion of the investigator, may not be able to comply with all study procedures (including compliance issues related to feasibility/logistics)