Wee1 Kinase Inhibitor AZD1775 and Combination Chemotherapy in Treating Children, Adolescents and Young Adults With Relapsed or Refractory Acute Myeloid Leukemia

This phase I trial studies the side effects and best dose of wee1 kinase inhibitor AZD1775 when given together with fludarabine, cytarabine, and filgrastim (FLAG) combination chemotherapy in treating children, adolescents and young adults with relapsed or refractory acute myeloid leukemia. Wee1 kinase inhibitor AZD1775 may help combination chemotherapy work better by making tumor cells more sensitive to the drugs. Drugs used in chemotherapy, such as fludarabine and cytarabine, may prevent tumor cells from multiplying by damaging their deoxyribonucleic acid (DNA), which in turn stops the tumor from growing. Giving wee1 kinase inhibitor AZD1775 and FLAG chemotherapy may work better in treating patients with acute myeloid leukemia.

PRIMARY OBJECTIVES:

I. To estimate the maximum tolerated dose (MTD) and/or recommended phase 2 dose of wee1 kinase inhibitor AZD1775 (AZD1775) administered orally daily for 5 days in combination with FLAG (fludarabine, cytarabine, filgrastim) chemotherapy in children and adolescents with recurrent or refractory acute myeloid leukemia (AML).

II. To define and describe the toxicities of AZD1775 in combination with FLAG chemotherapy administered on this schedule.

III. To characterize the pharmacokinetics AZD1775 in combination with FLAG chemotherapy in pediatric patients with relapsed or refractory AML.

SECONDARY OBJECTIVES:

I. To preliminarily define the antitumor activity of AZD1775 plus FLAG within the confines of a phase 1 study.

II. To evaluate pharmacodynamic biomarkers of wee1 inhibition with administration of AZD1775 including cyclin-dependent kinase 1 (CDC2) phosphorylation at tyrosine 15 (Tyr15) and induction of gamma H2AX in pre- and post-treatment leukemic blasts.

III. To explore mechanisms of apoptosis induction and cell death in leukemic blasts treated with AZD1775 plus chemotherapy.

IV. To identify genomic alterations in leukemic blasts that may correlate with response to therapy with AZD1775 plus FLAG chemotherapy.

OULTINE: This is a phase I dose escalation study of wee1 kinase inhibitor AZD1775.

Patients receive filgrastim intravenously (IV) or subcutaneously (SC) daily, fludarabine intravenously IV over 30 minutes, cytarabine IV over 1-3 hours and wee1 kinase inhibitor AZD1775 orally (PO) on days 1-5. Patients who meet criteria for complete remission (CR), complete remission with partial recovery of platelet count (CRp) or partial response (PR) may receive a second course of therapy. Courses repeat every 28 days for up to 2 courses in the absence of disease progression or unacceptable toxicity.

After completion of study treatment, patients are followed up periodically.

Eligibility Criteria:
Inclusion Criteria:

* Patients must have had histologic verification of AML at original diagnosis; AML is defined according to World Health Organization (WHO) classification with \>= 5% blasts in the bone marrow (M2/M3 bone marrow), with or without extramedullary disease
* Relapsed patients:

  * Second or greater relapse OR
  * AML in first relapse AND has received \>= 450 mg/m\^2 daunorubicin equivalents
  * NOTE: for the purposes of determining eligibility for this protocol, the following cardiotoxicity multipliers will be used to determine daunorubicin equivalents:

    * Doxorubicin: 1
    * Mitoxantrone: 3
    * Idarubicin: 3
    * Epirubicin: 0.5
* Refractory patients:

  * Patients must have received at least two attempts at remission induction, which may consist of up to two different therapy courses; Children's Oncology Group (COG) AAML1031 de novo therapy including induction I and induction II is one remission attempt
* Patients must have the status of central nervous system (CNS)1 or CNS2 only, and no clinical signs or neurologic symptoms suggestive of CNS leukemia, such as cranial palsy
* Patients must have a body surface area \>= 0.35 m\^2 at the time of study enrollment
* Patient's current disease state must be one for which there is no known curative therapy or therapy proven to prolong survival with an acceptable quality of life
* Karnofsky \>= 50% for patients \> 16 years of age and Lansky \>= 50 for patients =\< 16 years of age; patients who are unable to walk because of paralysis, but who are up in a wheelchair, will be considered ambulatory for the purpose of assessing the performance score
* Patients must have recovered from the acute toxic effects of all prior chemotherapy, immunotherapy, stem cell transplant or radiotherapy prior to entering this study; all prior treatment-related toxicities must have resolved to =\< grade 2 or meet the inclusion/exclusion criteria prior to enrollment

  * Myelosuppressive chemotherapy:

    * Must be 14 days after the last dose of myelosuppressive chemotherapy (excluding hydroxyurea)
    * Cytoreduction with hydroxyurea can be initiated and continued for up to 24 hours prior to the start of therapy
  * Intrathecal cytotoxic therapy:

    * No waiting period is required for patients having received intrathecal cytarabine, methotrexate, and/or hydrocortisone
    * At least 14 days must have elapsed since receiving liposomal cytarabine (DepoCyte) by intrathecal injection
    * Intrathecal cytarabine given at the time of diagnostic lumbar puncture (LP) to evaluate for relapse prior to study enrollment is allowed
  * Biologic (anti-neoplastic agent):

    * At least 7 days since the completion of therapy with a biologic agent such retinoids, or DLI (donor lymphocyte infusion without conditioning)
    * Note: for agents that have known adverse events occurring beyond 7 days after administration, this period must be extended beyond the time during which acute adverse events are known to occur
  * Interleukins, Interferons and Cytokines (other than hematopoietic growth factors):

    * \>= 21 days after the completion of interleukins, interferon or cytokines (other than hematopoietic growth factors)
  * Antibodies:

    * \>= 21 days must have elapsed from infusion of last dose of antibody, and toxicity related to prior antibody therapy must be recovered to grade =\< 1
  * Radiation therapy (RT):

    * At least 14 days after local palliative x-ray telescope (XRT) (small port); at least 84 days must have elapsed if prior traumatic brain injury (TBI), craniospinal XRT or if \>= 50% radiation of pelvis; at least 42 days must have elapsed if other substantial bone marrow (BM) radiation
  * Stem Cell Transplant (SCT) without TBI:

    * No evidence of active graft vs. host disease and at least 84 days must have elapsed after transplant or stem cell infusion
    * Patients must be off all systemic immunosuppressive therapy for at least 2 weeks, excluding hydrocortisone for physiologic cortisol replacement
  * Growth factors:

    * At least 14 days after the last dose of a long-acting growth factor (e.g. Neulasta) or 7 days for short-acting growth factor; for agents that have known adverse events occurring beyond 7 days after administration, this period must be extended beyond the time during which adverse events are known to occur; the duration of this interval must be discussed with the study chair
  * Patients must not have received prior exposure to AZD1775
* Platelet count \>= 20,000/mm\^3 (may receive platelet transfusions); these patients must not be known to be refractory to red cell or platelet transfusion
* Creatinine clearance or radioisotope glomerular filtration rate (GFR) \>= 70 mL/min/1.73 m\^2 or a serum creatinine based on age/gender as follows:

  * Age: maximum serum creatinine (mg/dL) 1 to \< 2 years: 0.6 (male), 0.6 (female) 2 to \< 6 years: 0.8 (male), 0.8 (female) 6 to \< 10 years: 1 (male), 1 (female) 10 to \< 13 years: 1.2 (male), 1.2 (female) 13 to \< 16 years: 1.5 (male), 1.4 (female) \>= 16 years: 1.7 (male), 1.4 (female)
* Bilirubin (sum of conjugated + unconjugated) =\< 1.5 x upper limit of normal (ULN) for age
* Serum glutamic pyruvic transaminase (SGPT) (alanine aminotransaminase \[ALT\]) =\< 225 units per liter (U/L); for the purpose of this study, the ULN for SGPT is 45 U/L
* Serum albumin \>= 2 g/dL
* All patients and/or their parents or legally authorized representatives must sign a written informed consent; assent, when appropriate, will be obtained according to institutional guidelines
* Specimens for cytogenetic analysis are required, and must be obtained prior to therapy initiation; for patient with refractory disease, the diagnostic specimen may be used

Exclusion Criteria:

* Pregnancy or breast-feeding:

  * Pregnant or breast-feeding women may not be entered on this study; pregnancy tests must be obtained in girls who are post-menarchal
  * Males or females of reproductive potential may not participate unless they have agreed to use an effective double barrier contraceptive method for the entire duration of protocol therapy
* Corticosteroids should not be used as anti-emetic therapy; corticosteroid therapy is not permissible except for the following indications:

  * As treatment or prophylaxis for anaphylactic reactions
  * As a treatment for symptoms of cytarabine (Ara-C) syndrome (including fever, rash, or conjunctivitis)
  * Physiologic replacement stress-dosing as indicated for suspected or confirmed adrenal insufficiency
* Patients who are currently receiving another investigational drug are not eligible
* Patients who are currently receiving other anti-cancer agents are not eligible except for hydroxyurea (which may be continued until 24 hours prior to start of protocol therapy)
* Anti-graft versus host disease (GVHD) agents post-transplant:

  * Patients who are receiving cyclosporine, tacrolimus or other agents to prevent graft-versus-host disease post bone marrow transplant are not eligible for this trial
  * Patients must be off all systemic immunosuppressive therapy for at least 2 weeks, excluding hydrocortisone for physiologic cortisol replacement
* Patients who are currently receiving drugs that are strong or moderate inhibitors and/or inducers of cytochrome P450 family 3 subfamily A polypeptide 4 (CYP3A4) or sensitive CYP3A4 substrates and CYP3A4 substrates with a narrow therapeutic range are not eligible; the use of aprepitant as an antiemetic is prohibited due to early drug interaction data demonstrating increased exposure to AZD1775; caution should be exercised with concomitant administration of AZD1775 and agents that are sensitive substrates of cytochrome P450 family 2 subfamily C polypeptide 8 (CYP2C8), family 2 subfamily C polypeptide 9 (2C9) and family 2 subfamily C polypeptide 19 (2C19), or substrates of this enzyme with narrow therapeutic ranges, as well as agents that are inhibitors or substrates of P-glycoprotein (P-gp)
* Patients must not have received enzyme inducing anticonvulsants for at least 14 days prior to enrollment
* Patients must be able to swallow capsules whole; nasogastric or gastrostomy (G) tube administration is not allowed
* Patients who have an uncontrolled infection are not eligible
* Patients with any of the following diagnoses are not eligible:

  * Acute promyelocytic leukemia (APL)
  * Down syndrome
  * Fanconi anemia, Kostmann syndrome, Shwachman syndrome or any other known bone marrow failure syndrome
  * Wilson's disease and any other disorder of copper metabolism
  * Juvenile myelomonocytic leukemia (JMML)
* Patients who have received a prior solid organ transplantation are not eligible
* Patients who in the opinion of the investigator may not be able to comply with the safety monitoring requirements of the study are not eligible