Vorinostat and Azacitidine in Treating Patients With Myelodysplastic Syndromes or Acute Myeloid Leukemia

This phase I/II trial studies the side effects and best dose of vorinostat and azacitidine and to see how well they work in treating patients with myelodysplastic syndromes or acute myeloid leukemia. Vorinostat may stop the growth of cancer or abnormal cells by blocking some of the enzymes needed for cell growth. Drugs used in chemotherapy, such as azacitidine, work in different ways to stop the growth of cancer or abnormal cells, either by killing the cells, by stopping them from dividing, or by stopping them from spreading. Giving vorinostat together with azacitidine may kill more cancer or abnormal cells.

PRIMARY OBJECTIVES:

I. To evaluate the safety and toxicity of vorinostat in combination with azacitidine in patients with myelodysplastic syndromes (MDS) and some select patients with acute myeloid leukemia (AML) (step 1).

II. To identify doses of both vorinostat and azacitidine for safe combination of the 2 agents that can be administered in repetitive cycles over time for use in phase II studies.

III. To determine the response rate of patients treated with the combination of suberoylanilide hydroxamic acid (SAHA) (vorinostat) and azacitidine at the doses established as safe and effective in Step 1 in an expanded cohort of patients with MDS.

IV. To obtain preliminary data on the effects of treatment with the combination of vorinostat and Aza C (azacitidine) in patients with MDS on a series of biologic surrogate markers including: deoxyribonucleic acid (DNA) methylation of specific genes (e.g. p15); histone acetylation; hematopoietic progenitor growth and differentiation; the fate of the MDS clone and changes in gene expression by array profiling.

SECONDARY OBJECTIVES:

I. Determine effect of treatment with the combination on time to response, time to leukemic transformation and frequency of transformation to leukemia in patients with MDS during the phase II segment of the study.

OUTLINE: This is a phase I, dose-escalation study followed by a phase II study.

Patients receive azacitidine subcutaneously (SC) once daily (QD) on days 1-7 and vorinostat orally (PO) 2-3 times daily on days 3-5, 3-9, or 3-16. Treatment repeats every 28 days for at least 4 courses in the absence of disease progression or unacceptable toxicity.

After completion of study treatment, patients are followed monthly for 6 months and then every 2 months thereafter.

Eligibility Criteria:
Inclusion Criteria:

* Eligibility Criteria for the phase I portion (step 1); patients must have a diagnosis of either MDS according to French-American-British (FAB) and International Prognostic Scoring System (IPSS) criteria, or a diagnosis of AML according to FAB or World Health Organization (WHO) criteria

  * MDS: All patients must have an established diagnosis of myelodysplastic syndrome confirmed by peripheral blood and bone marrow maturational abnormalities in the erythroid, megakaryocytic and granulocytic series, defined according to the French -American-British (F.A.B.) classification

    * Refractory anemia (RA), defined as having anemia with =\< 1% blasts in the peripheral blood and dyserythropoiesis; neutropenia and thrombocytopenia may also be present but are less common; the bone marrow is usually normocellular or hypercellular with \< 5% blast cells
    * Refractory anemia with ring sideroblasts (RARS), defined as refractory anemia above, but also including the presence of ringed sideroblasts comprising \>= 15% of all nucleated cells in the bone marrow
    * Refractory anemia with excess blasts (RAEB), defined as having 5-20% myeloblasts in the bone marrow and less than 5% blasts in the peripheral blood, together with abnormalities in erythroid megakaryocytic and granulocytic maturation consistent with myelodysplasia
    * Refractory anemia with excess blasts in transformation (RAEB-T), defined as 21-29% myeloblasts in the marrow, or more than 5% blasts in the peripheral blood, or Auer rods in granulocytic precursors in the marrow or blood (with \< 30% myeloblasts in the marrow)
    * Chronic myelomonocytic leukemia (CMMoL) defined as an absolute monocytosis (\> 1 x 10\^9/liter) with \< 5% blasts in the peripheral blood and \< 20% blasts in the bone marrow; additional criteria include a white blood cell (WBC) =\< 13 x 10\^9/L
    * For patients with refractory anemia or refractory anemia with ring sideroblasts and IPSS =\< 0.5, must also meet at least one of the following criteria to be eligible:

      * Symptomatic anemia requiring packed red blood cell (PRBC) transfusions for at least 3 months prior to study entry (document), or
      * Thrombocytopenia with platelet counts =\< 50,000/ml or significant clinical hemorrhage (e.g., gastrointestinal \[GI\], gastric ulcer \[GU\] or gynecologic \[GYN\] hemorrhage requiring platelet transfusion; petechiae alone do not constitute sufficient hemorrhage)
      * Neutropenia with absolute neutrophil count \< 1000/ul with an infection requiring treatment with antibiotics
    * Patients with MDS classified according to IPSS criteria with intermediate -1, intermediate -2 or high risk disease are eligible
    * Patients with low risk MDS (IPSS Score \< 0.5) must additionally meet criteria as outlined for patients with refractory anemia or refractory anemia with ring sideroblasts above
* AML - phase 1 only; patients with AML are not eligible for phase II; patients with AML defined according to FAB or WHO criteria will be eligible for the phase I portion of this study (phase 1 only); additional selection criteria:

  * De novo AML or AML evolving from MDS associated with intermediate or poor risk cytogenetics in patients who decline standard chemotherapy or who have failed or relapsed following one prior regimen
  * Additionally, patient's disease must be stable i.e. WBC =\< 25 x 10\^9/L who have exhibited stable WBC not requiring intervention for WBC control with hydroxyurea, chemotherapy or leukophoresis for \> 4 weeks
* No corticosteroids, interferon or retinoids within one month prior to study
* No treatment with granulocyte colony-stimulating factor (G-CSF), granulocyte-macrophage colony-stimulating factor (GM-CSF), or other hematopoietic growth factors, e.g. erythropoietin within one month prior to study
* No prior treatment with azacitidine, decitabine or vorinostat
* Patients should not have taken valproic acid, or any other histone deacetylase inhibitor, for at least 2 weeks prior to enrollment
* Patients may not have been treated with an investigational agent in the 28 days prior to study entry and must have recovered from all side effects
* Patients previously treated for cancer other than leukemia with chemotherapy or radiation therapy may be eligible; they may not have received radiation or chemotherapy within the past 12 months and must have been free of any evidence of the malignancy during the past 3 years; patients with a prior history of leukemia who relapse with MDS are ineligible
* Eastern Cooperative Oncology Group (ECOG) performance status =\< 2 (Karnofsky \>= 60%)
* Life expectancy of greater than 2 months
* Total bilirubin =\< 1.5 x upper limit of normal (ULN); unless the patient has active hemolysis, or the elevation is secondary to ineffective erythropoiesis
* Aspartate aminotransferase (AST) (serum glutamic oxaloacetic transaminase \[SGOT\])/alanine aminotransferase (ALT) (serum glutamate pyruvate transaminase \[SGPT\]) =\< 2.5 X institutional ULN
* Creatinine within normal institutional limits OR creatinine clearance \>= 60 mL/min/1.73 m\^2 for patients with creatinine levels above institutional normal
* Treatment must begin within 2 to 4 weeks of completing prestudy tests
* Women of child-bearing potential and men must agree to use adequate contraception (hormonal or barrier method of birth control; abstinence) prior to study entry and for the duration of study participation; should a woman become pregnant or suspect she is pregnant while participating in this study, she should inform her treating physician immediately
* Ability to understand and the willingness to sign a written informed consent document

Exclusion Criteria:

* Patients who have had chemotherapy or radiotherapy within 12 months prior to entering the study or those with another malignancy who have had evidence of the malignancy during the 3 years prior to study entry
* Patients may not be receiving any other investigational agents
* Patients may not have central nervous system (CNS) involvement with MDS or AML
* History of allergic reactions attributed to compounds of similar chemical or biologic composition to vorinostat or other agents used in study
* Pregnant women are excluded from this study; breastfeeding should be discontinued if the mother is treated with vorinostat
* Any other serious medical or psychiatric illness which would limit survival to \< 3 months or prevent the granting of informed consent
* Uncontrolled or severe congestive heart failure; (does not include patients with high output congestive heart failure \[CHF\] states secondary to anemia which can be controlled)
* \>= 30% myeloblasts in the bone marrow or M6 leukemia as defined by FAB criteria (phase II component of the study)
* Prior treatment with G-CSF, GM-CSF, or other hematopoietic growth factors, erythropoietin within one month prior to study; no interferon or retinoids within one month prior to study
* Known positive serology for human immunodeficiency virus (HIV); appropriate studies will be undertaken in patients receiving combination anti-retroviral therapy when indicated
* Active systemic bacterial, fungal of viral infection; infection should be fully treated and the patient afebrile for 7 days before study entry
* Prior history of leukemia (step 2 phase II component of the study only)
* Uncontrolled intercurrent illness including, but not limited to, ongoing or active infection, symptomatic congestive heart failure, unstable angina pectoris, clinically significant cardiac arrhythmia, or psychiatric illness/social situations that would limit compliance with study requirements
* Patients with advanced hepatic tumors are excluded