Inclusion Criteria:  Patients must have histologically confirmed rare solid tumors that have progressed on standard therapy or for whom there is no longer standard of care therapy. Other rare tumor types may be acceptable at the discretion of the principal investigator (PI)  Patients must not be eligible for a higher priority study, such as a disease specific study of phase 2 or higher or a randomized study. Specifically, patients with pheochromocytoma and paraganglioma at the clinical center will be eligible for this study if they are not eligible for the NCT04394858 due to prior PARP inhibitor, dacarbazine (DTIC) or temozolomide therapy Patients must have measurable disease as defined by Response Evaluation Criteria in Solid Tumors (RECIST) version (v)1.1, with at least one lesion that can be accurately measured in at least one dimension (longest diameter to be recorded for non-nodal lesions and short axis for nodal lesions) Patients consenting to biopsies must have a tumor site amenable to biopsy, and this needs to be a lesion separate to those considered for RECIST measurable lesions  Prior to entering the study, patients must have:  >= 3 weeks since completion of radiation therapy or major surgery  >= 5 half-lives or 3 weeks (whichever is shorter) since completion of biologic therapy or chemotherapy  Should be at least 6 weeks out from nitrosoureas and mitomycin C >= 2 weeks since any prior administration of a study drug in a Phase 0 or equivalent study >= 1 week from palliative radiation therapy (patients on study may be eligible for palliative radiotherapy to non-targeted lesions after 2 cycles of therapy at the PI's discretion) recovered to eligibility levels from prior toxicity or adverse events. Treatment with bisphosphonates is permitted Adults age >= 18 years; children/adolescents age >= 12 years to 17 years with body surface area (BSA) >= 1.5 m^2 Eastern Cooperative Oncology Group (ECOG) performance status =< 2 (Karnofsky >= 60%) Absolute neutrophil count >= 1,500/mcL Platelets >= 100,000/mcL Hemoglobin >= 8 g/dL Total bilirubin =< 1.5 X institutional upper limit of normal (=< 3 x upper limit of normal in the presence of documented Gilbert's syndrome or liver metastases at baseline) Aspartate aminotransferase (AST) (serum glutamic-oxaloacetic transaminase [SGOT])/alanine aminotransferase (ALT) (serum glutamate pyruvate transaminase [SGPT]) =< 3 X upper limit of normal (ULN) For adult patients (> 18 years of age): =< 1.5 X institutional ULN OR creatinine clearance >= 60 mL/min/1.73 m^2 for patients with creatinine levels > 1.5 mg/dL  For pediatric patients (< 18 years of age), a serum creatinine based on age and gender as follows:  Age: 2 to < 6 years; Maximum serum creatinine (mg/dL): 0.8 (male); 0.8 (female) Age: 6 to < 10 years; Maximum serum creatinine (mg/dL): 1 (male); 1 (female) Age: 10 to < 13 years; Maximum serum creatinine (mg/dL): 1.2 (male); 1.2 (female) Age: 13 to < 16 years; Maximum serum creatinine (mg/dL): 1.5 (male); 1.4 (female) Age: 16 to < 18 years; Maximum serum creatinine (mg/dL): 1.7 (male); 1.4 (female) The threshold creatinine values in this table were derived from the Schwartz formula for estimating glomerular giltration rate (GFR), utilizing child length and stature data published by the Center for Disease Control and Prevention (CDC) Talazoparib and temozolomide can cause fetal harm based on animal reproductive and genetic toxicity studies. For this reason, women of child-bearing potential and men must agree to use adequate contraception (hormonal or barrier method of birth control; abstinence) prior to study entry and for the duration of study participation and for at least 7 months after dosing with study drugs ceases. Should a woman become pregnant or suspect she is pregnant while she or her partner is participating in this study, she should inform her treating physician immediately. Men treated or enrolled on this protocol must also agree to use adequate contraception prior to the study, for the duration of study participation, and 7 months after completion of study drug administration  Biopsies are optional on this study. In lieu of baseline biopsies, patients are encouraged to submit at registration archival tumor biopsy tissue from a previous research study or medical care providing it meets the minimum collection and preservations requirements. Criteria for the submission of archival tissue are:  Tissue must have been collected within 3 months prior to registration Patient must not have received any intervening therapy for their cancer since the collection of the tumor sample Human immunodeficiency virus (HIV)-infected patients on effective anti-retroviral therapy with undetectable viral load within 6 months are eligible for this trial. For these patients, an HIV viral load test must be completed within 28 days prior to enrollment Patients with a prior or concurrent malignancy whose natural history or treatment does not have the potential to interfere with the safety or efficacy assessment of the investigational regimen are eligible for this trial Patients with known history or current symptoms of cardiac disease, or history of treatment with cardiotoxic agents, should have a clinical risk assessment of cardiac function using the New York Heart Association Functional Classification. To be eligible for this trial, patients should be class 2B or better Ability to understand and the willingness to sign a written informed consent document  Exclusion Criteria:  Sensory/motor neuropathy >= grade 2 Patients who are receiving any other investigational agents Patients with active brain metastases or carcinomatous meningitis are excluded from this clinical trial. Patients with treated brain metastases, whose brain metastatic disease has remained stable for >= 1 month without requiring steroid and anti-seizure medication are eligible to participate History of allergic reactions attributed to compounds of similar chemical composition to study drugs  Known clinically significant liver disease, including active viral, alcoholic, or other hepatitis; cirrhosis; fatty liver; and inherited liver disease  Patients with past or resolved hepatitis B infection (defined as having a negative hepatitis B surface antigen [HBsAg] test and a positive anti-HBc [antibody to hepatitis B core antigen] antibody test) are eligible. For these patients, HBsAg and anti-HBc tests must be done within 28 days prior to enrollment Patients positive for hepatitis C virus (HCV) antibody are eligible only if polymerase chain reaction (PCR) is negative for HCV ribonucleic acid (RNA). For these patients, an HCV RNA test must be done within 28 days prior to enrollment Uncontrolled intercurrent illness including, but not limited to, serious untreated infection, symptomatic respiratory failure/congestive heart failure, unstable angina pectoris, cardiac arrhythmia, or psychiatric illness/social situations that would limit compliance with study requirements Pregnant women are excluded from this study because temozolomide and talazoparib have demonstrated fetal harm in animal reproductive studies. Because there is an unknown but potential risk for adverse events in nursing infants secondary to treatment of the mother with the study drugs, breastfeeding should be discontinued prior to the first dose of study drug and women should refrain from nursing throughout the treatment period and for 1 months following the last dose of study drug