Inclusion Criteria:  REGISTRATION - INCLUSION CRITERIA Age >= 18 years Histologically confirmed diagnosis of recurrent or metastatic salivary gland cancer not amenable to curative-intent therapy  Measurable disease as defined by Response Evaluation Criteria in Solid Tumors (RECIST) version (v)1.1 criteria  NOTE: Tumor lesions in a previously irradiated area are considered measurable disease if progression has been demonstrated in such lesions. Disease that is measurable by physical examination only is not eligible  Prior treatment:  Prior treatment with checkpoint inhibitor(s) allowed Any number of lines of prior therapy in the recurrent/metastatic setting is permitted at the investigator's discretion  Eastern Cooperative Oncology Group (ECOG) performance status (PS) 0 or 1  NOTE: PS must be assessed again within 7 days prior to first dose of study drug  Hemoglobin >= 9.0 g/dL (obtained =< 8 days prior to registration)  NOTE: Must be met without growth factor support and no transfusions <14 days prior to testing Absolute neutrophil count (ANC) >= 1500/mm^3 (obtained =< 8 days prior to registration) Platelet count >= 100,000/mm^3 (obtained =< 8 days prior to registration) Total bilirubin =< 1.5 x upper limit of normal (ULN) (obtained =< 8 days prior to registration) Alanine aminotransferase (ALT) and aspartate transaminase (AST) =< 2.5 x ULN (=< 5 x ULN for patients with liver involvement) (obtained =< 8 days prior to registration) Prothrombin time (PT)/international normalized ratio (INR)/activated partial thromboplastin time (aPTT) =< 1.5 x ULN OR if patient is receiving anticoagulant therapy and INR or aPTT is within target range of therapy (obtained =< 8 days prior to registration) Creatinine =< 1.5 x ULN OR calculated creatinine clearance >= 45 ml/min using the Cockcroft-Gault formula (obtained =< 8 days prior to registration)  Negative pregnancy test done =< 7 days prior to registration, for persons of childbearing potential only  Note: If testing done for eligibility is > 72 hours prior to first dose, then pregnancy testing must be repeated and result must be negative for patient to receive treatment Persons able to become pregnant OR able to father a child must be willing to use an adequate method of contraception while on treatment and for 180 days after last treatment Life expectancy >= 12 weeks Provide written informed consent Willing to return to enrolling institution for follow-up (during the active monitoring phase of the study) Willingness to provide mandatory blood specimens for correlative research Willingness to provide mandatory tissue specimens for correlative research RE-REGISTRATION (SECOND COURSE) ELIGIBILITY  Either  Stopped initial treatment with study treatment after attaining an investigator-determined confirmed complete response (CR) based on RECIST 1.1, and  Was treated with at least 8 cycles of study treatment before discontinuing treatment, and Received at least 2 treatments with pembrolizumab beyond the date when the initial CR was declared OR Had stable disease (SD), partial response (PR), or CR and stopped study treatment after completion of 35 administrations (approximately 2 years) of study treatment for reasons other than disease progression or intolerability  Experienced an investigator-determined radiographic disease progression by RECIST 1.1 after stopping initial treatment, and  No new anticancer treatment was administered after the last dose of study treatment, and The participant meets all of the safety parameters listed in the inclusion criteria and none of the safety parameters listed in the exclusion criteria, and The study is ongoing  Exclusion Criteria:  REGISTRATION - EXCLUSION CRITERIA  Any of the following because this study involves an agent that has known genotoxic, mutagenic and teratogenic effects:  Pregnant persons Nursing persons Persons of childbearing potential and persons able to father a child who are unwilling to employ adequate contraception Persons expecting to conceive or father children during study treatment or within 180 days (6 months) after the last treatment  Any of the following prior therapies:  Surgery < 3 weeks prior to registration Systemic anti-cancer therapy < 3weeks prior to registration  Radiotherapy < 2 weeks prior to registration OR Palliative radiation < 1 week prior to registration  NOTES: Must have recovered from all radiation related adverse effects (=< grade 1) Must not currently require corticosteroids Must not have had radiation pneumonitis  Live vaccine < 4 weeks prior to registration  NOTES: Examples of live vaccines include, but are not limited to, the following: measles, mumps, rubella, varicella/zoster (chicken pox), yellow fever, rabies, Bacillus Calmette-Guerin (BCG), and typhoid vaccine. Seasonal influenza vaccines for injection are generally killed virus vaccines and are allowed; however, intranasal influenza vaccines (e.g., FluMist) are live attenuated vaccines and are not allowed Received an investigational agent or used an investigational device or participated in a study of an investigational agent < 4 weeks prior to registration  Known active human immunodeficiency virus (HIV) infection (defined as patients who are not on anti-retroviral treatment and have detectable viral load and CD4+ < 500/ml)  NOTE: HIV-positive patients who are well controlled on anti-retroviral therapy are allowed to enroll  Active autoimmune disease requiring systemic treatment < 2 years prior to registration, documented history of severe autoimmune disease, or a syndrome that requires systemic steroids or immunosuppressive agents with use of disease modifying agents, corticosteroids or immunosuppressive drugs  NOTE: Exceptions are allowed for:  Vitiligo Resolved childhood asthma/atopy Intermittent use of bronchodilators or inhaled steroids Daily steroids at dose of =< 10 mg of prednisone (or equivalent) Local steroid injections Stable hypothyroidism on replacement therapy Stable diabetes mellitus on therapy (with or without insulin) Sjogren's syndrome Replacement therapy (e.g., thyroxine, insulin, or physiologic corticosteroid replacement therapy for adrenal or pituitary insufficiency, etc.) is not considered a form of systemic treatment and is allowed  Current or prior use of immunosuppressive medication < 14 days prior to registration  NOTE: The following are exceptions to this criterion:  Intranasal, inhaled, topical steroids, or local steroid injections (e.g., intraarticular injection)  Systemic corticosteroids at physiologic doses not to exceed 10 mg/day of prednisone or its equivalent  Steroids as premedication for hypersensitivity reactions (e.g., premedication for computed tomography [CT] scans)  Uncontrolled intercurrent illness including, but not limited to:  Ongoing or active infection requiring systemic therapy Interstitial lung disease or clinically significant pleural effusion Clinically significant ascites Serious, chronic gastrointestinal conditions associated with diarrhea (e.g., Crohn's disease or others) Known history of hepatitis B (i.e., known positive hepatitis B virus [HBV] surface antigen [HBsAg] reactive) Known active hepatitis C (i.e., positive for hepatitis C virus [HCV] ribonucleic acid [RNA] detected by polymerase chain reaction [PCR]) Known active tuberculosis (TB) Symptomatic congestive heart failure Unstable angina pectoris Unstable cardiac arrhythmia or Psychiatric illness/social situations that would limit compliance with study requirements (e.g., substance abuse) Co-morbid systemic illnesses or other severe concurrent disease or current evidence of any condition, therapy, or laboratory abnormality which, in the judgment of the investigator, would make the patient inappropriate for entry into this study or interfere significantly with the proper assessment of safety and toxicity of the prescribed regimens  Failure to recover to =< grade 1 (or baseline) from adverse events due to previously administered therapies or prior surgery  Exceptions: Neuropathy, fatigue, and/or alopecia may be grade 1  Known active central nervous system (CNS) metastases  NOTE: Patients with previously treated brain metastases may participate provided all of the following are true:  They are stable (without evidence of progression by imaging =< 4 weeks prior to registration and any neurologic symptoms have returned to baseline) Have no evidence of new or enlarging brain metastases, and Are not using steroids =< 14 days prior to registration Known leptomeningeal disease Hypersensitivity (>= grade 3) to pembrolizumab or any of its excipients Previous serious adverse event (>= grade 3) attributed to prior checkpoint inhibitor therapy History of (non-infectious) pneumonitis that required steroids or has current pneumonitis  History of grade >= 3 immune-related adverse event or any grade of immune-related neurologic or ocular adverse event while receiving immunotherapy  Note: Patients who had endocrine adverse events =< grade 2 are allowed to enroll if they are stable on appropriate replacement therapy and asymptomatic  Other active malignancy < 2 years prior to registration  EXCEPTIONS: Non-melanotic skin cancer, superficial bladder cancer, papillary thyroid cancer, or carcinoma-in-situ of the cervix or others curatively treated and now considered to be at less than 30% risk of relapse History of allogenic tissue/solid organ transplant