Inclusion Criteria:  The participant must provide written informed consent. Male/Female participants must be ≥18 years of age at the day of signing informed consent and must be willing to comply with the study specific procedures. Histologically confirmed metastatic or advanced, unresectable HER2 negative (0, 1+ by IHC or ISH amplified < 2.0) breast cancer which is not eligible for curative treatment. Cohort 1: Germline mutation in BRCA1 or BRCA2 that is predicted to be deleterious (known or predicted to be detrimental/lead to loss of function) irrespective of HRD status. Cohort 2: Germline mutation in ATM, BARD1, CHEK2, FANCC, PALB2, RAD51C, RAD51D, SLX4, XRCC2 that is predicted to be deleterious (known or predicted to be detrimental/lead to loss of function) irrespective of HRD status. Cohort 3: High HRD status and no germline mutation in one of the above mentioned genes of cohort 1 or cohort 2. Cohort 3: Availability of FFPE tumor material for further validation of HRD status (bridging tests). Cohorts 2 and 3: Patients must have been treated with first line chemotherapy, if this chemotherapy is standard of care in this therapy situation. Prior platinum in the (neo)adjuvant setting is allowed as long as 12 months from last dose to study entry have elapsed. Participants with ER/PR positive breast cancer must have exhausted previous combination therapy of CDK4/6 inhibitors with endocrine treatment. Measurable disease based on RECIST v1.1. Provision of a recently obtained (within 12 months before study inclusion) core or excisional biopsy of a tumor lesion. Note: If submitting unstained cut slides, newly cut slides should be submitted to the testing laboratory within 14 days from the date slides are cut. ECOG performance status 0-1. Female participants must have a negative urine or serum pregnancy test within 72 h prior to first dose of trial treatment, no breastfeeding. Female participants of childbearing potential must agree to use sufficient methods of contraception as outlined in section 12.3.2 Contraception Requirements during treatment plus an additional 120 days after the last dose of study medication. Male participants must agree to use sufficient methods of contraception as outlined in section 12.3.2 Contraception Requirements during treatment plus an additional 120 days after the last dose of study medication.  Adequate organ function defined as:  Absolute neutrophile count ≥1500/µL Platelets ≥100 000/µL Hemoglobin ≥10.0 g/dL or ≥6.2 mmol/L Geschätzte Kreatinin-Clearance ≥51 mL/min berechnet mit der Cockcroft-Gault-Gleichung oder basierend auf dem 24-Stunden-Sammelurin Total bilirubin ≤1.5 ×ULN OR direct bilirubin ≤ULN for participants with total bilirubin levels >1.5 × ULN AST (SGOT) and ALT (SGPT) ≤2.5 × ULN (≤5 × ULN for participants with liver metastases) International normalized ratio (INR) OR prothrombin time (PT) ≤1.5 × ULN unless participant is receiving anticoagulant therapy as long as PT or aPTT is within therapeutic range of intended use of anticoagulants  Exclusion Criteria:  Has histologically confirmed HER2 positive (3+ by IHC or ISH amplified ≥ 2.0) breast cancer. Cohorts 1 and 2: germline mutations in BRCA1, BRCA2, ATM, BARD1, CHEK2, FANCC, PALB2, RAD51C, RAD51D, SLX4, XRCC2 that are considered to be non-detrimental (e.g., "variants of uncertain/unknown clinical significance" or "benign polymorphism" etc.). Cohort 3: no high tumor HRD. Rapidly progressive disease which requires combination chemotherapy. Current participation in another investigational trial within 4 weeks prior to the first dose of trial treatment Known hypersensitivity to pembrolizumab or olaparib or any of its excipients. Prior systemic anti-cancer therapy within 4 weeks prior to allocation or no recovery from all AEs due to previous therapies to ≤ grade 1, excluding alopecia and ≤ grade 2 peripheral neuropathy. Prior treatment with a checkpoint inhibitor or a PARP inhibitor. No complete recovery from prior surgery or radiotherapy. Starting study treatment is allowed not before 2 weeks after major surgery. Prior malignancy unless curatively treated and disease-free for less than 3 years prior to study entry. Within this timeframe, prior adequately treated non-melanoma skin cancer, transitional cell carcinoma, carcinoma in situ of the prostate, of the cervix, of the breast or in situ or stage I grade 1 endometrial cancer is eligible. Uncontrolled brain metastases (Participants with previously treated brain metastases may participate provided they are radiologically stable, i.e. without evidence of progression for at least 4 weeks (note that the assessment of the brain metastases should be performed during study screening for this purpose), clinically stable and without requirement of steroid treatment for at least 14 days prior to first dose of study treatment). Live vaccination within 30 days prior to study entry. Has active autoimmune disease that has required systemic treatment in the past 2 years (i.e. with use of disease modifying agents, corticosteroids or immunosuppressive drugs). Replacement therapy (e.g., thyroxine, insulin, or physiologic corticosteroid replacement therapy for adrenal or pituitary insufficiency, etc.) is not considered a form of systemic treatment. Has an active infection requiring systemic therapy. Has a history of (non-infectious) pneumonitis that required steroids or has current pneumonitis.  Known history of the following infections:  Human Immunodeficiency Virus (HIV). Acute or chronic Hepatitis B or Hepatitis C Active Tuberculosis Resting ECG indicating uncontrolled, potentially reversible cardiac conditions, as judged by the investigator (e.g., unstable ischemia, uncontrolled symptomatic arrhythmia, congestive heart failure, QTcF prolongation >500 ms, electrolyte disturbances, etc.), or patients with congenital long QT syndrome. Patients with myelodysplastic syndrome (MDS) or acute myeloid leukaemia (AML) or with features suggestive of MDS/AML. Preexisting use of known strong CYP3A inhibitors (e.g. itraconazole, telithromycin, clarithromycin, protease inhibitors boosted with ritonavir or cobicistat, indinavir, saquinavir, nelfinavir, boceprevir, telaprevir) or moderate CYP3A inhibitors (e.g. ciprofloxacin, erythromycin, diltiazem, fluconazole, verapamil). The required washout period prior to starting trial treatment is 2 weeks. Preexisting use of known strong (e.g. phenobarbital, enzalutamide, phenytoin, rifampicin, rifabutin, rifapentine, carbamazepine, nevirapine and St John's Wort ) or moderate CYP3A inducers (e.g. bosentan, efavirenz, modafinil). The required washout period prior to starting trial treatment is 5 weeks for enzalutamide or phenobarbital and 3 weeks for other agents. Previous allogenic bone marrow transplant or double umbilical cord blood transplantation (dUCBT). Poor medical risk due to a serious, uncontrolled medical disorder, non-malignant systemic disease or active, uncontrolled infection; any condition that interferes with pembrolizumab or olaparib treatment. Unability to swallow or gastrointestinal disorders with reduced absorption of olaparib. Psychiatric or substance abuse disorders. A woman of childbearing potential who has a positive urine pregnancy test within 72 hours prior to inclusion. If the urine test is positive or cannot be confirmed as negative, a serum pregnancy test will be required. Participants being pregnant or breastfeeding, or expecting to conceive or father children within the projected duration of the study, starting with the screening visit through 120 days after the last dose of trial treatment. Any other condition in opinion of the investigator that would interfere with applied systemic treatment or other trial procedures