Inclusion Criteria:  Participants must have histologically or cytologically confirmed prostate cancer. Either fresh biopsy or archival tissue can be used for confirmation. Age >= 18 years. Patients must have metastatic castration resistant prostate cancer (mCRPC) with progression based on Prostate Cancer Working Group 3 (PCWG3) criteria. Patients must have adenocarcinoma histology. Prior treatment with at least one novel hormonal agents (NHA) such as abiraterone acetate, enzalutamide, apalutamide, darolutamide etc. Prior treatment with one line of taxane-based chemotherapy administered in either the hormone sensitive or castrate-resistant setting. Patients must have recovered (CTCAE grade =< 1) from the acute effects of chemotherapy except for residual alopecia or grade 2 peripheral neuropathy prior to study entry. A washout period of at least 21 days is required between last chemotherapy dose and treatment initiation Patients must have prior orchiectomy and/or ongoing androgen-deprivation therapy and a castrate level of serum testosterone (< 50 ng/dL or < 1.7 nmol/L) Patients must have a 68Ga-PSMA-11 PET scan with at least 3 PSMA-positive lesions (maximum standardized uptake value [SUVmax] greater than SUVmax of liver) as determined by nuclear medicine review prior to start of lead-in treatment with abemaciclib Patients must have Eastern Cooperative Oncology Group (ECOG) performance status (PS) of 0-2 Patients must have life expectancy of > 6 months  Patients must have adequate organ function as outlined below and bone marrow reserve  White blood cell (WBC) > 3.0 Absolute neutrophil count (ANC) > 1.5 Hemoglobin (Hgb) > 9.0 Platelets (Plt) > 100,000 Total bilirubin =< 1.5 x the institutional upper limit of normal (ULN). For patients with known Gilbert's Syndrome =< 3 ULN is permitted Aspartate aminotransferase (AST)(serum glutamic oxaloacetic transaminase (SGOT)) =< 3 X institutional upper limit of normal (=< 5.0 ULN for patients with liver metastases) Alanine aminotransferase (ALT)(serum glutamic pyruvic transaminase (SGPT)) =< 3 X institutional upper limit of normal (=< 5.0 ULN for patients with liver metastases) Creatinine =< 1.5 x within institutional upper limit of normal OR creatinine clearance glomerular filtration rate (GFR) >= 60 mL/min/1.73 m, calculated using the Cockcroft-Gault equation, unless data exists supporting safe use at lower kidney function values, no lower than 30 mL/min/1.73 m^2 Patient must be able to swallow oral medications Patients must have the ability to understand a written informed consent document, and the willingness to sign it Human immunodeficiency virus (HIV)-infected individuals on effective anti-retroviral therapy with undetectable viral load within 6 months are eligible for this trial For participants with evidence of chronic hepatitis B virus (HBV) infection, the HBV viral load must be undetectable on suppressive therapy, if indicated Individuals with a history of hepatitis C virus (HCV) infection must have been treated and cured. For individuals with HCV infection who are currently on treatment, they are eligible if they have an undetectable HCV viral load Individuals with a prior or concurrent malignancy whose natural history or treatment does not have the potential to interfere with the safety or efficacy assessment of the investigational regimen are eligible for this trial Patients with reproductive potential must agree to use effective contraception and to not donate sperm during the study and for at least 3 months following the last dose of study treatment. Effective method of contraception means male condom with spermicide, female condom with spermicide, diaphragm with spermicide, cervical sponge, or cervical cap with spermicide  Exclusion Criteria:  Patients with small cell or neuroendocrine carcinoma histology. Patients with a super scan seen in the baseline bone scan. Super scan refers to a bone scan with diffusely increased skeletal radioisotope uptake relative to soft tissue Patients with prior treatment with CDK4/6 inhibitors Patients with previous treatment with PSMA-targeted radioligand therapy Patients with previous treatment with Strontium-89, Samarium-153, Rhenium-186, Rhenium-188, Radium-223 or hemi-body irradiation within 6 months prior to study entry Any systemic anti-cancer therapy within 3 weeks of study entry Patients who have experienced significant radiation-related adverse events (AEs) from prior radiation treatment (>= grade 3) or have experienced persistent radiation-related AEs that have not resolved by the time of study randomization Patients with prior radiation treatment to lungs or liver Patients with a history of central nervous system (CNS) metastases are ineligible unless they have received prior therapy (surgery, radiation therapy (RT), gamma knife) and are, asymptomatic, and not receiving corticosteroids for this indication. Head imaging is not required Patients with symptoms of cord compression or impending cord compression Patients with concurrent serious medical conditions as determined by primary investigator Patients with other significant malignancies that are expected to alter life expectancy or interfere with disease assessment. Patients with adequately treated skin cancer, non-muscle-invasive bladder cancer and patients with prior history of malignancy who have been disease free for more than 3 years are eligible. Patients with history of in-situ/early stage melanoma will not be excluded Patients who have not recovered from adverse events due to prior anti-cancer therapy to =< grade 1 or baseline (other than alopecia or peripheral neuropathy) Patients with serious and/or uncontrolled preexisting medical condition(s) that, in the judgment of the investigator, would preclude participation in this study (for example, interstitial lung disease, severe dyspnea at rest or requiring oxygen therapy, severe renal impairment [e.g. estimated creatinine clearance < 30ml/min], history of major surgical resection involving the stomach or small bowel, or preexisting Crohn's disease or ulcerative colitis or a preexisting chronic condition resulting in baseline grade 2 or higher diarrhea) The patient has active systemic bacterial infection (requiring intravenous (IV) antibiotics at time of initiating study treatment), fungal infection, or detectable viral infection (such as known human immunodeficiency virus positivity or with known active hepatitis B or C (for example, hepatitis B surface antigen positive). Screening is not required for enrollment The patient has a personal history of any of the following conditions: syncope of cardiovascular etiology, ventricular arrhythmia of pathological origin (including, but not limited to, ventricular tachycardia and ventricular fibrillation), or sudden cardiac arrest. Patients currently receiving any other investigational therapeutic agents.