Key Inclusion Criteria  Part 1- Evaluable disease per RECIST Version 1.1 Part 2 - Measurable disease per RECIST Version 1.1  Patient has ECOG performance status of 0-1  One or more documented primary oncogenic PIK3CA mutation(s) in blood and/or tumor per local assessment  - Other potentially oncogenic PIK3CA mutations may be considered but must be approved by the Sponsor prior to enrollment.  Part 1 - Ability to provide archived tumor tissue or be willing to undergo pretreatment tumor biopsy to assess PIK3CA status retrospectively Part 2 - Submit tumor tissue prior to study drug initiation for determination of PIK3CA mutation retrospectively.  Key Inclusion for RLY-2608 Single Agent Arm  - Disease that is refractory to standard therapy, intolerant to standard therapy, or has declined standard therapy. Part 1- histologically or cytologically confirmed diagnosis of unresectable or metastatic solid tumor  Part 2 - Unresectable or metastatic solid tumor with PIK3CA mutation(s) and one of the following tumor types:  Group 1: clear cell ovarian cancer Group 2: head and neck squamous cell carcinoma Group 3: cervical cancer Group 4: other solid tumors, excluding colorectal, clear cell ovarian, head and neck squamous cell, and cervical cancers Group 5: unresectable or metastatic solid tumors with PIK3CA double mutations  Key Inclusion for RLY-2608 + Fulvestrant Arm  - Male or postmenopausal female with histologically or cytologically confirmed diagnosis of HR+, HER2- unresectable or metastatic breast cancer that is not amenable to curative therapy. Premenopausal or perimenopausal females must have a histologically or cytologically confirmed diagnosis of HR+ HER2- advanced or metastatic breast cancer that is not amenable to curative therapy and must have been previously treated with GnRH agonist at least 4 weeks prior to start of study drug  [ For Part 1 and Part 2]: Had previous treatment for advanced or metastatic breast cancer with:  ≤1 chemotherapy regimen, ≥1 cyclin-dependent kinases (CDK) 4/6 inhibitor, and ≥1 antiestrogen therapy including, but not limited to, selective estrogen-receptor degraders (eg, fulvestrant), selective estrogen receptor modulators (eg, tamoxifen), and aromatase inhibitors (AI) (letrozole, anastrozole, exemestane)  [For Part 2, Group 2]: Received prior treatment with a PI3Kα inhibitor and discontinued the inhibitor due to intolerance and not disease progression, where intolerance is defined as treatment discontinuation due to treatment related AE (eg. hyperglycemia, rash, diarrhea, stomatitis) other than severe hypersensitivity reaction and/or life-threatening reactions, such as anaphylaxis and Stevens-Johnson syndrome.  Key Exclusion Criteria  Prior treatment with PI3Kα inhibitors (except for RLY-2608 + fulvestrant arm, Part 2, Group 2).  Type 1 or Type 2 diabetes requiring antihyperglycemic medication, or fasting plasma glucose ≥140 mg/dL and glycosylated hemoglobin (HbA1c) ≥7.0%.  History of hypersensitivity to PI3K inhibitors  QT interval corrected using Fridericia's formula (QTcF) > 480 msec or history of prolonged QT syndrome, Torsades de pointes or familial history of prolonged QT syndrome Clinically significant, uncontrolled cardiovascular disease CNS metastases or primary CNS tumor that is associated with progressive neurologic symptoms