# Fine-Tuning a Personalized OpenBioLLM Using Offline Reinforcement Learning
## 3 Methods
### Template Based on Chain of Thought  
Chain of Thought is a cognitive process that involves connecting a series of logical steps or reasoning processes in series to solve complex problems or draw conclusions. By using a chain-of-thought-based template, the large language model can analyze and solve problems step by step in a structured manner, thereby improving the transparency and explainability of the solution. In addition, this template can guide the large model to generate personalized treatment plans and provide personalized treatment recommendations for different patients based on their specific conditions. In the process of data enhancement, cot_1 is used as a chain of thought prompt template, which receives inputs such as the patient's demographic information, medical history, physiological indicators, and **current treatment decisions**. The purpose of designing this template is to allow OpenBioLLM to analyze the patient's condition and analyze the reasons for a given treatment decision based on the patient's condition. The content of the cot_1 template is as follows. ```
There is a patient in the ICU, and their physiological indicators are as follows:

Demographics:

Age: 65 years old

Sex: male

Weight: 75 kg

Readmission to ICU: yes

Elixhauser score: 12 points

Vital signs:

SOFA (Sequential Organ Failure Assessment): 6 points

SIRS (Systemic Inflammatory Response Syndrome): yes

GCS (Glasgow Coma Scale): 14

Heart rate: 110 beats/min

SysBP (Systolic Blood Pressure): 95 mmHg

DiaBP (Diastolic Blood Pressure): 60 mmHg

MeanBP (Mean Blood pressure): 75 mmHg

Shock index: 1.5

Respiratory rate: 28 times/min

Temperature: 37.8 \({}^{\circ}\)C
SpO2 (peripheral capillary oxygen saturation): 88%

Laboratory values:

Potassiumium: 4.0 mmol/L

Sodium: 135 mmol/L

Chloride: 98 mmol/L

Glucose : 120 mg/dL

BUN (blood urea nitrogen): 45 mg/dL

Creatinine: 2.0 mg/dL

Magnesium: 1.8 mg/dL

CO\({}_{2}\) (carbon dioxide): 32 mmol/L

Hemoglobin: 10 g/dL

WBC (white blood cell count): 15,000/mm

Platelets: 150,000/mm\({}^{3}\)

PTT (Partial Thromoplastin Time): 40 s

PT (Prothrombin Time): 15 s

INR (International Normalized Ratio): 1.5

pH : 7.45

PaCO2 (Partial Pressure of Carbon Dioxide in Arterial Blood): 50 mmHg

Base Excess: 4 mEq/L

Bicarbonate: 28 mmol/L

The ventilator treatment plan given by the doctor is as follows:

PEEP: 10 cmH2O

Tidal Volume: 500 mL

FiO2: 0.6

Please analyze the patient's situation and the doctor's thinking process in making this treatment decision, and output the text. The doctor's third-person perspective in this book is replaced with the first-person perspective I think. Furthermore, the same treatment plan is given below. The output content obtained by the OpenBioLLM through cot_1 is as follows. It makes a detailed analysis of the patient's condition and treatment decision, explains the possible problems and areas that need attention for each indicator, and gives treatment suggestions. This content will be used as t_treatment to fine-tune the OpenBioLLM to align the OpenBioLLM with the treatment decision of SAC-CQL. The answer of edit. I as t_treatment

Based on the patient's physiological indicators, the following is my analysis of the patient's condition and my thought process in formulating a treatment plan:

1. Demographic characteristics:

A 65-year-old male weighing 75 kg is an elderly patient and may be at risk for multiple chronic diseases. ICU readmission indicates that the patient has had serious health problems before and may have higher requirements for treatment. Elixhauser score of 12 points indicates that the patient has multiple complications, which may affect the treatment plan and prognosis. 2. Vital signs:

SOFA score of 6 points indicates that the patient is at risk for multiple organ failure. The presence of SIRS indicates that the patient may be experiencing a systemic inflammatory response. GCS score of 14 points indicates that the patient's level of consciousness is acceptable, but lower than normal, and neurological function may need to be closely monitored.