# Hybrid Summarization of Medical Records for Predicting Length of Stay in the Intensive Care Unit
## 4 Experiments and Results
### Abstractive Summarization  
For this step, we used the outputs of the three best extractive summarization methods, based on their ROUGE-L F-scores, as input to the T5-base model. We worked on fine-tuning the pre-trained model using the concept masking technique. The hyperparameters of the T5-base model included an \"AdamW\" optimizer and a learning rate of \(1\times 10^{-4}\). The number of training epochs was only two, and the \"max input size\" was set to \"512\". We ran the models locally for two epochs on a NVIDIA GeForce GTX Ti GPU with Max-Q Design for a total of about 8 h. Similarly to the extractive summarization step, we submitted the models for testing to the CodaLab competition, with our best-performing hybrid summarizer displayed on the leaderboard: (CodaLab leaderboard of the BioNLP Workshop 2023 Shared Task 1A: Problem List Summarization, 2023, [[https://codalab.lisen.upscalay.fr/competitions/12388#results](https://codalab.lisen.upscalay.fr/competitions/12388#results)]([https://codalab.lisen.upscalay.fr/competitions/12388#results](https://codalab.lisen.upscalay.fr/competitions/12388#results)) (accessed 27 June 2024)). The results are shown in Table 2.  
All of the methods performed better than the baseline [19], with the combination of concept-based summarization and T5 summarization being our best proposed method: achieving a precision of 43.83%, a recall of 14.99%, and an F-score of 20.32%. Its generated summaries are to be used as features for LOS prediction. While the benchmark method [27] is still at the top of existing methods with an F-score of 32.77%, our best method surpasses its precision of 41.69%.  
\begin{table}
\begin{tabular}{l c c c} \hline \hline
**Method** & \(\mathbf{P}_{LCS}\) & \(\mathbf{R}_{LCS}\) & \(\mathbf{F}_{LCS}\) \\ \hline Baseline [19] & - & - & 15 \\ \hline Graph + Concept + T5 & 39.44 & 15.18 & 19.66 \\ BERT + T5 & 40.19 & 15.41 & 20.30 \\ Concept + T5 & 43.83 & 14.99 & 20.32 \\ \hline CUED [27] & 41.69 & 30.51 & 32.77 \\ \hline \hline \end{tabular}
\end{table}
Table 2: ROUGE scores in percentage (%) for proposed hybrid summarization methods in comparison with baseline and benchmark methods.  
It was observed that the precision values were higher than the recall values. This suggests that the model was better at avoiding false positives, which means that it accurately identified instances that were not relevant. However, it may have missed a significant number of true positive instances, indicating that some relevant information was not captured.  
There are two possible reasons for the low recall. Firstly, the abstractive method might have failed to generate the necessary concepts. This is logical considering the small dataset and the absence of any data augmentation techniques, which could have limited the model's ability to generate comprehensive summaries. Secondly, the extractive method might have removed sentences containing important information required for the abstractive summarization step. This implies that during the extraction process, some relevant sentences were not retained, resulting in a loss of vital information in the generated summaries.  
The trade-off between precision and recall depends on the specific requirements of the summaries. In this case, as the summaries are used for ICU LOS prediction, having high precision is desirable. This means that the generated summaries are more likely to provide accurate and relevant information, enabling more precise predictions. Therefore we deem the output from the concept-based and T5 summarization combination method, which has higher precision then the benchmark method, to be suitable as input for the next step of the ICU LOS classification.