# Hybrid Summarization of Medical Records for Predicting Length of Stay in the Intensive Care Unit
## 4 Experiments and Results
### Extractive Summarization  
In the extractive summarization phase, six methods were employed to summarize the daily progress notes, including two methods that we proposed based on the combination of existing approaches. The concept-based approach involved identifying relevant medical concepts using the QuickUMLS library and constructing a similarity matrix on concept overlap between sentences to create a graph representation with sentences as nodes and edges reflecting concept similarity and selecting the seven top-ranked sentences to form the summary using PageRank. On the other hand, the graph-based methods utilized TF-IDF vectorization and cosine similarity to construct a graph representation of the sentences, and PageRank was used to select the top seven sentences. Based on these two methods, we developed a third one whereby the graph is formed from the cosine similarity matrix of the sentences, the nodes are sentences, and the edges are the cosine similarity of the sentences based on the concepts' similarity scores; these concepts were extracted using QuickUMLS. In the topic-based technique, sentences were tokenized and transformed into numerical vectors using TF-IDF values, and cosine similarity was employed to select the seven top-ranked sentences. we proposed another technique that combined concept extraction with TF-IDF, whereby concepts were extracted using QuickUMLS, and TF-IDF scores were calculated alongside concept frequencies to determine sentence importance; the model ultimately selected the seven top-ranked sentences. Lastly, the clustering-based method employed BERT, which is a transformer-based model: specifically, the \"distilbert-base-uncased\" variant, with the \"hidden\" parameter set to \"\([-1,-2]\)\". The number of sentences to be present in the summary was set to seven. These models were trained on a \"Google Colab\" notebook for approximately 3 h. We then submitted them for testing to a CodaLab competition (CodaLab Competition for the BioNLP Workshop 2023 Shared Task 1A: Problem List Summarization, 2023, [[https://codalab.lisp.upsaclay.fr/competitions/12388](https://codalab.lisp.upsaclay.fr/competitions/12388)]([https://codalab.lisp.upsaclay.fr/competitions/12388](https://codalab.lisp.upsaclay.fr/competitions/12388)) (accessed 27 June 2024)) that was created for the \"BioNLP Workshop 2023 Shared Task 1A: Problem List Summarization\" challenge in order to manage system submissions and show the leaderboard. The CodaLab was configured to receive system prediction outputs in a text file format, and it ran a ROUGE-L evaluation script to produce scores. Our results are summarized in Table 1. The concept-based approach achieved a relatively high precision of 1.35%, indicating that the selected summary sentences closely matched the reference summaries. However, its recall of 41.87% suggests that it may have missed some important information, resulting in a lower F-score of 2.56%. The graph-based method, on the other hand, performed poorly in terms of precision (0.0006%) and F-score (0.001%). This suggests that the selected summary sentences may not be relevant to the reference summaries. Although it exhibited a higher recall of 0.08%, it failed to produce meaningful summaries overall. Combining the concept and graph approaches in the concept + graph-based method showed a slight improvement in precision (1.36%) and F-score (2.59%) compared to concept-based alone. However, its recall remained similar to concept-based, indicating that the addition of graph-based did not significantly enhance the quality of the summaries. \begin{table}
\begin{tabular}{l c c c} \hline \hline
**Method** & \(\mathbf{P_{LCS}}\) & \(\mathbf{R_{LCS}}\) & \(\mathbf{F_{LCS}}\) \\ \hline Concept-based & 1.35 & 41.87 & 2.56 \\ Graph-based & 0.0006 & 0.08 & 0.001 \\ Concept + Graph-based & 1.36 & 41.74 & 2.59 \\ Topic-based & 1.29 & 42.89 & 2.47 \\ Topic + Concept-based & 0.64 & 10.73 & 1.19 \\ Clustering-based & 1.69 & 38.91 & 3.17 \\ \hline \hline \end{tabular}
\end{table}