## 5 Experiments and Results
### Evaluation Metrics  
To quantitatively evaluate the performance of our Multi-Element Contextual Hypergraph Extractive Summarizer (MCHES), we employ a set of widely used metrics that capture different aspects of summarization quality. These metrics include ROUGE scores, which assess the overlap between the generated summaries and the reference summaries, and advanced embedding-based metrics that evaluate semantic similarity.  
ROUGE (Recall-Oriented Understudy for Gisting Evaluation) metrics are standard for evaluating automatic summarization and machine translation systems. They measure the overlap between the system-generated summary and a set of reference summaries. ROUGE-N measures the overlap of N-grams between the generated summary and the reference summaries. For unigrams and bigrams, we use ROUGE-1 and ROUGE-2.  
The effectiveness of the trained model is assessed by using standard summarization metrics such as ROUGE-N (for assessing the overlap of N-grams between the generated summary and the reference), ROUGE-L (which considers the longest common subsequence), and ROUGE-S (which measures skip-bigram overlap):  
\[\text{ROUGE-N}=\frac{\sum_{s\in\text{Reference Summaries}}\sum_{gram_{n}\in s }\text{Count}_{\text{match}}(gram_{n})}{\sum_{s\in\text{Reference Summaries}}\sum_{gram_{n}\in s }\text{Count}(gram_{n})}, \tag{18}\]  
This comprehensive optimization and evaluation framework ensures that MCHES not only accurately captures the key elements of the original texts but also generates summaries that are coherent and closely aligned with human judgment.  
BERTScore leverages [39] the pre-trained contextual embeddings from BERT to measure the semantic quality of the generated summaries. It computes the cosine similarity between the tokens of the system-generated summary and the tokens of the reference summary:  
\[\text{BERTScore}=\frac{1}{N}\sum_{i=1}^{N}\max_{j}\cos(\text{BERT}_{\text{ sys}}(i),\text{BERT}_{\text{ref}}(j)) \tag{19}\]  
where \(\text{BERT}_{\text{sys}}(i)\) and \(\text{BERT}_{\text{ref}}(j)\) are the BERT embeddings of the \(i\)-th token in the system summary and the \(j\)-th token in the reference summary, respectively, and \(N\) is the number of tokens in the system summary.  
MoverScore extends [40] the idea of BERTScore by using Earth Mover's Distance to calculate a distance-based score between the embeddings of the system and the reference summaries. It captures both semantic meaning and word order:  
\[\text{MoverScore}=1-\text{EMD}(\text{BERT}_{\text{sys}},\text{BERT}_{\text{ ref}}) \tag{20}\]  
\begin{table}
\begin{tabular}{l l l l l} \hline \hline
**Dataset** & **Domain** & **\# Pairs** & **Avg. Doc** & **Avg. Sum** \\  & & & **Tokens** & **Tokens** \\ \hline XSum & News & 226,711 & 431 & 23 \\ \hline CNN/DailyMail & News & 311,971 & 781 & 56 \\ \hline PubMed & Scientific & 93,207 & 3011 & 203 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Summary of datasets used in the experimental evaluation.  
where EMD(BERTsys,BERTref) is the Earth Mover's Distance between the BERT embeddings of the system and the reference summaries.  
These metrics collectively provide a comprehensive view of both the fidelity and fluency of the generated summaries, allowing us to rigorously assess the effectiveness of the proposed summarization framework.