Following this instruction :
1. Strictly adhere to PEP-8 standards, ensuring code is clean, well-structured, and follows Python's style guide for readability.
2. Use appropriate modules like `typing` for type hints, ensuring code clarity, maintainability, and leveraging Python's standard libraries effectively.
3. Focus on time, space, robustness, scalability; write optimized, efficient code that can handle large datasets and scale seamlessly.
4. Implement comprehensive error handling, catching exceptions gracefully, providing meaningful error messages, and ensuring code robustness under unexpected conditions.
5. Ensure the developer agent writes advanced, maintainable Python code, balancing performance and readability while adhering to best practices.

TASK : Generalised robustness metrics Pipeline (logits or predictions or loss values)

NOTE : using the only torch module we have implement below scratch 

Problem Statement:
Develop a metrics pipeline using PyTorch for evaluating text generation tasks. The pipeline should calculate:

Perplexity: Measures the inverse probability of generated words in context.
BLEU (Text Generation): Precision for n-gram overlap between generated and reference texts.
ROUGE-L: Longest common subsequence (LCS) overlap with references.
Content Preservation Score: Measures content preservation between generated and reference text.
Distinct-1 and Distinct-2: Diversity metrics for unigram and bigram diversity in generated text.
The metrics should be adaptable to different batch sizes, n-gram counts, and specific requirements in text generation evaluation.





### 1. **Perplexity**  
Perplexity (\(P\)) is a measure of the inverse probability of predicted words in context, with lower values indicating better performance (higher probability).

For a batch of text sequences, perplexity is calculated as:
\[
P = \exp \left( -\frac{1}{N} \sum_{i=1}^{N} \log p(w_i) \right)
\]
where:
- \(N\) is the total number of words in the generated text.
- \(p(w_i)\) is the predicted probability of the \(i\)-th word in the sequence, given the preceding words.

In PyTorch, this can be implemented by calculating the negative log probability for each word and averaging it across the batch, then taking the exponential.

### 2. **BLEU (Text Generation)**  
BLEU (Bilingual Evaluation Understudy) score measures n-gram overlap between the generated text and reference text, emphasizing precision. BLEU with n-gram up to \(N\) can be calculated as:
\[
\text{BLEU} = \exp \left( \sum_{n=1}^N w_n \log \text{Precision}_n \right) \times \text{BP}
\]
where:
- \(w_n\) are the weights for each n-gram precision (commonly \(w_n = \frac{1}{N}\) for uniform weighting).
- \(\text{Precision}_n\) is the modified precision score for each n-gram.
- \(\text{BP}\) is the brevity penalty, which penalizes short generated sequences compared to the reference, calculated as:
  \[
  \text{BP} = 
  \begin{cases} 
      1 & \text{if } c > r \\ 
      \exp\left(1 - \frac{r}{c}\right) & \text{if } c \leq r 
  \end{cases}
  \]
  where \(c\) is the length of the generated sequence, and \(r\) is the length of the reference.

### 3. **ROUGE-L**  
ROUGE-L measures the longest common subsequence (LCS) overlap between generated and reference texts, focusing on recall. It’s defined as:
\[
\text{ROUGE-L} = \frac{(1 + \beta^2) \times \text{LCS-precision} \times \text{LCS-recall}}{\text{LCS-precision} + \beta^2 \times \text{LCS-recall}}
\]
where:
- \(\text{LCS-precision} = \frac{\text{LCS length}}{\text{length of generated text}}\)
- \(\text{LCS-recall} = \frac{\text{LCS length}}{\text{length of reference text}}\)
- \(\beta\) is typically set to favor recall over precision, often \(\beta = 1\) for a balanced F-score.

### 4. **Content Preservation Score**  
Content preservation is usually computed as a similarity score between generated text and reference text, often based on a semantic similarity measure (e.g., cosine similarity between sentence embeddings) or a term-based metric like Jaccard similarity:
\[
\text{Content Preservation Score} = \frac{|G \cap R|}{|G \cup R|}
\]
where:
- \(G\) is the set of unique content terms in the generated text.
- \(R\) is the set of unique content terms in the reference text.

Alternatively, if using embeddings, compute cosine similarity as:
\[
\text{Cosine Similarity} = \frac{\text{Emb}(G) \cdot \text{Emb}(R)}{\|\text{Emb}(G)\| \times \|\text{Emb}(R)\|}
\]
where \(\text{Emb}(G)\) and \(\text{Emb}(R)\) are the embeddings for generated and reference texts.

### 5. **Distinct-1 and Distinct-2**  
Distinct-n scores measure diversity by calculating the ratio of unique n-grams to total n-grams in the generated text. This metric is helpful to evaluate the lexical variety in text generation.

For Distinct-1 (unigram diversity) and Distinct-2 (bigram diversity), these are defined as:
\[
\text{Distinct-1} = \frac{\text{Unique unigrams in generated text}}{\text{Total unigrams in generated text}}
\]
\[
\text{Distinct-2} = \frac{\text{Unique bigrams in generated text}}{\text{Total bigrams in generated text}}
\]
Higher Distinct scores indicate more diverse and less repetitive text generation.

---