Kaylav11/t5-base

Model Card for Model ID

Final Project

Model Details

The model is designed to perform extractive and abstractive text summarization. Given a longer piece of text (e.g., an article, a research paper, a news report), it will generate a shorter version that retains the most critical information and main ideas. The primary output will be a coherent and grammatically correct summary, significantly reducing reading time while maintaining essential context.

Model Description

This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.

• Developed by: Kayla
• Model type: t5-base

Dataset and Training Details

We will use the CNN/DailyMail dataset for fine-tuning. This dataset is widely used for abstractive text summarization.

Evaluation Metrics

• ROUGE (Recall-Oriented Understudy for Gisting Evaluation): ROUGE metrics compare an automatically produced summary against a set of human-produced reference summaries.
• BLEU (Bilingual Evaluation Understudy): While more commonly used for machine translation, BLEU can also be applied to summarization to assess the precision of n-gram overlap. It measures how many n-grams in the generated text appear in the reference text.
• Human Evaluation (Qualitative): Although not directly computable within the notebook, we will qualitatively assess a small sample of generated summaries

Intended Uses and Limitations

For this project, we will utilize the t5-base model from the Hugging Face Transformers library as our base Large Language Model. T5 (Text-to-Text Transfer Transformer) is a powerful encoder-decoder model that frames all NLP tasks as a text-to-text problem, making it highly versatile for tasks like summarization. The chosen model will be fine-tuned on a specific summarization dataset. This approach leverages the extensive knowledge pre-trained into t5-base on a massive text corpus and then adapts it to the nuances and specifics of the summarization task, leading to superior performance compared to training from scratch.

How to load and use the model (code snippet).

''' { from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

#Define the model ID on Hugging Face Hub hub_model_id = "model id"

#Load the tokenizer loaded_tokenizer = AutoTokenizer.from_pretrained(hub_model_id)

#Load the model loaded_model = AutoModelForSeq2SeqLM.from_pretrained(hub_model_id)

#Move model to appropriate device (CPU or GPU if available) device = "cuda" if torch.cuda.is_available() else "cpu" loaded_model.to(device)

#Example usage: Summarize a piece of text def summarize_text_from_hub(text, model, tokenizer, max_length=150): input_text = "summarize: " + text inputs = tokenizer(input_text, return_tensors="pt", max_length=512, truncation=True).to(device) output_ids = model.generate(inputs["input_ids"], max_new_tokens=max_length, num_beams=4, early_stopping=True) return tokenizer.decode(output_ids[0], skip_special_tokens=True)

#Test with an example article example_article = """The Amazon rainforest is the largest tropical rainforest in the world, covering an area of about 5.5 million square kilometers (2.1 million square miles). It spans across nine countries, with the majority of it located in Brazil. The Amazon is incredibly biodiverse, home to millions of species of plants, animals, and insects. It plays a crucial role in regulating the Earth's climate by absorbing vast amounts of carbon dioxide and producing oxygen. Deforestation, primarily for cattle ranching and agriculture, poses a significant threat to the rainforest, leading to habitat loss and contributing to climate change. Efforts are underway to conserve this vital ecosystem."""

print("\n--- Original Article ---") print(example_article)

generated_summary = summarize_text_from_hub(example_article, loaded_model, loaded_tokenizer)

print("\n--- Generated Summary ---") print(generated_summary)

} '''