Update README.md

README.md

@@ -0,0 +1,49 @@
+---
+language: "[en]"
+datasets:
+- Spotify Podcasts Dataset
+metrics:
+- ROUGE
+---
+
+# T5 for Automatic Podcast Summarisation
+
+This model is the result of fine-tuning [t5-base](https://huggingface.co/t5-base) on the [Spotify Podcast Dataset](https://arxiv.org/abs/2004.04270).
+
+It is based on [Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) which was pretrained on the [C4 dataset](https://huggingface.co/datasets/c4).
+
+
+Paper: [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf)
+
+Authors: Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu
+
+## Intended uses & limitations
+This model is intended to be used for automatic podcast summarisation. As creator provided descriptions
+were used for training, the model also learned to generate promotional material in its summaries, as such
+some post processing may be required on the model's outputs.
+
+#### How to use
+A 'summarize:' tag must be appended to the source text before it is passed to the T5 model.
+
+```python
+from transformers import T5Tokenizer, T5ForConditionalGeneration
+
+tokenizer = T5Tokenizer.from_pretrained('paulowoicho/t5-podcast-summarisation')
+model = T5ForConditionalGeneration.from_pretrained('paulowoicho/t5-podcast-summarisation')
+
+podcast_transcript = 'summarize: ' + podcast_transcript
+tokens = tokenizer.encode(podcast_transcript, return_tensors="pt")
+summary_ids = model.generate(tokens, max_length=150, num_beams=2, repetition_penalty=2.5, length_penalty=1.0, early_stopping=True)
+output = tokenizer.decode(summary_ids[0], skip_special_tokens=True)
+
+print(output)
+```
+
+## Training data
+
+This model is the result of fine-tuning [t5-base](https://huggingface.co/t5-base) on the [Spotify Podcast Dataset](https://arxiv.org/abs/2004.04270).
+[Pre-processing](https://github.com/paulowoicho/msc_project/blob/master/reformat.py) was done on the original data before fine-tuning.
+
+## Training procedure
+
+Training was largely based on [Fine-tune T5 for Summarization](https://github.com/abhimishra91/transformers-tutorials/blob/master/transformers_summarization_wandb.ipynb) by [Abhishek Kumar Mishra](https://github.com/abhimishra91)