Back to all models
translation mask_token:
Query this model
πŸ”₯ This model is currently loaded and running on the Inference API. ⚠️ This model could not be loaded by the inference API. ⚠️ This model can be loaded on the Inference API on-demand.
JSON Output
API endpoint
								$
								curl -X POST \
-H "Authorization: Bearer YOUR_ORG_OR_USER_API_TOKEN" \
-H "Content-Type: application/json" \
-d '"json encoded string"' \
https://api-inference.huggingface.co/models/mrm8488/t5-small-finetuned-squadv1
Share Copied link to clipboard

Monthly model downloads

mrm8488/t5-small-finetuned-squadv1 mrm8488/t5-small-finetuned-squadv1
63 downloads
last 30 days

pytorch

tf

Contributed by

mrm8488 Manuel Romero
118 models

How to use this model directly from the πŸ€—/transformers library:

			
Copy to clipboard
from transformers import AutoTokenizer, AutoModelWithLMHead tokenizer = AutoTokenizer.from_pretrained("mrm8488/t5-small-finetuned-squadv1") model = AutoModelWithLMHead.from_pretrained("mrm8488/t5-small-finetuned-squadv1")

T5-small fine-tuned on SQuAD

Google's T5 (small) fine-tuned on SQuAD v1.1 for Q&A downstream task.

Details of T5

The T5 model was presented in Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer by Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu in Here the abstract:

Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new β€œColossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code.

model image

Details of the downstream task (Q&A) - Dataset πŸ“š 🧐 ❓

Dataset ID: squad from HugginFace/NLP

Dataset Split # samples
squad train 87599
squad valid 10570

How to load it from nlp

train_dataset  = nlp.load_dataset('squad, split=nlp.Split.TRAIN)
valid_dataset = nlp.load_dataset('squad', split=nlp.Split.VALIDATION)

Check out more about this dataset and others in NLP Viewer

Model fine-tuning πŸ‹οΈβ€

The training script is a slightly modified version of this awesome one by Suraj Patil

Results πŸ“

Metric # Value
EM 76.95
F1 85.71

Model in Action πŸš€

from transformers import AutoModelWithLMHead, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("mrm8488/t5-small-finetuned-squadv1")
model = AutoModelWithLMHead.from_pretrained("mrm8488/t5-small-finetuned-squadv1")

def get_answer(question, context):
  input_text = "question: %s  context: %s </s>" % (question, context)
  features = tokenizer([input_text], return_tensors='pt')

  output = model.generate(input_ids=features['input_ids'], 
               attention_mask=features['attention_mask'])

  return tokenizer.decode(output[0])

context = "Manuel have created RuPERTa-base (a Spanish RoBERTa) with the support of HF-Transformers and Google"
question = "Who has supported Manuel?"

get_answer(question, context)

# output: 'HF-Transformers and Google'

Created by Manuel Romero/@mrm8488 | LinkedIn

Made with in Spain