sjrhuschlee commited on
Commit
f451ebc
1 Parent(s): cb2b286

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +78 -0
README.md CHANGED
@@ -1,3 +1,81 @@
1
  ---
2
  license: mit
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: mit
3
+ datasets:
4
+ - squad_v2
5
+ - squad
6
+ language:
7
+ - en
8
+ library_name: transformers
9
+ tags:
10
+ - question-answering
11
+ - squad
12
+ - squad_v2
13
+ - t5
14
  ---
15
+
16
+ # flan-t5-large for Extractive QA
17
+
18
+ This is the [flan-t5-large](https://huggingface.co/google/flan-t5-large) model, fine-tuned using the [SQuAD2.0](https://huggingface.co/datasets/squad_v2) dataset. It's been trained on question-answer pairs, including unanswerable questions, for the task of Extractive Question Answering.
19
+
20
+ This model was trained using LoRA available through the [PEFT library](https://github.com/huggingface/peft).
21
+
22
+ NOTE: The <cls> token must be manually added to the beginning of the question for this model to work properly. It uses the <cls> token to be able to make "no answer" predictions. The t5 tokenizer does not automatically add this special token which is why it is added manually.
23
+
24
+ ## Overview
25
+ **Language model:** flan-t5-large
26
+ **Language:** English
27
+ **Downstream-task:** Extractive QA
28
+ **Training data:** SQuAD 2.0
29
+ **Eval data:** SQuAD 2.0
30
+ **Infrastructure**: 1x NVIDIA 3070
31
+
32
+ ## Model Usage
33
+
34
+ ### Using Transformers
35
+ This uses the merged weights (base model weights + LoRA weights) to allow for simple use in Transformers pipelines. It has the same performance as using the weights separately when using the PEFT library.
36
+ ```python
37
+ import torch
38
+ from transformers import(
39
+ AutoModelForQuestionAnswering,
40
+ AutoTokenizer,
41
+ pipeline
42
+ )
43
+ model_name = "sjrhuschlee/flan-t5-large-squad2"
44
+
45
+ # a) Using pipelines
46
+ nlp = pipeline('question-answering', model=model_name, tokenizer=model_name)
47
+ qa_input = {
48
+ 'question': f'{nlp.tokenizer.cls_token}Where do I live?', # '<cls>Where do I live?'
49
+ 'context': 'My name is Sarah and I live in London'
50
+ }
51
+ res = nlp(qa_input)
52
+ # {'score': 0.984, 'start': 30, 'end': 37, 'answer': ' London'}
53
+
54
+ # b) Load model & tokenizer
55
+ model = AutoModelForQuestionAnswering.from_pretrained(model_name)
56
+ tokenizer = AutoTokenizer.from_pretrained(model_name)
57
+
58
+ question = f'{tokenizer.cls_token}Where do I live?' # '<cls>Where do I live?'
59
+ context = 'My name is Sarah and I live in London'
60
+ encoding = tokenizer(question, context, return_tensors="pt")
61
+ start_scores, end_scores = model(
62
+ encoding["input_ids"],
63
+ attention_mask=encoding["attention_mask"],
64
+ return_dict=False
65
+ )
66
+
67
+ all_tokens = tokenizer.convert_ids_to_tokens(input_ids[0].tolist())
68
+ answer_tokens = all_tokens[torch.argmax(start_scores):torch.argmax(end_scores) + 1]
69
+ answer = tokenizer.decode(tokenizer.convert_tokens_to_ids(answer_tokens))
70
+ # 'London'
71
+ ```
72
+
73
+ ### Using with Peft
74
+ **NOTE**: This requires code in the PR https://github.com/huggingface/peft/pull/473 for the PEFT library.
75
+ ```python
76
+ #!pip install peft
77
+
78
+ from peft import LoraConfig, PeftModelForQuestionAnswering
79
+ from transformers import AutoModelForQuestionAnswering, AutoTokenizer
80
+ model_name = "sjrhuschlee/flan-t5-large-squad2"
81
+ ```