Update README.md
Browse files
README.md
CHANGED
@@ -116,4 +116,18 @@ prompt = "mnli hypothesis: Your contributions were of no help with our students'
|
|
116 |
"premise: Your contribution helped make it possible for us to provide our students with a quality education. target:"
|
117 |
out = generator(prompt, return_full_text=False, max_new_tokens=5, top_k=1)
|
118 |
# [{'generated_text': ' contradiction'}]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
119 |
```
|
|
|
116 |
"premise: Your contribution helped make it possible for us to provide our students with a quality education. target:"
|
117 |
out = generator(prompt, return_full_text=False, max_new_tokens=5, top_k=1)
|
118 |
# [{'generated_text': ' contradiction'}]
|
119 |
+
```
|
120 |
+
|
121 |
+
You can create prompt-like inputs for the model using the `data_utils.py` script provided.
|
122 |
+
```python
|
123 |
+
from datasets import load_dataset
|
124 |
+
from data_utils import form_text, split_text
|
125 |
+
|
126 |
+
dataset = load_dataset('glue', 'mnli', split='validation_mismatched')
|
127 |
+
dataset = dataset.map(
|
128 |
+
form_text, remove_columns=['hypothesis', 'premise','label', 'idx'])
|
129 |
+
# dataset[0] {'text': "mnli hypothesis: Your contributions were of no help with our students' education. premise: Your contribution helped make it possible for us to provide our students with a quality education. target: contradiction<|endoftext|>"}
|
130 |
+
dataset = dataset.map(split_text, remove_columns=['text'])
|
131 |
+
# dataset[0] {'prompt_text': "mnli hypothesis: Your contributions were of no help with our students' education. premise: Your contribution helped make it possible for us to provide our students with a quality education. target:",
|
132 |
+
# 'class_label': 'contradiction'}
|
133 |
```
|