patrickvonplaten
commited on
Commit
•
506cafc
1
Parent(s):
f5991a6
Update README.md
Browse files
README.md
CHANGED
@@ -6,12 +6,12 @@
|
|
6 |
-Rohan V Kashyap ([Rohan](https://huggingface.co/Rohan))
|
7 |
-Vivek V Kashyap ([Vivek](https://huggingface.co/Vivek))
|
8 |
|
9 |
-
##Dataset
|
10 |
|
11 |
[Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoning](https://huggingface.co/datasets/cosmos_qa).This dataset contains a set of 35,600 problems that require commonsense-based reading comprehension, formulated as multiple-choice questions.Understanding narratives requires reading between the lines, which in turn, requires interpreting the likely causes and effects of events, even when they are not mentioned explicitly.The questions focus on factual and literal understanding of the context paragraph, our dataset focuses on reading between the lines over a diverse collection of people's everyday narratives.
|
12 |
|
13 |
|
14 |
-
###Example
|
15 |
|
16 |
```json
|
17 |
{"Context":["It's a very humbling experience when you need someone
|
@@ -32,7 +32,7 @@ dress him every morning?"],
|
|
32 |
}
|
33 |
```
|
34 |
|
35 |
-
##How to use
|
36 |
|
37 |
```bash
|
38 |
# Installing requirements
|
@@ -77,21 +77,21 @@ print(f"the predction of the dataset : {final_output}")
|
|
77 |
The Correct answer:-Option 1
|
78 |
```
|
79 |
|
80 |
-
##Preprocessing
|
81 |
|
82 |
The texts are tokenized using the GPT2 tokenizer.To feed the inputs of multiple choice we concatenated context and question as first input and all the 4 possible choices as the second input to our tokenizer.
|
83 |
|
84 |
## Evaluation
|
85 |
|
86 |
-
The following tables summarize the scores obtained by the **GPT2-CosmosQA**.The ones marked as (
|
87 |
|
88 |
| Model | Dev Acc | Test Acc |
|
89 |
|:---------------:|:-----:|:-----:|
|
90 |
-
| BERT-FT Multiway
|
91 |
-
| GPT-FT
|
92 |
| GPT2-CosmosQA | 60.3 | 59.7 |
|
93 |
|
94 |
-
##Inference
|
95 |
|
96 |
This project was mainly to test the common sense understanding of the GPT2-model.We finetuned on a Dataset known as CosmosQ requires reasoning beyond the exact text spans in the context.The above results shows that GPT2 model is doing better than most of the base line models given that it only used to predict the next word in the pre-training objective.
|
97 |
|
|
|
6 |
-Rohan V Kashyap ([Rohan](https://huggingface.co/Rohan))
|
7 |
-Vivek V Kashyap ([Vivek](https://huggingface.co/Vivek))
|
8 |
|
9 |
+
## Dataset
|
10 |
|
11 |
[Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoning](https://huggingface.co/datasets/cosmos_qa).This dataset contains a set of 35,600 problems that require commonsense-based reading comprehension, formulated as multiple-choice questions.Understanding narratives requires reading between the lines, which in turn, requires interpreting the likely causes and effects of events, even when they are not mentioned explicitly.The questions focus on factual and literal understanding of the context paragraph, our dataset focuses on reading between the lines over a diverse collection of people's everyday narratives.
|
12 |
|
13 |
|
14 |
+
### Example
|
15 |
|
16 |
```json
|
17 |
{"Context":["It's a very humbling experience when you need someone
|
|
|
32 |
}
|
33 |
```
|
34 |
|
35 |
+
## How to use
|
36 |
|
37 |
```bash
|
38 |
# Installing requirements
|
|
|
77 |
The Correct answer:-Option 1
|
78 |
```
|
79 |
|
80 |
+
## Preprocessing
|
81 |
|
82 |
The texts are tokenized using the GPT2 tokenizer.To feed the inputs of multiple choice we concatenated context and question as first input and all the 4 possible choices as the second input to our tokenizer.
|
83 |
|
84 |
## Evaluation
|
85 |
|
86 |
+
The following tables summarize the scores obtained by the **GPT2-CosmosQA**.The ones marked as (^) are the baseline models.
|
87 |
|
88 |
| Model | Dev Acc | Test Acc |
|
89 |
|:---------------:|:-----:|:-----:|
|
90 |
+
| BERT-FT Multiway^| 68.3.| 68.4 |
|
91 |
+
| GPT-FT ^ | 54.0 | 54.4. |
|
92 |
| GPT2-CosmosQA | 60.3 | 59.7 |
|
93 |
|
94 |
+
## Inference
|
95 |
|
96 |
This project was mainly to test the common sense understanding of the GPT2-model.We finetuned on a Dataset known as CosmosQ requires reasoning beyond the exact text spans in the context.The above results shows that GPT2 model is doing better than most of the base line models given that it only used to predict the next word in the pre-training objective.
|
97 |
|