Update README.md
Browse files
README.md
CHANGED
@@ -7,7 +7,7 @@ inference: false
|
|
7 |
|
8 |
We have trained a model to evaluate if a paraphrase is a semantic variation to the input query or just a surface level variation. Data augmentation by adding Surface level variations does not add much value to the NLP model training. if the approach to paraphrase generation is "OverGenerate and Rank" , Its important to have a robust model of scoring/ ranking paraphrases. NLG Metrics like bleu ,BleuRT, gleu , Meteor have not proved very effective in scoring paraphrases.
|
9 |
|
10 |
-
```
|
11 |
from transformers import AutoTokenizer, AutoModelForSequenceClassification
|
12 |
import torch
|
13 |
import pandas as pd
|
@@ -55,7 +55,7 @@ print("Paraphrase type detection=====", list(zip(para_pairs, labels)))
|
|
55 |
For more robust results, filter out the paraphrases which are not semantically
|
56 |
similar using a model trained on NLI, STS task and then apply the ranker .
|
57 |
|
58 |
-
```
|
59 |
from transformers import AutoTokenizer, AutoModelWithLMHead
|
60 |
from transformers import AutoModelForSequenceClassification
|
61 |
from sentence_transformers import SentenceTransformer, util
|
|
|
7 |
|
8 |
We have trained a model to evaluate if a paraphrase is a semantic variation to the input query or just a surface level variation. Data augmentation by adding Surface level variations does not add much value to the NLP model training. if the approach to paraphrase generation is "OverGenerate and Rank" , Its important to have a robust model of scoring/ ranking paraphrases. NLG Metrics like bleu ,BleuRT, gleu , Meteor have not proved very effective in scoring paraphrases.
|
9 |
|
10 |
+
```python
|
11 |
from transformers import AutoTokenizer, AutoModelForSequenceClassification
|
12 |
import torch
|
13 |
import pandas as pd
|
|
|
55 |
For more robust results, filter out the paraphrases which are not semantically
|
56 |
similar using a model trained on NLI, STS task and then apply the ranker .
|
57 |
|
58 |
+
```python
|
59 |
from transformers import AutoTokenizer, AutoModelWithLMHead
|
60 |
from transformers import AutoModelForSequenceClassification
|
61 |
from sentence_transformers import SentenceTransformer, util
|