Update README.md
Browse files
README.md
CHANGED
@@ -33,13 +33,18 @@ I'm interested in using encoder-based extraction of named legal document section
|
|
33 |
|
34 |
### Using ROBERTA for segmentation involves combining sentences A and B into single input to ROBERTA. See below:
|
35 |
|
36 |
-
|
|
|
|
|
37 |
|
38 |
and the embedding for [cls] can be used in a binary classifier.
|
39 |
|
40 |
### But, the architecture used here (implemented via ) is:
|
41 |
|
42 |
-
1. standard ROBERTA model
|
|
|
|
|
|
|
43 |
2. classification of [CLS] token embedding:
|
44 |
|
45 |
```
|
|
|
33 |
|
34 |
### Using ROBERTA for segmentation involves combining sentences A and B into single input to ROBERTA. See below:
|
35 |
|
36 |
+
1. standard ROBERTA model on pairwise sentences ((512 / 2) - 3 tokens, max, per sentence)
|
37 |
+
|
38 |
+
[cls] A [sep] B [SEP]
|
39 |
|
40 |
and the embedding for [cls] can be used in a binary classifier.
|
41 |
|
42 |
### But, the architecture used here (implemented via ) is:
|
43 |
|
44 |
+
1. standard ROBERTA model, but instead input is just
|
45 |
+
|
46 |
+
[CLS] A [SEP]
|
47 |
+
|
48 |
2. classification of [CLS] token embedding:
|
49 |
|
50 |
```
|