Update README.md
Browse files
README.md
CHANGED
@@ -1,5 +1,7 @@
|
|
1 |
## About
|
2 |
-
Here we share a pretrained
|
|
|
|
|
3 |
|
4 |
### Usage
|
5 |
Download and try it out
|
@@ -14,7 +16,7 @@ python test.py --test_file test.txt
|
|
14 |
### Test file format
|
15 |
Modify the test examples in `test.txt` to play with it.
|
16 |
|
17 |
-
The test file is tab
|
18 |
|
19 |
### Example output
|
20 |
![](https://i.imgur.com/xpl87KO.png)
|
|
|
1 |
## About
|
2 |
+
Here we share a pretrained BERT model that is aware of math tokens. The math tokens are treated specially and tokenized using [pya0](https://github.com/approach0/pya0), which adds very limited new tokens for latex markup (total vocabulary is just 31,061).
|
3 |
+
|
4 |
+
This model is trained on 4 x 2 Tesla V100 with a total batch size of 64, using Math StackExchange data with 2.7 million sentence pairs for 7 epochs.
|
5 |
|
6 |
### Usage
|
7 |
Download and try it out
|
|
|
16 |
### Test file format
|
17 |
Modify the test examples in `test.txt` to play with it.
|
18 |
|
19 |
+
The test file is tab-separated, the first column is additional positions you want to mask for the right-side sentence (useful for masking tokens in math markups). A zero means no additional mask positions.
|
20 |
|
21 |
### Example output
|
22 |
![](https://i.imgur.com/xpl87KO.png)
|