asahi417 commited on
Commit
8ec27fd
1 Parent(s): 39067cd

commit files to HF hub

Browse files
Files changed (1) hide show
  1. README.md +18 -0
README.md ADDED
@@ -0,0 +1,18 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Vocabulary Trimmed [lmqg/mt5-base-zhquad-qag](https://huggingface.co/lmqg/mt5-base-zhquad-qag): `lmqg/mt5-base-zhquad-qag-trimmed-50000`
2
+ This model is a trimmed version of [lmqg/mt5-base-zhquad-qag](https://huggingface.co/lmqg/mt5-base-zhquad-qag) by [`vocabtrimmer`](https://github.com/asahi417/lm-vocab-trimmer), a tool for trimming vocabulary of language models to compress the model size.
3
+ Following table shows a summary of the trimming process.
4
+
5
+ | | lmqg/mt5-base-zhquad-qag | lmqg/mt5-base-zhquad-qag-trimmed-50000 |
6
+ |:---------------------------|:---------------------------|:-----------------------------------------|
7
+ | parameter_size_full | 582,384,384 | 275,032,320 |
8
+ | parameter_size_embedding | 384,155,136 | 76,803,072 |
9
+ | vocab_size | 250,101 | 50,002 |
10
+ | compression_rate_full | 100.0 | 47.23 |
11
+ | compression_rate_embedding | 100.0 | 19.99 |
12
+
13
+
14
+ Following table shows the parameter used to trim vocabulary.
15
+
16
+ | language | dataset | dataset_column | dataset_name | dataset_split | target_vocab_size | min_frequency |
17
+ |:-----------|:----------------------------|:-----------------|:---------------|:----------------|--------------------:|----------------:|
18
+ | zh | vocabtrimmer/mc4_validation | text | zh | validation | 50000 | 2 |