asahi417 commited on
Commit
256d6d3
1 Parent(s): f29b7d7

commit files to HF hub

Browse files
Files changed (1) hide show
  1. README.md +18 -0
README.md ADDED
@@ -0,0 +1,18 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Vocabulary Trimmed [google/mt5-small](https://huggingface.co/google/mt5-small): `vocabtrimmer/mt5-small-trimmed-ko`
2
+ This model is a trimmed version of [google/mt5-small](https://huggingface.co/google/mt5-small) by [`vocabtrimmer`](https://github.com/asahi417/lm-vocab-trimmer), a tool for trimming vocabulary of language models to compress the model size.
3
+ Following table shows a summary of the trimming process.
4
+
5
+ | | google/mt5-small | vocabtrimmer/mt5-small-trimmed-ko |
6
+ |:---------------------------|:-------------------|:------------------------------------|
7
+ | parameter_size_full | 300,176,768 | 119,178,624 |
8
+ | parameter_size_embedding | 256,114,688 | 75,116,544 |
9
+ | vocab_size | 250,112 | 73,356 |
10
+ | compression_rate_full | 100.0 | 39.7 |
11
+ | compression_rate_embedding | 100.0 | 29.33 |
12
+
13
+
14
+ Following table shows the parameter used to trim vocabulary.
15
+
16
+ | language | dataset | dataset_column | dataset_name | dataset_split | target_vocab_size | min_frequency |
17
+ |:-----------|:----------------------------|:-----------------|:---------------|:----------------|:--------------------|----------------:|
18
+ | ko | vocabtrimmer/mc4_validation | text | ko | validation | | 2 |