MaxJeblick commited on
Commit
1f7b856
·
1 Parent(s): bcf35ee

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -5
README.md CHANGED
@@ -1,5 +1,4 @@
1
- Small dummy LLama2-type Model useable for Unit/Integration tests.
2
- Ensure that model input ids are < 100, see code below.
3
 
4
 
5
  ```python
@@ -14,13 +13,11 @@ config.intermediate_size = 24
14
  config.num_attention_heads = 2
15
  config.num_hidden_layers = 2
16
  config.num_key_value_heads = 2
17
- config.vocab_size = 100
18
 
19
  tokenizer = AutoTokenizer.from_pretrained(model_name)
20
 
21
-
22
  model = AutoModelForCausalLM.from_config(config)
23
- print(model.num_parameters()) # 5340
24
 
25
  model.push_to_hub(repo_name, private=False)
26
  tokenizer.push_to_hub(repo_name, private=False)
 
1
+ Small dummy LLama2-type Model useable for Unit/Integration tests.
 
2
 
3
 
4
  ```python
 
13
  config.num_attention_heads = 2
14
  config.num_hidden_layers = 2
15
  config.num_key_value_heads = 2
 
16
 
17
  tokenizer = AutoTokenizer.from_pretrained(model_name)
18
 
 
19
  model = AutoModelForCausalLM.from_config(config)
20
+ print(model.num_parameters()) # 770_940
21
 
22
  model.push_to_hub(repo_name, private=False)
23
  tokenizer.push_to_hub(repo_name, private=False)