puneeshkhanna commited on
Commit
d5488b9
1 Parent(s): b61e7ea

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -21,7 +21,7 @@ Falcon3-7B-Base supports 4 languages (english, french, spanish, portuguese) and
21
  - Architecture
22
  - Transformer based causal decoder only architecture
23
  - 28 decoder blocks
24
- - Grouped query attention (GQA) for faster inference: 12 query heads and 4 KV heads
25
  - Wider head dimension: 256
26
  - High RoPE value to support long context understanding: 1000042
27
  - 32k context length
 
21
  - Architecture
22
  - Transformer based causal decoder only architecture
23
  - 28 decoder blocks
24
+ - Grouped query attention (GQA) for faster inference: 12 query heads and 4 key value heads
25
  - Wider head dimension: 256
26
  - High RoPE value to support long context understanding: 1000042
27
  - 32k context length