xformAI commited on
Commit
0b541b4
1 Parent(s): e1a0b2c

Update README.md

Browse files

This is a GQA version of the original model facebook/opt-125m. In this version, the original MHA architecture is preserved but instead of having a single K/V head, different K/V heads corresponding to the same group have the same mean-pooled K or V values. It has 6 groups of KV heads per layer instead of original 12 KV heads in the MHA implementation.

Files changed (1) hide show
  1. README.md +4 -1
README.md CHANGED
@@ -1,3 +1,6 @@
1
  ---
2
  license: mit
3
- ---
 
 
 
 
1
  ---
2
  license: mit
3
+ language:
4
+ - en
5
+ library_name: transformers
6
+ ---