Update README.md
Browse files
README.md
CHANGED
@@ -11,10 +11,10 @@ license: bigscience-openrail-m
|
|
11 |
[GeoV](https://github.com/geov-ai/geov)-9B is a 9 billion parameter causal language model.
|
12 |
|
13 |
The GeoV model was designed by Georges Harik and uses
|
14 |
-
[Rotary Positional Embeddings with Relative distances (RoPER)](
|
15 |
by [Georges Harik](https://twitter.com/gharik) and [Varuna Jayasiri](https://twitter.com/vpj).
|
16 |
|
17 |
-
[RoPER](
|
18 |
in addition to using relative positions in the attention score calculation by RoPE embeddings,
|
19 |
adds relative positional information explicitly to value embeddings.
|
20 |
Specifically, it incorporates the relative positions of the tokens paid attention to.
|
|
|
11 |
[GeoV](https://github.com/geov-ai/geov)-9B is a 9 billion parameter causal language model.
|
12 |
|
13 |
The GeoV model was designed by Georges Harik and uses
|
14 |
+
[Rotary Positional Embeddings with Relative distances (RoPER)](https://research.labml.ai/RoPER.html)
|
15 |
by [Georges Harik](https://twitter.com/gharik) and [Varuna Jayasiri](https://twitter.com/vpj).
|
16 |
|
17 |
+
[RoPER](https://research.labml.ai/RoPER.html),
|
18 |
in addition to using relative positions in the attention score calculation by RoPE embeddings,
|
19 |
adds relative positional information explicitly to value embeddings.
|
20 |
Specifically, it incorporates the relative positions of the tokens paid attention to.
|