ai-forever
/

RUDOLPH-2.7B-FBC2

Model card Files Files and versions Community

sberbank-ai commited on Oct 6, 2022

Commit

f02496e

•

1 Parent(s): 66c0378

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -54,7 +54,7 @@ RUDOLPH 2.7B is a Transformer-based decoder model with the following parameters:
 The primary proposed method is to modify the sparse transformer's attention mask to better control modalities. It allows us to calculate the transitions of modalities in both directions, unlike another similar work DALL-E Transformer, which used only one direction, "text to image". The proposed "image to right text" direction is achieved by extension sparse attention mask to the right for auto-repressively text generation with both image and left text condition.
-<img src="https://raw.githubusercontent.com/ai-forever/ru-dolph/master/pics/attention_mask_27b.png" height="20" border="2"/>
 # Authors

 The primary proposed method is to modify the sparse transformer's attention mask to better control modalities. It allows us to calculate the transitions of modalities in both directions, unlike another similar work DALL-E Transformer, which used only one direction, "text to image". The proposed "image to right text" direction is achieved by extension sparse attention mask to the right for auto-repressively text generation with both image and left text condition.
+<img src="https://raw.githubusercontent.com/lizagonch/ru-dolph/develop_v1/pics/attention_masks_2700m.png" height="20" border="2"/>
 # Authors