sberbank-ai
commited on
Commit
•
98e17db
1
Parent(s):
4dc33e2
Update README.md
Browse files
README.md
CHANGED
|
@@ -29,12 +29,12 @@ The model was prepared as a baseline for FusionBrain Challenge 2.0 (as a part of
|
|
| 29 |
* VQA – COCO dataset with prepared question set.
|
| 30 |
* Text Recognition in the Wild
|
| 31 |
|
| 32 |
-
# Details of architecture
|
| 33 |
|
| 34 |
-
|
| 35 |
|
| 36 |
|
| 37 |
-
|
| 38 |
|
| 39 |
The primary proposed method is to modify the sparse transformer's attention mask to better control modalities. It allows us to calculate the transitions of modalities in both directions, unlike another similar work DALL-E Transformer, which used only one direction, "text to image". The proposed "image to right text" direction is achieved by extension sparse attention mask to the right for auto-repressively text generation with both image and left text condition.
|
| 40 |
|
|
|
|
| 29 |
* VQA – COCO dataset with prepared question set.
|
| 30 |
* Text Recognition in the Wild
|
| 31 |
|
| 32 |
+
# Details of architecture
|
| 33 |
|
| 34 |
+
### Parameters
|
| 35 |
|
| 36 |
|
| 37 |
+
### Sparse Attention Mask
|
| 38 |
|
| 39 |
The primary proposed method is to modify the sparse transformer's attention mask to better control modalities. It allows us to calculate the transitions of modalities in both directions, unlike another similar work DALL-E Transformer, which used only one direction, "text to image". The proposed "image to right text" direction is achieved by extension sparse attention mask to the right for auto-repressively text generation with both image and left text condition.
|
| 40 |
|