

TITLE
Add an efficient vision transformer backbone in ICLR 2022: CrossFormer

COMMENTS
1

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
### Model description

The CrossFormer has three new things that does not exist in other ViTs (such as Swin):

1. The cross-scale embedding layer(CEL) that generate cross-scale embeddings as ViT's input.
2. The long-short distance attention (LSDA) mechanism, which is an efficient replacement of the vanilla self-attention and shows better performance than Swin
3. A dynamic relative position bias, a kind of relative position bias that support dynamic group size.

### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

The open source website: https://github.com/cheerss/CrossFormer
The paper was accepted in ICLR 2022: https://openreview.net/forum?id=_PHymLIxuI