Before revision:  These encodings are often implemented as learnable fixed-dimension vectors or sinusoidal functions of different frequencies , which are not possible to accommodate variable-length input sequences. This inevitably limits a wider application of transformers in vision, where many tasks require changing the input size on-the-fly. In this paper, we propose to employ a conditional position encoding scheme, which is conditioned on the local neighborhood of the input token.
After revision:  These encodings are often implemented as learnable fixed-dimension vectors or sinusoidal functions of different frequencies , which are pre-defined and independent of input tokens, CPE is dynamically generated and conditioned on the local neighborhood of the input token.