

TITLE
Positinal Encoding for T5 family of models

COMMENTS
1

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
### Feature request

Please create a hook to allow users to modify the T5 family of models and change the default positional embeddings to custom positional embeddings.

### Motivation

I am trying to build a T5 version that takes non-text input, and the traditional positional encodings are getting in the way, and there is no way to switch it off, or to make them learnable parameters, etc.
BART gives limited access to positional encodings, but T5 family gives nearly 0 access.

### Your contribution

If i knew where the positional encoding were calculated and added in to the input_ids, I could create this hook myself