Invalid argument:  Incompatible shapes: [24,1536,12,514] vs. [24,1536,12,513]
Author: user06039
Pull request: no

I had a 16 class classification dataset, but I am getting an error when using longformer, what am I doing wrong here?

```
from transformers import LongformerTokenizerFast, TFLongformerForSequenceClassification
import tensorflow as tf
import pickle

tokenizer = LongformerTokenizerFast.from_pretrained('allenai/longformer-base-4096')
model = TFLongformerForSequenceClassification.from_pretrained('allenai/longformer-base-4096', num_labels=16, gradient_checkpointing=True)

df = pd.read_csv("dataset.csv")
df1 = pd.read_csv("dataset1.csv")

y_train = pickle.load(open("y_train.pkl", "rb"))
y_test = pickle.load(open("y_test.pkl", "rb"))

x_train = tokenizer(df.posts.tolist(), max_length=1500, return_tensors="tf", padding="max_length", truncation=True)
x_test = tokenizer(df1.posts.tolist(), max_length=1500, return_tensors="tf", padding="max_length", truncation=True)

print(y_train.nunique()) # return 16


model.fit(x_train, y_train, batch_size=24, 
          steps_per_epoch=steps_per_epoch, 
          validation_data=(x_test, y_test))
```

Why do I get this shape mismatch error? What am I doing wrong.

```
WARNING:tensorflow:The parameters `output_attentions`, `output_hidden_states` and `use_cache` cannot be updated when calling a model.They have to be set to True/False in the config object (i.e.: `config=XConfig.from_pretrained('name', output_attentions=True)`).
WARNING:tensorflow:AutoGraph could not transform <bound method Socket.send of <zmq.sugar.socket.Socket object at 0x7fd3d7fdf9a0>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: module, class, method, function, traceback, frame, or code object was expected, got cython_function_or_method
To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert
WARNING: AutoGraph could not transform <bound method Socket.send of <zmq.sugar.socket.Socket object at 0x7fd3d7fdf9a0>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: module, class, method, function, traceback, frame, or code object was expected, got cython_function_or_method
To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert
WARNING:tensorflow:The parameter `return_dict` cannot be set in graph mode and will always be set to `True`.
WARNING:tensorflow:From /home/intellectfaces/Desktop/Work/apps/env/lib/python3.8/site-packages/tensorflow/python/ops/array_ops.py:5043: calling gather (from tensorflow.python.ops.array_ops) with validate_indices is deprecated and will be removed in a future version.
Instructions for updating:
The `validate_indices` argument has no effect. Indices are always validated on CPU and never validated on GPU.
WARNING:tensorflow:The parameters `output_attentions`, `output_hidden_states` and `use_cache` cannot be updated when calling a model.They have to be set to True/False in the config object (i.e.: `config=XConfig.from_pretrained('name', output_attentions=True)`).
WARNING:tensorflow:The parameter `return_dict` cannot be set in graph mode and will always be set to `True`.
---------------------------------------------------------------------------
InvalidArgumentError                      Traceback (most recent call last)
<ipython-input-17-c08295c7f1ca> in <module>
----> 1 model.fit(x_train, y_train, batch_size=24, 
      2           steps_per_epoch=steps_per_epoch,
      3           validation_data=(x_test, y_test))

~/Desktop/Work/apps/env/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_batch_size, validation_freq, max_queue_size, workers, use_multiprocessing)
   1155                 _r=1):
   1156               callbacks.on_train_batch_begin(step)
-> 1157               tmp_logs = self.train_function(iterator)
   1158               if data_handler.should_sync:
   1159                 context.async_wait()

~/Desktop/Work/apps/env/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py in __call__(self, *args, **kwds)
    865     tracing_count = self.experimental_get_tracing_count()
    866     with trace.Trace(self._name) as tm:
--> 867       result = self._call(*args, **kwds)
    868       compiler = "xla" if self._jit_compile else "nonXla"
    869       new_tracing_count = self.experimental_get_tracing_count()

~/Desktop/Work/apps/env/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py in _call(self, *args, **kwds)
    926         # Lifting succeeded, so variables are initialized and we can run the
    927         # stateless function.
--> 928         return self._stateless_fn(*args, **kwds)
    929     else:
    930       _, _, _, filtered_flat_args = \

~/Desktop/Work/apps/env/lib/python3.8/site-packages/tensorflow/python/eager/function.py in __call__(self, *args, **kwargs)
   3016       (graph_function,
   3017        filtered_flat_args) = self._maybe_define_function(args, kwargs)
-> 3018     return graph_function._call_flat(
   3019         filtered_flat_args, captured_inputs=graph_function.captured_inputs)  # pylint: disable=protected-access
   3020 

~/Desktop/Work/apps/env/lib/python3.8/site-packages/tensorflow/python/eager/function.py in _call_flat(self, args, captured_inputs, cancellation_manager)
   1958         and executing_eagerly):
   1959       # No tape is watching; skip to running the function.
-> 1960       return self._build_call_outputs(self._inference_function.call(
   1961           ctx, args, cancellation_manager=cancellation_manager))
   1962     forward_backward = self._select_forward_and_backward_functions(

~/Desktop/Work/apps/env/lib/python3.8/site-packages/tensorflow/python/eager/function.py in call(self, ctx, args, cancellation_manager)
    589       with _InterpolateFunctionError(self):
    590         if cancellation_manager is None:
--> 591           outputs = execute.execute(
    592               str(self.signature.name),
    593               num_outputs=self._num_outputs,

~/Desktop/Work/apps/env/lib/python3.8/site-packages/tensorflow/python/eager/execute.py in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
     57   try:
     58     ctx.ensure_initialized()
---> 59     tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
     60                                         inputs, attrs, num_outputs)
     61   except core._NotOkStatusException as e:

InvalidArgumentError: 2 root error(s) found.
  (0) Invalid argument:  Incompatible shapes: [24,1536,12,514] vs. [24,1536,12,513]
	 [[node gradient_tape/tf_longformer_for_sequence_classification/longformer/encoder/layer_._0/attention/self/BroadcastGradientArgs_1 (defined at <ipython-input-17-c08295c7f1ca>:1) ]]
	 [[tf_longformer_for_sequence_classification/longformer/encoder/layer_._11/attention/self/cond_1/pivot_t/_985/_1717]]
  (1) Invalid argument:  Incompatible shapes: [24,1536,12,514] vs. [24,1536,12,513]
	 [[node gradient_tape/tf_longformer_for_sequence_classification/longformer/encoder/layer_._0/attention/self/BroadcastGradientArgs_1 (defined at <ipython-input-17-c08295c7f1ca>:1) ]]
0 successful operations.
0 derived errors ignored. [Op:__inference_train_function_95332]

Function call stack:
train_function -> train_function
​
```

### Environment info
transformers version: 4.3.3
Platform: Ubuntu 20.04 LTS
Python version: 3.8.x
PyTorch version (GPU?): 1.8.0+cu111
Tensorflow version (GPU?): 2.5.0-dev20210311
CUDA: cuda_11.1
Using GPU in script?: Yes
Using distributed or parallel set-up in script?: No

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

I have the same error.

Me too