"ValueError: initial_value must be specified." error when compiling bert for text classification
Author: djellalmohamedaniss
Pull request: no
Labels: name: bug

### System Info

I'm following the hugging face tutorial for the sequence classification and while trying to fine-tune 'distilbert-base-multilingual-cased' I'm having the following error when running the model.compile() method.

I'm having the same error when using bert-uncased.

```
ValueError                                Traceback (most recent call last)
<ipython-input-22-68f5487774e6> in <module>
----> 1 model.compile(optimizer=optimizer)

/tf/Raph/jupyter/tf2-37/lib/python3.7/site-packages/transformers/modeling_tf_utils.py in compile(self, optimizer, loss, metrics, loss_weights, weighted_metrics, run_eagerly, steps_per_execution, **kwargs)
   1035                 run_eagerly=run_eagerly,
   1036                 experimental_steps_per_execution=steps_per_execution,
-> 1037                 **kwargs,
   1038             )
   1039 

/tf/Raph/jupyter/tf2-37/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py in compile(self, optimizer, loss, metrics, loss_weights, weighted_metrics, run_eagerly, **kwargs)
    547       experimental_steps_per_execution = kwargs.pop(
    548           'experimental_steps_per_execution', 1)
--> 549       self._configure_steps_per_execution(experimental_steps_per_execution)
    550 
    551       # Initializes attrs that are reset each time `compile` is called.

/tf/Raph/jupyter/tf2-37/lib/python3.7/site-packages/tensorflow/python/training/tracking/base.py in _method_wrapper(self, *args, **kwargs)
    455     self._self_setattr_tracking = False  # pylint: disable=protected-access
    456     try:
--> 457       result = method(self, *args, **kwargs)
    458     finally:
    459       self._self_setattr_tracking = previous_value  # pylint: disable=protected-access

/tf/Raph/jupyter/tf2-37/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py in _configure_steps_per_execution(self, steps_per_execution)
    581         steps_per_execution,
    582         dtype='int64',
--> 583         aggregation=variables.VariableAggregationV2.ONLY_FIRST_REPLICA)
    584 
    585   @property

/tf/Raph/jupyter/tf2-37/lib/python3.7/site-packages/tensorflow/python/ops/variables.py in __call__(cls, *args, **kwargs)
    260       return cls._variable_v1_call(*args, **kwargs)
    261     elif cls is Variable:
--> 262       return cls._variable_v2_call(*args, **kwargs)
    263     else:
    264       return super(VariableMetaclass, cls).__call__(*args, **kwargs)

/tf/Raph/jupyter/tf2-37/lib/python3.7/site-packages/tensorflow/python/ops/variables.py in _variable_v2_call(cls, initial_value, trainable, validate_shape, caching_device, name, variable_def, dtype, import_scope, constraint, synchronization, aggregation, shape)
    254         synchronization=synchronization,
    255         aggregation=aggregation,
--> 256         shape=shape)
    257 
    258   def __call__(cls, *args, **kwargs):

/tf/Raph/jupyter/tf2-37/lib/python3.7/site-packages/tensorflow/python/ops/variables.py in getter(**kwargs)
     65 
     66   def getter(**kwargs):
---> 67     return captured_getter(captured_previous, **kwargs)
     68 
     69   return getter

/tf/Raph/jupyter/tf2-37/lib/python3.7/site-packages/tensorflow/python/distribute/distribute_lib.py in creator(next_creator, **kwargs)
   2855     def creator(next_creator, **kwargs):
   2856       _require_strategy_scope_strategy(strategy)
-> 2857       return next_creator(**kwargs)
   2858 
   2859     self._var_creator_scope = variable_scope.variable_creator_scope(creator)

/tf/Raph/jupyter/tf2-37/lib/python3.7/site-packages/tensorflow/python/ops/variables.py in <lambda>(**kws)
    235                         shape=None):
    236     """Call on Variable class. Useful to force the signature."""
--> 237     previous_getter = lambda **kws: default_variable_creator_v2(None, **kws)
    238     for _, getter in ops.get_default_graph()._variable_creator_stack:  # pylint: disable=protected-access
    239       previous_getter = _make_getter(getter, previous_getter)

/tf/Raph/jupyter/tf2-37/lib/python3.7/site-packages/tensorflow/python/ops/variable_scope.py in default_variable_creator_v2(next_creator, **kwargs)
   2644       synchronization=synchronization,
   2645       aggregation=aggregation,
-> 2646       shape=shape)
   2647 
   2648 

/tf/Raph/jupyter/tf2-37/lib/python3.7/site-packages/tensorflow/python/ops/variables.py in __call__(cls, *args, **kwargs)
    262       return cls._variable_v2_call(*args, **kwargs)
    263     else:
--> 264       return super(VariableMetaclass, cls).__call__(*args, **kwargs)
    265 
    266 

/tf/Raph/jupyter/tf2-37/lib/python3.7/site-packages/tensorflow/python/ops/resource_variable_ops.py in __init__(self, initial_value, trainable, collections, validate_shape, caching_device, name, dtype, variable_def, import_scope, constraint, distribute_strategy, synchronization, aggregation, shape)
   1516           aggregation=aggregation,
   1517           shape=shape,
-> 1518           distribute_strategy=distribute_strategy)
   1519 
   1520   def _init_from_args(self,

/tf/Raph/jupyter/tf2-37/lib/python3.7/site-packages/tensorflow/python/ops/resource_variable_ops.py in _init_from_args(self, initial_value, trainable, collections, caching_device, name, dtype, constraint, synchronization, aggregation, distribute_strategy, shape)
   1594             synchronization, aggregation, trainable, name))
   1595     if initial_value is None:
-> 1596       raise ValueError("initial_value must be specified.")
   1597     init_from_fn = callable(initial_value)
   1598 

ValueError: initial_value must be specified.
```

tensorflow version: 2.3.0

transformers version: 4.20.1

Python version: 3.7

Any ideas? thanks in advance.

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```python

model_name = 'distilbert-base-multilingual-cased'
model = TFAutoModelForSequenceClassification.from_pretrained(model_name, num_labels=3)

batch_size = 16
num_epochs = 5
batches_per_epoch = len(train) // batch_size
total_train_steps = int(batches_per_epoch * num_epochs)
optimizer, schedule = create_optimizer(init_lr=2e-5, num_warmup_steps=0, num_train_steps=total_train_steps)

loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)

model.compile(optimizer=optimizer, loss=loss)
```

### Expected behavior

a compilation without an error.

Comments:
cc @Rocketknight1 @gante 

Hi @djellalmohamedaniss, your version of TF is quite old - our support for TF 2.3 is very shaky, and we prefer TF >= 2.4, and TF 2.8 or 2.9 are even better!

Can you check with a more recent version of TF and let us know if the problem still exists?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.