WARNING:tensorflow:Gradients do not exist for variables ['tf_bert_model/bert/pooler/dense/kernel:0', 'tf_bert_model/bert/pooler/dense/bias:0'] when minimizing the loss. WARNING:tensorflow:Gradients do not exist for variables ['tf_bert_model/bert/pooler/dense/kernel:0', 'tf_bert_model/bert/pooler/dense/bias:0'] when minimizing the loss.
Author: dhirajgite
Pull request: no

# 🐛 Bug

## Information

model I am using (Bert, XLNet ...): Bert

language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

ip1 = Input(shape = (max_length+2,), dtype="int32")
ip2 = Input(shape = (max_length+2,), dtype="int32")
ip3 = Input(shape = (max_length+2,), dtype="int32")
Bert_model = TFBertModel.from_pretrained('bert-base-uncased')
ip = Bert_model(ip1, attention_mask=ip2, token_type_ids=ip3)[0][:,1:-1,:]
out = Bidirectional(LSTM(units=768))(ip)
out = Dense(384, activation='relu')(out)
out = Dropout(0.2)(out)
out = Dense(units=9, activation="softmax")(out)
model = Model([ip1, ip2, ip3], out)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce
When use Model.fit gives following warnings

WARNING:tensorflow:Gradients do not exist for variables ['tf_bert_model/bert/pooler/dense/kernel:0', 'tf_bert_model/bert/pooler/dense/bias:0'] when minimizing the loss.
WARNING:tensorflow:Gradients do not exist for variables ['tf_bert_model/bert/pooler/dense/kernel:0', 'tf_bert_model/bert/pooler/dense/bias:0'] when minimizing the loss.


Steps to reproduce the behavior:

1.
2.
3.

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version:
- Platform:windows 10
- Python version:3.7
- PyTorch version (GPU?):
- Tensorflow version (GPU?):2.1.0
- Using GPU in script?:NO- Using distributed or parallel set-up in script?:NO


Comments:
I am also encountering a similar issue from yesterday. It never happened before.
```
WARNING:tensorflow:Gradients do not exist for variables ['tf_bert_model_1/bert/pooler/dense/kernel:0', 'tf_bert_model_1/bert/pooler/dense/bias:0'] when minimizing the loss.
WARNING:tensorflow:Gradients do not exist for variables ['tf_bert_model_1/bert/pooler/dense/kernel:0', 'tf_bert_model_1/bert/pooler/dense/bias:0'] when minimizing the loss.
WARNING:tensorflow:Gradients do not exist for variables ['tf_bert_model_1/bert/pooler/dense/kernel:0', 'tf_bert_model_1/bert/pooler/dense/bias:0'] when minimizing the loss.
WARNING:tensorflow:Gradients do not exist for variables ['tf_bert_model_1/bert/pooler/dense/kernel:0', 'tf_bert_model_1/bert/pooler/dense/bias:0'] when minimizing the loss.
```

This https://github.com/huggingface/transformers/issues/5421#issuecomment-652626787 may be useful.

Closed by mistake

Hello everyone,
I am fine-tuning a BERT model from huggingface transformers for Named Entity Recognition Task in tensorflow. The input to the model is a single word and output is a tag of that word. I have created a custom generator function (data_generator) from where I am getting data while training. I have freezed the bert layer in training mode and added some layers on top of it to predict the tag of the given word.

**The code is this :** 

```python
from tensorflow.keras.layers import Input, Dense, Activation, Dropout, LSTM, GlobalMaxPool1D
from tensorflow.keras.models import Model
from tensorflow.keras.utils import to_categorical
from tensorflow.keras.preprocessing.sequence import pad_sequences

from transformers import BertTokenizer, TFBertModel, BertConfig

##Load the BERT tokenizer.

print('Loading BERT tokenizer...')
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased', do_lower_case=True)

bert = 'bert-base-uncased'

config = BertConfig(dropout=0.2, attention_dropout=0.2)
config.output_hidden_states = False
transformer_model = TFBertModel.from_pretrained(bert, config = config)

input_ids_in = Input(shape=(max_len,), name='input_token', dtype='int32')
input_masks_in = Input(shape=(max_len,), name='masked_token', dtype='int32')
embedding_layer = transformer_model(input_ids_in, attention_mask=input_masks_in)[0]

X = LSTM(50, return_sequences=True)(embedding_layer)
X = GlobalMaxPool1D()(X)
X = Dense(50, activation='relu')(X)
X = Dropout(0.2)(X)
X = Dense(num_labels, activation='softmax')(X)

model = Model(inputs=[input_ids_in, input_masks_in], outputs = X)

for layer in model.layers[:3]:
    layer.trainable = False

model.compile(loss='categorical_crossentropy', optimizer='adam')

train_gen = data_generator(sentences, tags, tag2ix, max_len, number_sent_per_batch)
model.fit(train_gen, epochs=1, steps_per_epoch=steps, verbose=1)
```

**The error I am getting is this :** 
```python
ValueError: in user code:

    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:571 train_function  *
        outputs = self.distribute_strategy.run(
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:951 run  **
        return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:2290 call_for_each_replica
        return self._call_for_each_replica(fn, args, kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:2649 _call_for_each_replica
        return fn(*args, **kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:541 train_step  **
        self.trainable_variables)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:1804 _minimize
        trainable_variables))
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/optimizer_v2/optimizer_v2.py:521 _aggregate_gradients
        filtered_grads_and_vars = _filter_grads(grads_and_vars)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/optimizer_v2/optimizer_v2.py:1219 _filter_grads
        ([v.name for _, v in grads_and_vars],))

    ValueError: No gradients provided for any variable: ['lstm_2/lstm_cell_2/kernel:0', 'lstm_2/lstm_cell_2/recurrent_kernel:0', 'lstm_2/lstm_cell_2/bias:0', 'dense_8/kernel:0', 'dense_8/bias:0', 'dense_9/kernel:0', 'dense_9/bias:0'].
```

I have gone through many links like :

<https://github.com/tensorflow/tensorflow/issues/1511>

<https://github.com/tensorflow/tensorflow/issues/27949>

<https://github.com/huggingface/transformers/issues/5421>

and many more.

There are many solutions provided in these github issues but couldn't find the solution of my error. I have even posted on stackoverflow (<https://stackoverflow.com/questions/62863374/valueerror-no-gradients-provided-for-any-variable-in-tensorflow-2-2-0>) but couldn't find the solution.

If someone can point the mistake, it would be of great help. Thanks in advance!

Tensorflow Version : 2.2.0

👀

需要冻结一部分参数， 我的理解是bert模型自带的利用CLS做的分类模型，没有使用最后的时候还传入了gradient进行模型的梯度更新，所以会报错。要冻结那部分参数！！！