Fine Tune Bert Ner using TFBertForTokenClassification.from_pretrained
Author: aks2193
Pull request: no
Labels: name: wontfix

Hey,

I am new to the transformers Bert Training world and am trying to fine tune Bert model for NER on a coNll like dataset. But its not training the model and giving the below error

ValueError: No gradients provided for any variable: ['tf_bert_for_token_classification_8/classifier/kernel:0', 'tf_bert_for_token_classification_8/classifier/bias:0'].


Below is my code

```py
tr_inputs = tf.convert_to_tensor(tr_inputs)
val_inputs = tf.convert_to_tensor(val_inputs)
tr_tags = tf.convert_to_tensor(tr_tags)
val_tags = tf.convert_to_tensor(val_tags)
tr_masks = tf.convert_to_tensor(tr_masks)
val_masks = tf.convert_to_tensor(val_masks)
tr_segs = tf.convert_to_tensor(tr_segs)
val_segs = tf.convert_to_tensor(val_segs)

input_features_dict = {"input_ids":tr_inputs, "attention_mask":tr_masks, "token_type_ids":tr_segs, 'labels':tr_tags}
val_features_dict = {"input_ids":val_inputs, "attention_mask":val_masks, "token_type_ids":val_segs, 'labels':tr_tags}
train_data = tf.data.Dataset.from_tensor_slices(input_features_dict)
batch_train_data = train_data.batch(batch_num)
valid_data = tf.data.Dataset.from_tensor_slices(val_features_dict)
batch_valid_data = valid_data.batch(batch_num)
modell = TFBertForTokenClassification.from_pretrained('bert-base-uncased',num_labels=len(tag2idx))
modell.layers[2].activation = tf.keras.activations.softmax
modell.layers[0].trainable = False
modell.compile(optimizer=optimizer, loss=loss, metrics=[metrics])
modell.fit(batch_train_data, epochs=epochs, validation_data=batch_val_data)
```

Not sure what needs to be done. Any advice/pointers on this would be highly helpful for me.

Comments:
What is your `metrics`?

```py
optimizer= AdamWeightDecay(
    learning_rate=5e-5,
    beta_1=0.9,
    beta_2=0.999,
    weight_decay_rate=0.01,
    epsilon=1e-6,
    exclude_from_weight_decay=['layer_norm', 'bias'])
optimizer._HAS_AGGREGATE_GRAD = False
loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
metrics=[tf.keras.metrics.SparseCategoricalAccuracy(name="acc")]
```

@jplu might know what's going on

Hello @aks2193!

Sorry for this, but for now you cannot use `.compile()` + `.fit()` to train a Token Classification model. To make it short, this is because a layer is not used and then the gradients will be None, something that `.fit()` cannot handle.

If you want to train a NER I suggest you to use the [example](https://github.com/huggingface/transformers/blob/master/examples/token-classification/run_tf_ner.py).

Hey @jplu 
Thanks for the reply. I tried to follow the link and below is how I changed my code

modell = TFBertForTokenClassification.from_pretrained('bert-base-uncased',num_labels=len(tag2idx))
modell.layers[2].activation = tf.keras.activations.softmax
modell.layers[0].trainable = False
modell.compile(optimizer=optimizer, loss=loss, metrics=[metrics])
modell.fit(batch_train_data, epochs=epochs, validation_data=batch_val_data)

def compute_metrics(pred):
        labels = pred.label_ids
        preds = pred.predictions.argmax(-1)
        precision, recall, f1, _ = precision_recall_fscore_support(labels, preds, average='binary')
        acc = accuracy_score(labels, preds)
        return {
            'accuracy': acc,
            'f1': f1,
            'precision': precision,
            'recall': recall
        }
training_args = TFTrainingArguments(
        output_dir='./bert_test',          # output directory
        num_train_epochs=5,              # total # of training epochs
        per_device_train_batch_size=32,  # batch size per device during training
        per_device_eval_batch_size=32,   # batch size for evaluation
        warmup_steps=500,                # number of warmup steps for learning rate scheduler
        weight_decay=0.01,               # strength of weight decay
        logging_dir='./logs',            # directory for storing logs
        learning_rate=3e-5,
    )

trainer = TFTrainer(
        model = modell,
        args=training_args,
        train_dataset=train_dataset,
        eval_dataset=val_dataset,
        compute_metrics=compute_metrics
    )
    
trainer.train()

But I am still getting the below error
 ValueError: Trying to create optimizer slot variable under the scope for tf.distribute.Strategy (<tensorflow.python.distribute.one_device_strategy.OneDeviceStrategy object at 0x7fb46c2dec50>), which is different from the scope used for the original variable (<tf.Variable 'tf_bert_for_token_classification_14/classifier/kernel:0' shape=(768, 10) dtype=float32
Make sure the slot variables are created under the same strategy scope. This may happen if you're restoring from a checkpoint outside the scope


Then I tried defining a strategy scope and include all the above code inside that

strategy = tf.distribute.OneDeviceStrategy(device="/cpu:0")
with strategy.scope():
    The ABOVE CODE

On doing this getting the below error

Mixing different tf.distribute.Strategy objects: <tensorflow.python.distribute.one_device_strategy.OneDeviceStrategy object at 0x7fb446d98eb8> is not <tensorflow.python.distribute.one_device_strategy.OneDeviceStrategy object at 0x7fb45fe2d7b8>

How does this exactly work?
How do I define the strategy scope for all this calculations?

Hey @aks2193 
I'm facing the same problem. Please let me know if you find a solution.

@aks2193 
Try replacing `strategy = tf.distribute.OneDeviceStrategy(device="/cpu:0")` to `training_args.strategy.scope()`. Worked for me.

@alibi123 has right, you are not properly instanciate your model. Please use the example as it is. You example won't work as well with the `TFTrainer` if you are setting the activation to `softmax` because we don't compute the loss from the logits.

@jplu I have a question about restoring weights from the checkpoint. How to do it correctly?

This is how I try to load weights:
```
>>> model = TFBertForTokenClassification.from_pretrained(settings.BERT_NAME)
>>> model.load_weights('/models/exp1/checkpoint/ckpt-55')
```
I get very long exception message starting with:
```
Nothing except the root object matched a checkpointed value. Typically this means that the checkpoint does not match the Python program. The following objects have no matching checkpointed value: [<tf.Variable 't
```

Here's my training code:
```
    dataset = get_dataset(in_fn, debug)
    args = TFTrainingArguments(
        os.path.join(os.path.join(settings.MODELS_DIR, save_name)),
        overwrite_output_dir=True,
        do_train=True,
        logging_dir=os.path.join(settings.DATA_DIR, 'exp1_logs'),
        save_total_limit=2,
    )
    with args.strategy.scope():
        model = TFBertForTokenClassification.from_pretrained(settings.BERT_NAME)
    trainer = TFTrainer(
        model=model,
        args=args,
        train_dataset=dataset,
    )
    trainer.train()
```
`BERT_NAME = 'bert-base-multilingual-cased'`

I was also trying to use ckpt path in `.from_pretrained()` but also got errors regarding format.


You cannot use Keras `load_weights` on a TF checkpoint. If you want to load your model you just have to use the path where you saved your model `model = TFBertForTokenClassification.from_pretrained("my_output_dir")`

@jplu Thank you for the quick response.
I've tried that in the first place, but get an error as well. 

This is my output dir:
```
checkpoint$ ls
checkpoint  ckpt-55.data-00000-of-00001  ckpt-55.index  ckpt-56.data-00000-of-00001  ckpt-56.index
```
Here are my attempts with error messages:
1:
```
config = BertConfig.from_pretrained(settings.BERT_NAME)
>>> model = TFBertForTokenClassification.from_pretrained('/models/exp1/checkpoint', config=config)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_utils.py", line 653, in from_pretrained
    [WEIGHTS_NAME, TF2_WEIGHTS_NAME], pretrained_model_name_or_path
OSError: Error no file named ['pytorch_model.bin', 'tf_model.h5'] found in directory /models/exp1/checkpoint or `from_pt` set to False
```
2:
```
>>> model = TFBertForTokenClassification.from_pretrained('/models/exp1/checkpoint/ckpt-55', config=config)
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_utils.py", line 711, in from_pretrained
    load_tf_weights(model, resolved_archive_file)
  File "/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_utils.py", line 268, in load_tf_weights
    with h5py.File(resolved_archive_file, "r") as f:
  File "/usr/local/lib/python3.6/dist-packages/h5py/_hl/files.py", line 408, in __init__
    swmr=swmr)
  File "/usr/local/lib/python3.6/dist-packages/h5py/_hl/files.py", line 173, in make_fid
    fid = h5f.open(name, flags, fapl=fapl)
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py/h5f.pyx", line 88, in h5py.h5f.open
OSError: Unable to open file (file signature not found)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_utils.py", line 714, in from_pretrained
    "Unable to load weights from h5 file. "
OSError: Unable to load weights from h5 file. If you tried to load a TF 2.0 model from a PyTorch checkpoint, please set from_pt=True.
```
3: I've even tried `from_pt=True` even though I used TFTrainer and TFBert
```
>>> model = TFBertForTokenClassification.from_pretrained('/models/exp1/checkpoint/ckpt-55', config=config, from_pt=True)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_utils.py", line 703, in from_pretrained
    return load_pytorch_checkpoint_in_tf2_model(model, resolved_archive_file, allow_missing_keys=True)
  File "/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_pytorch_utils.py", line 89, in load_pytorch_checkpoint_in_tf2_model
    pt_state_dict = torch.load(pt_path, map_location="cpu")
  File "/usr/local/lib/python3.6/dist-packages/torch/serialization.py", line 595, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
  File "/usr/local/lib/python3.6/dist-packages/torch/serialization.py", line 764, in _legacy_load
    magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: invalid load key, '\x00'.
```
4: I've also tried to add config.json into the output dir
```
>>> model = TFBertForTokenClassification.from_pretrained('/models/exp1/checkpoint')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_utils.py", line 653, in from_pretrained
    [WEIGHTS_NAME, TF2_WEIGHTS_NAME], pretrained_model_name_or_path
OSError: Error no file named ['pytorch_model.bin', 'tf_model.h5'] found in directory /models/exp1/checkpoint or `from_pt` set to False
```

As I said, these are normal errors because: **You cannot use Keras load_weights on a TF checkpoint.** You have to use your output dir not the file or the checkpoint dir: `model = TFBertForTokenClassification.from_pretrained('/models/exp1')`.

@jplu Sorry for bothering. But still doesn't work. It expects `'pytorch_model.bin', 'tf_model.h5'`.
```
>>> model = TFBertForTokenClassification.from_pretrained('/models/exp1')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_utils.py", line 653, in from_pretrained
    [WEIGHTS_NAME, TF2_WEIGHTS_NAME], pretrained_model_name_or_path
OSError: Error no file named ['pytorch_model.bin', 'tf_model.h5'] found in directory /models/exp1 or `from_pt` set to False

>>> model = TFBertForTokenClassification.from_pretrained('/models/exp1', from_pt=True)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_utils.py", line 653, in from_pretrained
    [WEIGHTS_NAME, TF2_WEIGHTS_NAME], pretrained_model_name_or_path
OSError: Error no file named ['pytorch_model.bin', 'tf_model.h5'] found in directory /models/exp1 or `from_pt` set to False

```

But my output_dir only contains `checkpoint` dir

This is because you are trying to load a PyTorch model into a TensorFlow one with `from_pt=True`, remove this parameter. If not working it means that your models have not been properly saved.

Did you call the `save` method of the trainer?

No, I haven't. Sorry, my bad. I thought that I can use checkpoints. 
Thanks for your help!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.