Using `-1` to mask the loss for the token is deprecated. Please use `-100` instead. 
Author: Neptune-Trojans
Pull request: no

- `transformers` version: 3.3.1
- Platform: Linux-4.15.0-118-generic-x86_64-with-debian-stretch-sid
- Python version: 3.6.8
- PyTorch version (GPU?): 1.3.1 (True)
- Tensorflow version (GPU?): 2.3.1 (True)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: No

### Who can help
@sgugger

## Information

Model I am using (Bert, XLNet ...): BERT

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Just run attached script

## Expected behavior
I dont have '-1' masks with my labels but I get those warnings.
Expected behavior not to have those warnings when I make training.

<!-- A clear and concise description of what you would expect to happen. -->
`python3.6/site-packages/tensorflow/python/autograph/impl/api.py:493: UserWarning: Using `-1` to mask the loss for the token is deprecated. Please use `-100` instead.`
  return py_builtins.overload_of(f)(*args)

Code that reproduces the issue:
```
import os
os.environ["CUDA_VISIBLE_DEVICES"] = "-1"

import transformers
import numpy as np
import tensorflow as tf
from transformers import BertConfig, TFTrainer, TFTrainingArguments, TFBertForTokenClassification

transformers.logging.set_verbosity_info()

labels = np.ones((32, 18))
labels_as_tensor = tf.convert_to_tensor(
    labels, dtype=tf.int32, dtype_hint=None, name=None
)

inputs_embeds = np.random.normal(size=(32, 18, 768))
inputs_embeds_as_tensor = tf.convert_to_tensor(
    inputs_embeds, dtype=tf.float32, dtype_hint=None, name=None
)

token_type_ids = np.ones((32, 18))
token_type_ids_as_tensor = tf.convert_to_tensor(
    token_type_ids, dtype=tf.int32, dtype_hint=None, name=None
)

batch = ({
'inputs_embeds': inputs_embeds_as_tensor,
'token_type_ids': token_type_ids_as_tensor
}, labels_as_tensor)

training_args = TFTrainingArguments(output_dir='~/tensorboard',
                                    overwrite_output_dir=True,
                                    learning_rate=0.001,
                                    logging_dir='~/tensorboard',
                                    debug=True,
                                    do_train=True,
                                    do_predict=True,
                                    num_train_epochs=2,
                                    per_device_train_batch_size=32,
                                    per_device_eval_batch_size=32,
                                    save_total_limit=3,
                                    evaluate_during_training=True,
                                    eval_steps=5)

with training_args.strategy.scope():
    config = BertConfig(num_labels=1274, output_hidden_states=False, num_hidden_layers=3)
    model = TFBertForTokenClassification(config)


trainer = TFTrainer(model=model, args=training_args)
trainer.train_loss = tf.keras.metrics.Sum()
trainer.create_optimizer_and_scheduler(20)
trainer.distributed_training_steps(batch)

```

Comments:
@Paul-Trax ,

The warning you saw is not because you have some `-1` in the labels. It is because the computation was done inside a tensorflow graph, which was compiled before the computation. While compiling a graph, the different branches are entered, so you saw the warning. Once the real computation begins, i.e. your labels and logits used for computation, everything is fine.

An example to see such effect is (note that there is a `@tf.function` before `compute_loss`):

     import tensorflow as tf
     from typing import Dict, List, Optional, Union
     import warnings


     def shape_list(x: tf.Tensor) -> List[int]:
         """
         Deal with dynamic shape in tensorflow cleanly.

         Args:
             x (:obj:`tf.Tensor`): The tensor we want the shape of.

         Returns:
             :obj:`List[int]`: The shape of the tensor as a list.
         """
         static = x.shape.as_list()
         dynamic = tf.shape(x)
         return [dynamic[i] if s is None else s for i, s in enumerate(static)]


     @tf.function
     def compute_loss(labels, logits):

         loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(
             from_logits=True, reduction=tf.keras.losses.Reduction.NONE
         )
         # make sure only labels that are not equal to -100
         # are taken into account as loss
         if tf.math.reduce_any(labels == -1):
             warnings.warn("Using `-1` to mask the loss for the token is deprecated. Please use `-100` instead.")
             active_loss = tf.reshape(labels, (-1,)) != -1
             print(f'During graph compiling - branch 1: {labels}')
             tf.print(f'Executed in graph - branch 1: {labels}')
         else:
             active_loss = tf.reshape(labels, (-1,)) != -100
             print(f'During graph compiling - branch 2: {labels}')
             tf.print(f'Executed in graph - branch 2: {labels}')

         reduced_logits = tf.boolean_mask(tf.reshape(logits, (-1, shape_list(logits)[2])), active_loss)

         labels = tf.boolean_mask(tf.reshape(labels, (-1,)), active_loss)

         return loss_fn(labels, reduced_logits)


     batch_size = 3
     seq_len = 5
     dim = 4
     labels = tf.constant(0, shape=[batch_size, seq_len])
     logits = tf.random.uniform(shape=[batch_size, seq_len, dim])

     loss = compute_loss(labels, logits)

     print(f'loss = {loss}')

You will see something like

    /home/imo/Desktop/venv/lib/python3.6/site-packages/tensorflow/python/autograph/impl/api.py:493: UserWarning: Using `-1` to mask the loss for the token is deprecated. Please use `-100` instead.
    return py_builtins.overload_of(f)(*args)
    During graph compiling - branch 1: Tensor("labels:0", shape=(3, 5), dtype=int32)
    During graph compiling - branch 2: Tensor("labels:0", shape=(3, 5), dtype=int32)
    Executed in graph - branch 2: Tensor("labels:0", shape=(3, 5), dtype=int32)
    loss = [1.5343634 1.610856  1.433133  1.4082022 1.5018827 1.0152265 1.563687
        1.2404382 1.1259079 1.7140993 1.4652599 1.6314502 1.5104814 1.389543
        1.45472  ]

If you remove the `tf.function` above `compute_loss`, there is no graph compiled, and you won't see the warning you had.

    During graph compiling - branch 2: [[0 0 0 0 0]  # There is no graph compiled, it is just our print statement.
      [0 0 0 0 0]
      [0 0 0 0 0]]
    Executed in graph - branch 2: [[0 0 0 0 0]
      [0 0 0 0 0]
      [0 0 0 0 0]]
    loss = [1.648417  1.2457228 1.5540932 1.7658947 1.4607204 1.529434  1.3607037
      1.6142995 0.9669408 1.316714  1.3906621 1.689343  1.3678703 1.324768
      1.5207067]

When `TFTrainer` is used, the computation is done in graph model.

Hi chiapas,
Thanks a lot for the detailed answer, it was really helpful!
I am closing the issue.