TFTrainer Example
Author: alexorona
Pull request: no
Labels: name: wontfix

Is there an example that uses TFTrainer to fine-tune a model with more than one input type? Encountering some difficulty in figuring out how TFTrainer wants the tensorflow dataset structured. It doesn't seem to like one constructed from conventional numpy slices, e.g. `train_dataset = tf.data.Dataset.from_tensor_slices((input_ids, attention_mask, token_type_ids))`. I've dug through the documentation and a two dozen notesbooks and can't find an example of what an appropriate dataset input looks like.



Comments:
I am facing issue with :
/usr/local/lib/python3.6/dist-packages/transformers/trainer_tf.py in __init__(self, model, args, train_dataset, eval_dataset, compute_metrics, prediction_loss_only, tb_writer, optimizers)
     87             self.tb_writer = tb_writer
     88         else:
---> 89             self.tb_writer = tf.summary.create_file_writer(self.args.logging_dir)
     90 
     91         if is_wandb_available():

AttributeError: 'dict' object has no attribute 'logging_dir'

One good working example of TFTrainer would be very helpful. Thanks

We're working on the examples and there should be one for every task soon (in PyTorch and TensorFlow). For your specific problem, I think it's missing a dictionary. There is a brand new tutorial from @joeddav on how to fine-tune a model on your custom dataset that should be helpful to you [here](https://huggingface.co/transformers/master/custom_datasets.html). Click on the TensorFlow button on the code examples to switch the code from PyTorch to TensorFlow, or on the open in colab button at the top where you can select the TensorFlow notebook that goes with the tutorial.

Yes, you want to pass a tuple to `from_tensor_slices` where the first element is a dict of `kwarg:input` and the second is the labels. So in your case:

```
train_dataset = tf.data.Dataset.from_tensor_slices((
    {"input_ids": input_ids, "attention_mask": attention_mask, "token_type_ids": token_type_ids},
    labels # just a tensor, or a dict if multiple outputs as with question answering
))
```

The minibatches in the format of the inputs dict will by passed as kwargs to the model at each train step. The [tutorial](https://huggingface.co/transformers/master/custom_datasets.html) @sgugger recommended has some more examples.

Thanks Joe and Sylvain!
- Transformers 3.0.2
- Windows
- Tensorflow 2.3
- Model: TFGPT2LMHeadModel 

@joeddav Hmmm... there might be an issue with parsing inputs for `TFGPT2LMHeadModel` or their might be problems with `_training_steps` (I remember reading that it was being deprecated or rewritten somewhere). I tried implementing the solution you indicated above, an extrapolation from the example that Sylvain linked to, and other variations, all with the same effect `ValueError: too many values to unpack (expected 2)` which triggers on this line in TFTrainer `for step, training_loss in enumerate(self._training_steps(train_ds, optimizer))`.

Tried:
- `input_ids`, `attention_mask`, `token_type_ids` each as a list of lists
- `input_ids`, `attention_mask`, `token_type_ids` each as a tf tensor
- `input_ids`, `attention_mask`, `token_type_ids` each as list of numpy arrays
- `input_ids`, `attention_mask`, `token_type_ids` each as a single numpy array
- Where `train_encodings` is a dictionary of the above input types (e.g. `train_encodings['input_ids'] = input_ids`, neither of these worked when passed to `TFTrainer`:
-- `train_dataset = tf.data.Dataset.from_tensor_slices(train_encodings)`
-- `train_dataset = tf.data.Dataset.from_tensor_slices((train_encodings))`

Since labels is not a recognized argument for `TFGPT2LMHeadModel`, presumably labels would be be just another key in `train_encodings` (e.g. `train_encodings['labels'] = labels)`.

@sgugger I encountered an encoding error when I was testing the inputs from IMDb reviews example. Here's a potential replacement that worked for me:
```
def read_imdb_split(split_dir):
    split_dir = Path(split_dir)
    texts = []
    labels = []
    for label_dir in ["pos", "neg"]:
        for text_file in (split_dir/label_dir).iterdir():
            f = open(text_file, encoding = 'utf-8')
            texts.append(f.read())
            f.close()
            labels.append(0 if label_dir is "neg" else 1)

    return texts, labels
```



@alexorona ahh, I believe this is an issue with TensorFlow LM-head models that we recently resolved – previously these models didn't take `labels` and didn't calculate the loss, so they didn't work with Trainer. Try building transformers from source and see if you still have the issue. (You can install from source by cloning the repo or just doing `pip install --upgrade git+https://github.com/huggingface/transformers.git`). Then you'll want to prepare your dataset so that the `labels` are the encoded `input_ids`:
```
tf.data.Dataset.from_tensor_slices((dict(train_encodings), train_encodings.input_ids))
```
If `train_encodings` are of type `BatchEncoding`, I believe you'll have to explicitly cast them as a dict as I do above.

We should document this in `TFTrainer`.

@joeddav Thanks! So I kind of got this to work, but could use some clarification on your last comment. After building from source, this will run until eval if inputs are already tf tensors:

```
input_ids = tf.convert_to_tensor(input_ids)
attention_mask = tf.convert_to_tensor(attention_mask)
token_type_ids = tf.convert_to_tensor(token_types)
labels = tf.convert_to_tensor(labels)

train_encodings = {"input_ids": input_ids, 
                "attention_mask": attention_mask,
                "token_type_ids": token_type_ids}

train_dataset = tf.data.Dataset.from_tensor_slices((train_encodings,  labels))
```
I'm getting a warning that says `Converting sparse IndexedSlices to a dense Tensor of unknown shape` and an error that it can't find _prediction_loop -- `'TFTrainer' object has no attribute '_prediction_loop'` -- the latter of which is probably just a result of the changes to TFTrainer.

I'm not sure how to interpret `train_encodings.input_ids`. Are you saying that we should make `train_encodings `an object with the labels set to input_ids? Labels are usually in the range `[-100, 0, ..., config.vocab_size] `with `-100` indicating its not part of the target. There's a lot of situations and setups where you want a token in the `input_ids`, but you don't want to calculate loss on it (for example when distinguishing between the target input and the history).

> an error that it can't find _prediction_loop -- 'TFTrainer' object has no attribute '_prediction_loop' -- the latter of which is probably just a result of the changes to TFTrainer.

Yep, that's just a bug. `TFTrainer._prediction_step` is deprecated and it looks like we missed a reference to it. I think line 415 of `trainer_tf.py` just needs to be changed to call `self.prediction_step`.

> Are you saying that we should make train_encodings an object with the labels set to input_ids?

No, sorry. You're right there are lots of situations where you would need something more complex, I was just using that as the most basic example of passing in labels for LM training. You just want the labels to be of the same shape as `input_ids` with the range exactly as you described.

> I'm getting a warning that says Converting sparse IndexedSlices to a dense Tensor of unknown shape

What format are your `labels` in? I'm not sure why they'd be sparse. TFTrainer will calculate the loss by calling `model(batch_encodings, labels=batch_labels)` which returns the loss as the first element. Try just passing one instance to the model and see if you get any errors and check that the returned loss looks reasonable (i.e. not NaN or something). In your case, that'd look like,
```python
outputs = model({key: val[:1] for key, val in train_encodings.items()}, labels=labels[:1])
loss = outputs[0]
```

I got this working, thanks for the help.
```
def example_to_features(input_ids,attention_masks,token_type_ids,y):
  return {"input_ids": input_ids,
          "attention_mask": attention_masks,
          "token_type_ids": token_type_ids},y
train_ds = tf.data.Dataset.from_tensor_slices((input_ids_train,attention_masks_train,token_ids_train,label_ids_train)).map(example_to_features)
and use train_ds for training
trainer = TFTrainer(
      model=model,                         
      args=training_args,                  
      train_dataset=train_ds,         
      eval_dataset=test_ds             
  )
```
One question, when I do trainer.train(), it's not displaying progress, but I see in logs it's training. Is there some verbose option I am missing?

Yeah the TFTrainer is not using any progress bar. Will add them soonish (with an option to disable for people who prefer not to see them), like in the PyTorch Trainer.

Thank you, Also if chose to train native Keras way:
`optimizer = tf.keras.optimizers.Adam(learning_rate=5e-5)
model.compile(optimizer=optimizer, loss=model.compute_loss) # can also use any keras loss fn
model.fit(train_dataset.shuffle(1000).batch(16), epochs=3, batch_size=16)`
why is model.train() missing? I thought without it it still be eval mode right?

@astromad You can edit the `TFTrainer` file directly (or copy it from GitHub and add create your own variation, which is what I did). You can add a basic progress bar at about line 500:

```
train_iterator = tqdm(range(epochs_trained, int(epochs + 1)), desc="Epoch")
            for epoch_iter in train_iterator:
                # Reset the past mems state at the beginning of each epoch if necessary.
                if self.args.past_index >= 0:
                    self._past = None

                epoch_iterator = tqdm(train_ds, desc="Iteration")
                for step, batch in enumerate(epoch_iterator):
```

Additionally, there's a way to display training loss, but my progress is not that far. I built a custom variation of `Trainer` that does that, but haven't yet incorporated all the changes into `TFTrainer` because the structure is different. Here's my progress so far in introducing continuous display (note: it won't be accurate because there's a number I need to divide by):

                    training_loss = self.train_loss.result() / ((step + 1) * self.total_train_batch_size)
                    loss_update = 'Train Epoch ' + str(epochs_trained) + " (loss " + str(round(training_loss.numpy(), 3)) + ")" 
                    epoch_iterator.set_description(loss_update)

@joeddav Thanks again, Joe! Astromad's map function creates a `batch` inside of `TFTrainer` that is fed to `self.distributed_training_steps`. This is the same batch structure that results when you instead use `train_dataset = tf.data.Dataset.from_tensor_slices((train_encodings,  labels))`, as outlined above. In both cases, what is fed to `self.distributed_training_steps` is a tuple containing: 1) a dictionary object with `input_ids`, `attention_mask` and `token_type_ids` as keys and tf tensors as values, and 2) tf tensor for `labels`. 

When testing model inputs outside of the context of `TFTrainer` like this:
```
outputs = model({key: val[:1] for key, val in train_encodings.items()}, labels=labels[:1])
loss = outputs[0]
```
It seems that the `labels` are not being registered correctly. Here are the outputs:
- `output[0]` is `shape=(381,)` instead of `shape=(1,)`. `381` is the number of values in the `labels` tensor that do not equal `-100`, so it looks like this is a per token loss. 
- `output[1]` is `shape=(1, 511, 50257)` -- shouldn't this be `shape(1, 512, 50257)`?
- `output[2]` is `shape=(2, 1, 16, 512, 64)`

Strangely, inside of `TFTrainer` when I print out `training_loss = self.train_loss.result() / ((step + 1) * self.total_train_batch_size)`, it's correctly a `shape=(1,)` tensor.


just wanna share if this is useful, to construct a prediction from arbitrary sentence this is what I am using:
```
sequence = "my name is firstname lastname and my phone number is 111-222-3333"
tokens = tokenizer.tokenize(tokenizer.decode(tokenizer.encode(sequence)))
print('Tokens are',tokens)
inputs = tokenizer.encode_plus(sequence, return_tensors="tf")
print('Inputs are',inputs)
outputs = model(inputs)
print('output shape',np.shape(outputs))
**And the output**
Tokens are ['[CLS]', 'my', 'name', 'is', 'first', '##name', 'last', '##name', 'and', 'my', 'phone', 'number', 'is', '111', '##-', '##22', '##-', '##33', '##33', '[SEP]']
Inputs are {'input_ids': <tf.Tensor: shape=(1, 20), dtype=int32, numpy=
array([[  101,  2026,  2171,  2003,  2034, 18442,  2197, 18442,  1998,
         2026,  3042,  2193,  2003, 11118, 29624, 19317, 29624, 22394,
        22394,   102]], dtype=int32)>, 'token_type_ids': <tf.Tensor: shape=(1, 20), dtype=int32, numpy=
array([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]],
      dtype=int32)>, 'attention_mask': <tf.Tensor: shape=(1, 20), dtype=int32, numpy=
array([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]],
      dtype=int32)>}
output shape (1, 1, 20, 6)
``` 

@joeddav @astromad Very useful examples! It's training correctly using the methods outlined above. Just some kinks to work out. It also looks like the `model.generate` method does not currently support the use of `token_type_ids`. `temperature`, `top_k` and `top_p` do not seem to have any effect on outputs. The example provided in the documentation will not work. 

Here's an example of one that will work. It's a `gpt2-medium` model fine-tuned on Jane Austen's _Pride and Prejudice_:
```
tokenizer = tokenizer
input_sequence = "Mr. Darcy was acting strangely, she thought to herself. What had happened?"
inputs = tokenizer.encode_plus(input_sequence , return_tensors="tf")
outputs = model.generate(inputs['input_ids'],
                         attention_mask = inputs['attention_mask'],
                         max_length = 200, 
                         repetition_penalty = 2.0, 
                         pad_token_id = pad_token_id,
                         eos_token_id = eos_token_id)

print(tokenizer.decode(outputs.numpy().tolist()[0]))

'Mr. Darcy was acting strangely, she thought to herself. What had happened? She looked at the clock 
and saw that it must be half-past ten; but as soon afterwards came a knock on her door from Miss 
Lucas\'s father, who said he would come in an hour or two with his daughter. "Come here," cried 
Elizabeth, when they entered into Mrs.—Lennox Hall ;—"come now." Mr—Darcy followed them 
through their dining room towards another large hall where there were many other ladies of distinction 
seated round tables which formed one side by themselves before each table. The gentlemen sat down 
opposite him all together, while Jane stood nearby looking very pale for some reason unknown till then 
only known among those present. When Lydia heard this news, however,,she immediately began to speak 
aloud:\n 
'
```


This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.



I am currently attempting to train a TF model (using Keras) using a dataset pipeline constructed with tf.data where I use a tf.py_function that tokenizes batches of data _on the fly_.

```python
train_ds = tf.data.Dataset.from_tensor_slices((X_train, y_train)).batch(2)
def py_func(x):
  x = x.numpy() 
  x = [i.decode("utf-8") for i in x] 
  d = tokenizer(x, truncation=True, padding=True)
  return list(d.values())

def ds_map_fn(x,y): 
  flattened_output = tf.py_function(py_func, [x], [tf.int32, tf.int32])
  return {"input_ids": flattened_output[0], "attention_mask": flattened_output[1]},y

train_ds = train_ds.map(ds_map_fn)
for x,y in train_ds.take(2):
  print(x)
```

When I sample from this pipeline, the result looks sensible 

```
{'input_ids': <tf.Tensor: shape=(2, 20), dtype=int32, numpy=
array([[  101, 19443, 23698,  7710,  1195,  1274,   189,  1138,  1609,
          184,   102,     0,     0,     0,     0,     0,     0,     0,
            0,     0],
       [  101,  5837,  8752,  4163,  4654, 26063,  5077,  1611, 11644,
        22234,  1394,  2822,  5077, 22449,  1606,  1482,  1106,  4609,
         4039,   102]], dtype=int32)>, 'attention_mask': <tf.Tensor: shape=(2, 20), dtype=int32, numpy=
array([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]],
      dtype=int32)>}
{'input_ids': <tf.Tensor: shape=(2, 43), dtype=int32, numpy=
array([[  101,   170,  2944,  2312,  7925, 16234, 11275,  1109,  7085,
        13830,   142, 12323,  1658,  1144,  1508,  1164,  1479,  4252,
        15304,  1116,  1107,  3315, 13826,  1120,  1103, 12548,  1104,
          139, 23698,  7710,  6716, 18630,   189,  1884,  5539,  2240,
         2924,  1775,  4426,  2064,  1559, 20923,   102],
       [  101,   146,   182,  1136,  2157,  1128,  5380,   189,  6730,
          190,  1197,  4438,  1106,  2992,  1252,   178,  1329,  1917,
          178,  1138,  1106, 13803,  1128,  1335,  4847,  1358,  1110,
         1103,  1211, 18630,   189,  1884,   124,  5822,  2036,  1186,
         1658,  1545, 12649,  1403,   102,     0,     0]], dtype=int32)>, 'attention_mask': <tf.Tensor: shape=(2, 43), dtype=int32, numpy=
array([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
        0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
        0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]],
      dtype=int32)>}
```

However, when I try to train a model using this dataset pipeline, I get an error.  

Training code
```python
model = TFBertForSequenceClassification.from_pretrained("bert-base-cased", num_labels=2 )

optimizer = tf.keras.optimizers.Adam(learning_rate=5e-5)
model.compile(optimizer=optimizer, 
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
              metrics=tf.metrics.SparseCategoricalAccuracy()
              ) # can also use any keras loss fn
model.fit(train_ds,  epochs=3 )

```

error

```
OperatorNotAllowedInGraphError: iterating over `tf.Tensor` is not allowed: AutoGraph did convert this function. This might indicate you are trying to use an unsupported feature.
```

Any pointers as to what I am doing wrong would be appreciated. 

In the mean time, I have used the workaround where I first tokenize my data before I create a tf.data pipeline.  

```python
train_encodings = tokenizer(X_train, truncation=True, padding=True)
train_ds = tf.data.Dataset.from_tensor_slices((
    dict(train_encodings),
    y_train
))
```