How to use model.fit in GPT2 TF Model
Author: s4sarath
Pull request: no
Labels: name: wontfix

## ❓ Questions & Help

```import tensorflow as tf
from transformers import *
import numpy as np

# Load dataset, tokenizer, model from pretrained model/vocabulary
tokenizer = GPT2Tokenizer.from_pretrained('gpt2')
model = TFGPT2LMHeadModel.from_pretrainedmodel = TFGPT2LMHeadModel.from_pretrained('gpt2')
np.random.seed(0)
batch = 5
max_len = 750
inp = tar = np.random.randint(0, 50267, (batch, max_len))
dataset = tf.data.Dataset.from_tensor_slices((inp, tar))
dataset = dataset.batch(batch)
optimizer = tf.keras.optimizers.Adam(learning_rate=3e-5, epsilon=1e-08, clipnorm=1.0)
loss_function = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
metric = tf.keras.metrics.SparseCategoricalAccuracy('accuracy')
model.compile(optimizer=optimizer, loss=loss_function, metrics=[metric])
history = model.fit(dataset)
```

I would like to use ```model.fit``` on the dataset. Can anyone suggest me. Now getting following error 

```ValueError: Error when checking model target: the list of Numpy arrays that you are passing to your model is not the size the model expected. Expected to see 13 array(s), but instead got the following list of 1 arrays: [<tf.Tensor 'IteratorGetNext:1' shape=(None, 750) dtype=int64>]...```


Comments:
Hello! Are you sure this is the script with which you get your error? The `model.fit` argument `epoch` doesn't exist (it should be `epochs`) and your model has not been compiled beforehand. Could you provide an example script which throws the error you're mentioning?

Hi , I tried to minimize the code as much as possible. I did add compile
and epoch was a typo. Will update new code.

On Fri, Oct 4, 2019, 12:10 AM Lysandre Debut <notifications@github.com>
wrote:

> Hello! Are you sure this is the script with which you get your error? The
> model.fit argument epoch doesn't exist (it should be epochs) and your
> model has not been compiled beforehand. Could you provide an example script
> which throws the error you're mentioning?
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/1412?email_source=notifications&email_token=ACRE6KA765X6QGUG2K3FMQ3QMY4AXA5CNFSM4I5AVBUKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAJFPBA#issuecomment-538072964>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/ACRE6KA3PO2ED7PK4QPKUOLQMY4AXANCNFSM4I5AVBUA>
> .
>


Hi @LysandreJik - I have updated the code. The issue remains the same.

Hi, thanks for updating your code. You should be careful with the model's output. The `TFGPT2LMHeadModel` outputs a list of 13 tensors: the first one is the one you're interested in, which is a tensor of logits across the vocabulary.

This tensor shape is `(batch_size, sequence_length, config.vocab_size)`, while you seem to be giving your models targets that have the same shape as your inputs.

The 12 following tensors are the "pre-computed hidden-states (key and values in the attention blocks)". You won't be using these for keras' fit method, so you should adapt your model compile method to only calculate the loss on the first output. [This Stack Overflow question](https://stackoverflow.com/questions/40446488/training-only-one-output-of-a-network-in-keras) talks about computing a loss for a single output in a multi-output model.

You can read the relevant documentation [here](https://huggingface.co/transformers/model_doc/gpt2.html#tfgpt2lmheadmodel).

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


@s4sarath were you able to figure this out by chance? I'm having the same issue.

I'm having the same problem and I'm not sure @LysandreJik is correct. The output of `TFGPT2LMHeadModel` is a pair where the first item is the logits tensor and the second item is the twelve layer caches. So either
    
    model.compile(..., loss = [SparseCategoricalCrossentropy(from_logits = True), None], ...)

or

    model.compile(..., loss = [SparseCategoricalCrossentropy(from_logits = True), *[None]*12], ...)

ought to be the correct invocation. But neither of them works.

You're correct, they're the past, not the attentions.