Finetuning TFDistilBertForQuestionAnswering on SQuAD
Author: melwazir
Pull request: no
Labels: name: wontfix

## ❓ Questions & Help

<!-- A clear and concise description of the question. -->

Hi. I'm trying to finetune a TFDistilBertForQuestionAnswering model on the SQuAD 1.1 dataset, but I'm getting the following error at the "fit" statement:

ValueError: Failed to find data adapter that can handle input: (<class 'list'> containing values of types {"<class 'transformers.data.processors.squad.SquadFeatures'>"}), <class 'NoneType'>

Here is my code (running on Colab):

```
!pip install transformers
try:
  %tensorflow_version 2.x
except Exception:
  pass

import tensorflow as tf
from transformers import *
from transformers.data.processors.squad import SquadV1Processor

tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
model = TFDistilBertForQuestionAnswering.from_pretrained('distilbert-base-uncased')

import tensorflow_datasets as tfds
dataset = tfds.load("squad")

processor = SquadV1Processor()

training_examples = processor.get_examples_from_dataset(dataset, evaluate=False)
evaluation_examples = processor.get_examples_from_dataset(dataset, evaluate=True)

training_features = squad_convert_examples_to_features(
    examples=training_examples,
    tokenizer=tokenizer,
    max_seq_length=384,
    doc_stride=128,
    max_query_length=96,
    is_training=True,
)

test_features = squad_convert_examples_to_features(
    examples=evaluation_examples,
    tokenizer=tokenizer,
    max_seq_length=384,
    doc_stride=128,
    max_query_length=96,
    is_training=False,
)

optimizer = tf.keras.optimizers.Adam(learning_rate=3e-5, epsilon=1e-08, clipnorm=1.0)

model.compile(optimizer=optimizer, loss="mse", metrics=["mae"])
model.fit(training_features, validation_data = test_features ,epochs=3)
```

The documentation for `squad_convert_examples_to_features` says:

> Converts a list of examples into a list of features that can be directly given as input to a model. 

It doesn't specify whether that model is a TF or PT model (honestly there's a frustrating lack of TF examples for this repo in general).

Side question: Is my choice of loss function (mse) in this case correct?

Appreciate the help. Thanks!

Comments:
Hi, 

**Regarding the lack of tf examples**:
I am looking for a similar example ( a squad tf one) and found this [issue](https://github.com/huggingface/transformers/issues/2387), where @LysandreJik mentioned that he is currently working on exactly that. 

**Regarding your specific error**: 
squad_convert_examples_to_features allows you to specify if you want to receive the features as pytorch data (default, I believe) or tf dataset. Just use the argument **return_dataset="tf"**

Since I am looking for a similar example, I would be glad if you could share your code as soon as it works ;) 

Thanks for the reply @jwallat. I must have misunderstood that argument. I added it now but now I get an error. Well, two actually. I get this error on the first run:

AttributeError                            Traceback (most recent call last)

<ipython-input-23-615fcff126d8> in <module>()
----> 1 model.fit(training_features, validation_data = test_features ,epochs=3)

8 frames

/tensorflow-2.1.0/python3.6/tensorflow_core/python/keras/engine/training_utils.py in cast_if_floating_dtype_and_mismatch(targets, outputs)
   1339     if isinstance(target, np.ndarray):
   1340       target = ops.convert_to_tensor(target)
-> 1341     if target.dtype != out.dtype:
   1342       new_targets.append(cast_single_tensor(target, dtype=out.dtype))
   1343     else:

AttributeError: 'str' object has no attribute 'dtype'

If I just run the fit statement again, without any changes, I get a different error:

---------------------------------------------------------------------------

ValueError                                Traceback (most recent call last)

<ipython-input-4-865a9afcc901> in <module>()
----> 1 model.fit(training_features, validation_data = test_features ,epochs=3)

16 frames

/tensorflow-2.1.0/python3.6/tensorflow_core/python/autograph/impl/api.py in wrapper(*args, **kwargs)
    235       except Exception as e:  # pylint:disable=broad-except
    236         if hasattr(e, 'ag_error_metadata'):
--> 237           raise e.ag_error_metadata.to_exception(e)
    238         else:
    239           raise

ValueError: in converted code:

    /tensorflow-2.1.0/python3.6/tensorflow_core/python/keras/engine/training_v2.py:677 map_fn
        batch_size=None)
    /tensorflow-2.1.0/python3.6/tensorflow_core/python/keras/engine/training.py:2469 _standardize_tensors
        exception_prefix='target')
    /tensorflow-2.1.0/python3.6/tensorflow_core/python/keras/engine/training_utils.py:510 standardize_input_data
        'for each key in: ' + str(names))

    ValueError: No data provided for "output_1". Need data for each key in: ['output_1', 'output_2']


I've been at this for a whole day now and I'm stumped. I tried changing the loss function to sparse categorical crossentropy but it doesn't make any difference. I keep getting the same two errors. What am I doing wrong?

Edit: Excuse my beginner's incompetence. I've been digging through keras code for a while now and now I think the first error is the actual error because along the trace there's a `standardize` function which looks like it changes the original dataset in spite of throwing the error. So on the next run it bypasses the original error line and throws an error at a later stage.

Still not sure why the first error is happening though! Would appreciate some pointers!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.