TFBertForSequenceClassification - Feeding List of InputExamples
Author: amoelle
Pull request: no
Labels: name: wontfix

## ❓ Questions & Help

I used the "glue_convert_examples_to_features" function on my own InputExamples to get a List of InputFeatures. I want to do a Multi-Label Classification but I can not figure out how i need to feed the List of InputFeatures to the TFBertForSequenceClassification model.

    train_dataset = glue_convert_examples_to_features(train_examples, tokenizer, max_length=512, task='metis_ton')
    valid_dataset = glue_convert_examples_to_features(validation_examples, tokenizer, max_length=512, task='metis_ton')

    optimizer = tf.keras.optimizers.Adam(learning_rate=3e-5, epsilon=1e-08, clipnorm=1.0)
    loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
    metric = tf.keras.metrics.SparseCategoricalAccuracy('accuracy')
    model.compile(optimizer=optimizer, loss=loss, metrics=[metric])

    history = model.fit(train_dataset, epochs=2, batch_size=16,
                    validation_data=valid_dataset, validation_steps=7)

In this case "metis_ton" is my own Procsesor with labels corresponding to my data.
When i try to feed the list directly to model.fit() i get the following error:

    WARNING:tensorflow:Falling back from v2 loop because of error: Failed to find data adapter 
    that can handle input: (<class 'list'> containing values of types {"<class ' 
    transformers.data.processors.utils.InputFeatures'>"}), <class 'NoneType'>
    Please provide as model inputs either a single array or a list of arrays. You passed: inputs=[{
    "attention_mask": [
    1,
    1,
    ...

I then tried to split the data in X and y:

    input_ids = []
    attention_mask = []
    token_type_ids = []
    train_y = []
    for feature in train_dataset:
            input_ids.append(feature.input_ids)
            attention_mask.append(feature.attention_mask)
            token_type_ids.append(feature.token_type_ids)
            train_y.append(feature.label)
    train_X = [input_ids, attention_mask, token_type_ids]

    history = model.fit(train_X, train_Y,  epochs=2, batch_size=16,
                    validation_data=valid_dataset, validation_steps=7)

In this case i get the error

    Data cardinality is ambiguous:
      x sizes: 3
      y sizes: 362
    Please provide data which shares the same first dimension.

Then i tried to reshape the train_X data:

     train_X = list(map(list, zip(*train_X)))
     train_X = np.asarray(train_X)
     train_y = np.asarray(train_y)
     train_X.shape
     : (362, 3, 512)

Which results in the following error when calling model.fit():

    ValueError: Cannot reshape a tensor with 768 elements to shape [1,1,512,1] (512 elements) for 
    'tf_bert_for_sequence_classification/bert/embeddings/LayerNorm/Reshape' (op: 'Reshape') with 
    input shapes: [768], [4] and with input tensors computed as partial shapes: input[1] = [1,1,512,1].

Right now im out of ideas what i could try, can someone help me out?

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Same problem here. My workaround:

```
def my_workaround(data):
    '''Takes list of InputFeatures, returns arrays.'''
    # List of dicts
    data = [ feature.to_dict() for feature in data ]
    # Make one list for each entry in the dicts
    input_ids, attention_mask, token_type_ids, label = [], [], [], []
    for data_dict in data:
        input_ids.append(data_dict['input_ids'])
        attention_mask.append(data_dict['attention_mask'])
        token_type_ids.append(data_dict['token_type_ids'])
        label.append(data_dict['label'])
    # Stack in one array each
    input_ids = np.vstack(input_ids)
    attention_mask = np.vstack(attention_mask)
    token_type_ids = np.vstack(token_type_ids)
    label = np.vstack(label)
    # Return
    return label, input_ids, attention_mask, token_type_ids

y_train, *X_train = my_workaround(data_train)
```
It is not ideal, but I hope it helps :) 

same problem,i havent got a solution