glue.py when using mrpc and similar data does not work
Author: nargesam
Pull request: no
Labels: name: wontfix

## 🐛 Bug

<!-- Important information -->

Model I am using (Bert, XLNet....): BERT

Language I am using the model on (English, Chinese....): ENGLISH

The problem arise when using:
* [ ] the official example scripts: (give details):
I am using my dataset with format [idx, sentence1, sentence2, label] in form of dict of tf.data.Dataset and using glue_convert_examples_to_features() to convert my dataset to features

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
I am using MRPC task 

## To Reproduce

Steps to reproduce the behavior:
0- Using BertTokenizer.from_pretrained('bert-base-cased') to create tokenizer
1-read csv file as dataframe
2-convert dataframe to dict of tensors using tf.data.Dataset.from_tensor_slices(dict(train))
the element_spec of my data is : 

{'idx': TensorSpec(shape=(), dtype=tf.string, name=None),
 'sentence1': TensorSpec(shape=(), dtype=tf.string, name=None), 
'sentence2': TensorSpec(shape=(), dtype=tf.string, name=None), 
'label': TensorSpec(shape=(), dtype=tf.int32, name=None)}

3- when using glue_convert_examples_to_features(train_data, tokenizer, label_list=[1,0], max_length=128 , task='mrpc'), I will have an error in glue.py in glue_convert_examples_to_features : label = label_map[example.label]

## Expected behavior

<!-- A clear and concise description of what you expected to happen. -->

## Environment

* OS:
* Python version: 3.7
* PyTorch version: TF 2.1.0
* PyTorch Transformers version (or branch):
* Using GPU ?
* Distributed or parallel setup ?
* Any other relevant information:

## Additional context

<!-- Add any other context about the problem here. -->

I think you can't access the dictionary by example.label. When I copy paste it to my code, it will actually work with example["label"]. 



Comments:
Hello! Indeed, you can't access the dictionary by using `example.label`. Are you passing your `tf.data.Dataset` as the `examples` argument to the `glue_convert_examples_to_features`?

yes, train_data is the examples. and the spec is: 

{'idx': TensorSpec(shape=(), dtype=tf.string, name=None),
'sentence1': TensorSpec(shape=(), dtype=tf.string, name=None),
'sentence2': TensorSpec(shape=(), dtype=tf.string, name=None),
'label': TensorSpec(shape=(), dtype=tf.int32, name=None)}

shouldn't it look like this? 


I think the code needs to change to example["label"]. 

Here's the format of my data that is passed to glue_convert_examples_to_features():

{'idx': <tf.Tensor: shape=(), dtype=string, numpy=b'TEXT'>, 'sentence1': <tf.Tensor: shape=(), dtype=string, numpy=b"TEXT TEXT TEXT">, 'sentence2': <tf.Tensor: shape=(), dtype=string, numpy=b'text'>, 'label': <tf.Tensor: shape=(), dtype=int32, numpy=1>}


Do you mind letting me know on which version of transformers you're running your code?

For a couple of versions now we handle `tf.data.Dataset` using our `get_example_from_tensor_dict` method, as can be seen [here](https://github.com/huggingface/transformers/blob/master/src/transformers/data/processors/glue.py#L85). This transforms your dictionary into an `InputExample`. 

It should have been able to convert your dataset as well.

yes, I use the get_example_from_tensor_dict(). the problem is, even though I pass my label_list=[numpy.int64(1), numpy.int64(0)], and my data type is int32:  'label': <tf.Tensor: shape=(), dtype=int32, numpy=1>, 

when glue.py tries to run label = label_map[example.label], the type of example.label is <class: str>! which should not be, and should be int! 

What I did, was to clone your repo on my local device, changed that line to label = label_map[int(example.label)], and it works fine now! 

I am guessing when you create the label_map dictionary, the keys are str, but needs to be int. 


This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.