Error in running run_tf_text_classification.py
Author: rajesh-dhiman
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.4.2
- Platform: Windows-10-10.0.18362-SP0
- Python version: 3.8.0
- PyTorch version (GPU?): 1.7.1+cpu (False)
- Tensorflow version (GPU?): 2.4.1 (False)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

Model hub:

- for issues with a model report at https://discuss.huggingface.co/ and tag the model's creator.

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

## Information

Model I am using (Bert, XLNet ...):  microsoft/deberta-base

The problem arises when using: run_tf_text_classification.py

## To reproduce

Steps to reproduce the behavior:
just run
python run_tf_text_classification.py --model_name_or_path microsoft/deberta-base --output_dir classificationoutput --train_file PreparedData.csv --label_column_id 1 --do_train



[PreparedData.zip](https://github.com/huggingface/transformers/files/6288064/PreparedData.zip)


## Stack trace
[INFO|training_args.py:631] 2021-04-09 13:21:17,622 >> PyTorch: setting up devices
[INFO|training_args.py:554] 2021-04-09 13:21:17,629 >> The default value for the training argument `--report_to` will change in v5 (from all installed integrations to none). In v5, you will need to use `--report_to all` to get the same behavior as now. You should start updating your code and make this info disappear :-).
[INFO|training_args_tf.py:192] 2021-04-09 13:21:17,635 >> Tensorflow: setting up strategy
04/09/2021 13:21:18 - INFO - __main__ -   n_replicas: 1, distributed training: False, 16-bits training: False
04/09/2021 13:21:18 - INFO - __main__ -   Training/evaluation parameters TFTrainingArguments(output_dir='classificationoutput', overwrite_output_dir=False, do_train=True, do_eval=None, do_predict=False, evaluation_strategy=<IntervalStrategy.NO: 'no'>, prediction_loss_only=False, per_device_train_batch_size=8, per_device_eval_batch_size=8, per_gpu_train_batch_size=None, per_gpu_eval_batch_size=None, gradient_accumulation_steps=1, eval_accumulation_steps=None, learning_rate=5e-05, weight_decay=0.0, adam_beta1=0.9, adam_beta2=0.999, adam_epsilon=1e-08, max_grad_norm=1.0, num_train_epochs=3.0, max_steps=-1, lr_scheduler_type=<SchedulerType.LINEAR: 'linear'>, warmup_ratio=0.0, warmup_steps=0, logging_dir='runs\\Apr09_13-21-17_GC8SQLQ2E', logging_strategy=<IntervalStrategy.STEPS: 'steps'>, logging_first_step=False, logging_steps=500, save_strategy=<IntervalStrategy.STEPS: 'steps'>, save_steps=500, save_total_limit=None, no_cuda=False, seed=42, fp16=False, fp16_opt_level='O1', fp16_backend='auto', fp16_full_eval=False, local_rank=-1, tpu_num_cores=None, tpu_metrics_debug=False, debug=False, dataloader_drop_last=False, eval_steps=500, dataloader_num_workers=0, past_index=-1, run_name='classificationoutput', disable_tqdm=False, remove_unused_columns=True, label_names=None, load_best_model_at_end=False, metric_for_best_model=None, greater_is_better=None, ignore_data_skip=False, sharded_ddp=[], deepspeed=None, label_smoothing_factor=0.0, adafactor=False, group_by_length=False, report_to=['tensorboard'], ddp_find_unused_parameters=None, dataloader_pin_memory=True, skip_memory_metrics=False, tpu_name=None, tpu_zone=None, gcp_project=None, poly_power=1.0, xla=False)
[INFO|configuration_utils.py:463] 2021-04-09 13:21:19,339 >> loading configuration file https://huggingface.co/microsoft/deberta-base/resolve/main/config.json from cache at C:\Users\ XXXXXXXX/.cache\huggingface\transformers\e313266bff73867debdfa78c78a9a4966d5e78281ac4ed7048c178b16a37eba7.fb501413b9cef9cef6babdc543bb4153cbec58d52bce077647efba3e3f14ccf3
[INFO|configuration_utils.py:499] 2021-04-09 13:21:19,340 >> Model config DebertaConfig {
  "attention_probs_dropout_prob": 0.1,
  "hidden_act": "gelu",
  "hidden_dropout_prob": 0.1,
  "hidden_size": 768,
  "initializer_range": 0.02,
  "intermediate_size": 3072,
  "layer_norm_eps": 1e-07,
  "max_position_embeddings": 512,
  "max_relative_positions": -1,
  "model_type": "deberta",
  "num_attention_heads": 12,
  "num_hidden_layers": 12,
  "pad_token_id": 0,
  "pooler_dropout": 0,
  "pooler_hidden_act": "gelu",
  "pooler_hidden_size": 768,
  "pos_att_type": [
    "c2p",
    "p2c"
  ],
  "position_biased_input": false,
  "relative_attention": true,
  "transformers_version": "4.4.2",
  "type_vocab_size": 0,
  "vocab_size": 50265
}

[INFO|tokenization_utils_base.py:1702] 2021-04-09 13:21:20,647 >> loading file https://huggingface.co/microsoft/deberta-base/resolve/main/bpe_encoder.bin from cache at C:\Users\ XXXXXXXX/.cache\huggingface\transformers\b5857926db0a74705bc948686137f046f6ecbc4342162fa03c873a7407eb90ef.d9f36b1bee7c5e05c6b209f4839d4f94d59c2e71c73b1ad67935d66c41c24ff7
[INFO|tokenization_utils_base.py:1702] 2021-04-09 13:21:20,648 >> loading file https://huggingface.co/microsoft/deberta-base/resolve/main/added_tokens.json from cache at None
[INFO|tokenization_utils_base.py:1702] 2021-04-09 13:21:20,648 >> loading file https://huggingface.co/microsoft/deberta-base/resolve/main/special_tokens_map.json from cache at None
[INFO|tokenization_utils_base.py:1702] 2021-04-09 13:21:20,649 >> loading file https://huggingface.co/microsoft/deberta-base/resolve/main/tokenizer_config.json from cache at C:\Users\ XXXXXXXX/.cache\huggingface\transformers\c2bc27a1c7529c177696ff76b1e74cba8667be14e202359f20f9114e407f43e2.a39abb1c6179fb264c2db685f9a056b7cb8d4bc48d729888d292a2280debf8e2
[INFO|tokenization_utils_base.py:1702] 2021-04-09 13:21:20,650 >> loading file https://huggingface.co/microsoft/deberta-base/resolve/main/tokenizer.json from cache at None
04/09/2021 13:21:21 - WARNING - datasets.builder -   Using custom data configuration default-337be17b0e590a88
Downloading and preparing dataset csv/default (download: Unknown size, generated: Unknown size, post-processed: Unknown size, total: Unknown size) to C:\Users\ XXXXXXXX\.cache\huggingface\datasets\csv\default-337be17b0e590a88\0.0.0\2dc6629a9ff6b5697d82c25b73731dd440507a69cbce8b425db50b751e8fcfd0...
Dataset csv downloaded and prepared to C:\Users\ XXXXXXXX\.cache\huggingface\datasets\csv\default-337be17b0e590a88\0.0.0\2dc6629a9ff6b5697d82c25b73731dd440507a69cbce8b425db50b751e8fcfd0. Subsequent calls will reuse this data.
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
~\run_tf_text_classification.py in <module>
    350 
    351 if __name__ == "__main__":
--> 352     main()

~\run_tf_text_classification.py in main()
    284     )
    285 
--> 286     train_dataset, eval_dataset, test_ds, label2id = get_tfds(
    287         train_file=data_args.train_file,
    288         eval_file=data_args.dev_file,

~\run_tf_text_classification.py in get_tfds(train_file, eval_file, test_file, tokenizer, label_column_id, max_seq_length)
    121             print(ds[k])
    122             '''
--> 123             transformed_ds[k] = ds[k].map(
    124                 lambda example: tokenizer.batch_encode_plus(
    125                     (example[features_name[0]], example[features_name[1]]),

c:\python38\lib\site-packages\datasets\arrow_dataset.py in map(self, function, with_indices, input_columns, batched, batch_size, drop_last_batch, remove_columns, keep_in_memory, load_from_cache_file, cache_file_name, writer_batch_size, features, disable_nullable, fn_kwargs, num_proc, suffix_template, new_fingerprint)
   1405         test_inputs = self[:2] if batched else self[0]
   1406         test_indices = [0, 1] if batched else 0
-> 1407         update_data = does_function_return_dict(test_inputs, test_indices)
   1408         logger.info("Testing finished, running the mapping function on the dataset")
   1409 

c:\python38\lib\site-packages\datasets\arrow_dataset.py in does_function_return_dict(inputs, indices)
   1376             fn_args = [inputs] if input_columns is None else [inputs[col] for col in input_columns]
   1377             processed_inputs = (
-> 1378                 function(*fn_args, indices, **fn_kwargs) if with_indices else function(*fn_args, **fn_kwargs)
   1379             )
   1380             does_return_dict = isinstance(processed_inputs, Mapping)

~\run_tf_text_classification.py in <lambda>(example)
    122             '''
    123             transformed_ds[k] = ds[k].map(
--> 124                 lambda example: tokenizer.batch_encode_plus(
    125                     (example[features_name[0]], example[features_name[1]]),
    126                     truncation=True,

c:\python38\lib\site-packages\transformers\tokenization_utils_base.py in batch_encode_plus(self, batch_text_or_text_pairs, add_special_tokens, padding, truncation, max_length, stride, is_split_into_words, pad_to_multiple_of, return_tensors, return_token_type_ids, return_attention_mask, return_overflowing_tokens, return_special_tokens_mask, return_offsets_mapping, return_length, verbose, **kwargs)
   2432         )
   2433 
-> 2434         return self._batch_encode_plus(
   2435             batch_text_or_text_pairs=batch_text_or_text_pairs,
   2436             add_special_tokens=add_special_tokens,

c:\python38\lib\site-packages\transformers\tokenization_utils.py in _batch_encode_plus(self, batch_text_or_text_pairs, add_special_tokens, padding_strategy, truncation_strategy, max_length, stride, is_split_into_words, pad_to_multiple_of, return_tensors, return_token_type_ids, return_attention_mask, return_overflowing_tokens, return_special_tokens_mask, return_offsets_mapping, return_length, verbose, **kwargs)
    529                 ids, pair_ids = ids_or_pair_ids
    530 
--> 531             first_ids = get_input_ids(ids)
    532             second_ids = get_input_ids(pair_ids) if pair_ids is not None else None
    533             input_ids.append((first_ids, second_ids))

c:\python38\lib\site-packages\transformers\tokenization_utils.py in get_input_ids(text)
    509                 return text
    510             else:
--> 511                 raise ValueError(
    512                     "Input is not valid. Should be a string, a list/tuple of strings or a list/tuple of integers."
    513                 )

ValueError: Input is not valid. Should be a string, a list/tuple of strings or a list/tuple of integers.

​


Comments:
@Rocketknight1 can you help in this

@sgugger can you help in this

Hi, we have a new, simpler and more robust text classification in TensorFlow script contributed by @Rocketknight1 [here](https://github.com/huggingface/transformers/blob/master/examples/tensorflow/text-classification/run_text_classification.py), could you check it out?

Hi @LysandreJik   and @Rocketknight1 

I used it   run_text_classification.py in jupyter
%run run_text_classification.py \
--model_name_or_path roberta-base \
--output_dir classificationoutput \
--train_file PreparedData.csv \
--validation_file PreparedData.csv \
--do_train


PreparedData.csv looks like below
sentence,label
sent1,l1
sent2,l1
sent3,l2
sent3,l2

I got following error

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
~\run_text_classification.py in <module>
    532 
    533 if __name__ == "__main__":
--> 534     main()

~\run_text_classification.py in main()
    492 
    493         callbacks = [SavePretrainedCallback(output_dir=training_args.output_dir)]
--> 494         model.fit(
    495             training_dataset, validation_data=eval_dataset, epochs=training_args.num_train_epochs, callbacks=callbacks
    496         )

c:\python38\lib\site-packages\tensorflow\python\keras\engine\training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_batch_size, validation_freq, max_queue_size, workers, use_multiprocessing)
   1086           self._maybe_load_initial_epoch_from_ckpt(initial_epoch))
   1087       logs = None
-> 1088       for epoch, iterator in data_handler.enumerate_epochs():
   1089         self.reset_metrics()
   1090         callbacks.on_epoch_begin(epoch)

c:\python38\lib\site-packages\tensorflow\python\keras\engine\data_adapter.py in enumerate_epochs(self)
   1132     with self._truncate_execution_to_epoch():
   1133       data_iterator = iter(self._dataset)
-> 1134       for epoch in range(self._initial_epoch, self._epochs):
   1135         if self._insufficient_data:  # Set by `catch_stop_iteration`.
   1136           break

TypeError: 'float' object cannot be interpreted as an integer

​

Good catch, thank you! This was totally my fault, and has now been fixed in #11379 . If you pull the latest version of the library, training should work.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.