BERT Fine-tuning problems
Author: laetokang
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     
     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question 
     that you posted.
     -->

## Details
<!-- Description of your issue -->
Hello. I'm going to do a fine-tuning of BERT-base-uncased using the QA Dataset I made. However, the following error occurs: Could you tell me how to solve this problem?

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**:

```
Traceback (most recent call last):
  File "./examples/question-answering/run_squad.py", line 830, in <module>
    main()
  File "./examples/question-answering/run_squad.py", line 768, in main
    train_dataset = load_and_cache_examples(args, tokenizer, evaluate=False, output_examples=False)
  File "./examples/question-answering/run_squad.py", line 452, in load_and_cache_examples
    examples = processor.get_train_examples(args.data_dir, filename=args.train_file)
  File "/home/address/anaconda3/lib/python3.7/site-packages/transformers/data/processors/squad.py", line 525, in get_train_examples
    return self._create_examples(input_data, "train")
  File "/home/address/anaconda3/lib/python3.7/site-packages/transformers/data/processors/squad.py", line 552, in _create_examples
    title = entry["title"]
TypeError: string indices must be integers
Traceback (most recent call last):
  File "/home/address/anaconda3/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/address/anaconda3/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/address/anaconda3/lib/python3.7/site-packages/torch/distributed/launch.py", line 253, in <module>
    main()
  File "/home/address/anaconda3/lib/python3.7/site-packages/torch/distributed/launch.py", line 249, in main
    cmd=cmd)
subprocess.CalledProcessError: Command '['/home/address/anaconda3/bin/python', '-u', './examples/question-answering/run_squad.py', '--local_rank=1', '--model_type', 'bert', '--model_name_or_path', 'bert-base-uncased', '--do_train', '--do_eval', '--train_file', '/home/address/Desktop/address/train_split.json', '--predict_file', '/home/address/Desktop/address/val_split.json', '--learning_rate', '3e-5', '--num_train_epochs', '2', '--max_seq_length', '384', '--doc_stride', '128', '--output_dir', '../models/wwm_uncased_finetuned_squad/', '--per_gpu_eval_batch_size=3', '--per_gpu_train_batch_size=3']' returned non-zero exit status 1.
```

Comments:
Is your dataset following the SQuAD dataset format? It seems that what's making it crash is that there's no `title` entry.

You can take a look at how SQuAD is setup [here](https://rajpurkar.github.io/SQuAD-explorer/dataset/train-v1.1.json).

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.