noamaanMulla-03/transformers-issues huggingface dataset:
import error in version 3.3.0, conflict with local directory "datasets"
Author: nhsjgczryf
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 3.3.0
- Platform: Google Colab



Model I am using ：Bert


## To reproduce

Steps to reproduce the behavior:

Traceback (most recent call last):
  File "train.py", line 19, in <module>
    from mydataset import load_data,dist_load_data,load_data2
  File "/content/drive/My Drive/mrc4ner/mydataset.py", line 5, in <module>
    from transformers import BertTokenizer
  File "/usr/local/lib/python3.6/dist-packages/transformers/__init__.py", line 22, in <module>
    from .integrations import (  # isort:skip
  File "/usr/local/lib/python3.6/dist-packages/transformers/integrations.py", line 42, in <module>
    from .trainer_utils import PREFIX_CHECKPOINT_DIR, BestRun  # isort:skip
  File "/usr/local/lib/python3.6/dist-packages/transformers/trainer_utils.py", line 6, in <module>
    from .file_utils import is_tf_available, is_torch_available, is_torch_tpu_available
  File "/usr/local/lib/python3.6/dist-packages/transformers/file_utils.py", line 72, in <module>
    logger.debug(f"Succesfully imported datasets version {datasets.__version__}")
AttributeError: module 'datasets' has no attribute '__version__'
## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->

My code works well before, and there is a "datasets" folder in my working directory.  When my transformers version upgraded to 3.3.0, I get this error. If I change the name of the folder "datasets" or downgrade transformers to version 3.2.0, the error is get fixed.
Is this a bug?  Because it doesn't allow me to use "datasets" as a folder name.

Comments:
Sadly that is how python works, it will try to import the datasets library from a local folder if you have a folder named like this in the path your are working in. However, this should only work if there is a `__init__.py` in your folder named datasets. Removing that file should then solve the bug.

This change just broke [DeepChem](https://github.com/deepchem/deepchem).  In the short term we can work around it by pinning to an older version, but that's not a reasonable long term solution.  Directories called "datasets" are very common, and this will impact a lot of people.  Using a common, generic word as the top level package violates the [PEP 423](https://www.python.org/dev/peps/pep-0423/) guidelines for package naming.

Indeed, we are working on a fix and will release soon.

Great, thanks!

The patched release is on PyPi, tell us if you have any issue.

Works perfectly.  Thanks so much for the super fast fix!
Make transformers install check positive
Author: FremyCompany
Pull request: yes

# What does this PR do?

When transformers is correctly installed, I feel you should get a positive message. It's called huggingface not angryface, after all ;-) 

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. 
Given this concerns the documentation, mentioning @sgugger

Comments:
Remove config assumption in Trainer
Author: sgugger
Pull request: yes

# What does this PR do?

This PR tries to limit the access to `model.config` in `Trainer` to the minimum so that it works with regular PyTorch modules (as long as they accept dict inputs and return loss first like our models). The most challenging part was the storing/restoring of the `total_flos`, which I moved to the newly created `TrainerState`. It should work as before and be saved along the rest of the training state.

Comments:
Further Pretraining of Longformer RAM Consumption
Author: baradl
Pull request: no

Hello everybody,

I'm interested in continued pretraining of the longformer on in-domain text. I'm using the `run_language_modeling.py` script from the examples with the following options.

!python run_language_modeling.py \
    --output_dir=$output_dir$ \
    --model_type=longformer \
    --model_name_or_path=allenai/longformer-base-4096 \
    --do_train \
    --train_data_file=$train_file$ \
    --save_steps=5000 \
    --save_total_limit=1 \
    --overwrite_output_dir \
    --num_train_epochs=1 \
    --fp16 \
    --fp16_opt_level="O1" \
    --gradient_accumulation_steps=8 \
    --per_device_train_batch_size=1 \
    --logging_steps=100 \
    --line_by_line \
    --mlm

I tried running the script on Google Colab, spell.run and kaggle-notebooks with different behaviour. While on colab and spell.run the consumption of RAM continuously increases while training, it's constant on kaggle. On colab and spell.run the script terminates whenever the limit is reached (around 26gb on spell.run). Since AllenAI provides some code for pretraining as well I tried their implementation as well with the same behaviour, however it reaches RAM limits faster. The text file I'm using is rather small with a few hundred MB in size.

Can someone explain what's going on here and how solve this issue? I'm linking  @patrickvonplaten as it was suggested by the repo. Thanks a lot for any help!

Cheers


Comments:
Hi, there's been some issues with memory in our recent script versions. It is possible that on kaggle you have the very latest version of the scripts, or a way older version of the scripts where the memory issue didn't exist.

I recommend you use the latest version of the script, as the memory issue is fixed and it is the most feature-complete version.

Thanks for info and help! However, the issue persists in the current version of the script. 
Is it an issue specific to the longformer or do other models experience the same behaviour?
Do you know when the memory issues began so I can check whether older versions might work? 

So I tested older versions of the script, specifically from the commits that resulted in a new version (3.0.0, 3.0.1, 3.0.2, 3.1.0). The issue persists with all of them *but* only on google colab. RAM consumption begins at around 1.88gb and after around 5 minutes of training increases continuously.
I tried the same script from the commits on kaggle and observed a very different behaviour. RAM consumption at training is constant around 5.5gb. 
When running the script on roberta the same behaviour occurs.

I'd be happy about any help or suggestions!

So the docker image that kaggle uses seemed to be a work-around. If more ressources are necessary than those provided by kaggle then one can use the image easily on GCP. Though kaggle publishes the image as well in their repo.

However by now the issues seems resolved as I switched back to colab and no problems with RAM consumption occured. I'm not sure what caused this effect but no issues for now - thus I'm closing this thread.
Is there a helper script to preprocess data for T5 for masked language modeling? 
Author: abhisheksgumadi
Pull request: no
Labels: name: wontfix

Hi Team

   Thanks for the wonderful HuggingFace library ! 

I am now working with T5 on my own dataset. I want to know if there is any helper script that can automatically take text and mask a random set of tokens and also generate the expected output sequence for the pretraining unsupervised language modeling task.



Comments:
Not yet sadly - it's on my ToDo list. Hope to be able to work on it soon

I am working on a script for T5 based upon the current run_language_modeling.py, maybe I can share that once I am done and someone can confirm if it works as expected?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Hi, I'm working in the same task. [Here](https://github.com/huggingface/transformers/issues/7451) you can see my code if it helps!
Trainer should not modify its TrainingArguments
Author: sgugger
Pull request: yes

# What does this PR do?

This fixes a bug that took me some time to track in a notebook with several trainings. The bottom line is that `Trainer` should not modify its `TrainingArguments` so this fixes that part by saving the number of `max_steps` desired in the state instead of in the args. Also storing the number of training epochs for easy access in subclasses.

Comments:
Shouldn't we have a properly-typed self.state on the Trainer instance? I always find assigning instance properties within the code a bit messy

We can put everything in the new `TrainerState`. I just did the same as for `self.epoch` or `self.global_step` (and there are plenty more).

Yes I think that’d be nice 

Closing this PR as this will require a bit more work then :-)
Alphabetize model lists
Author: sgugger
Pull request: yes

# What does this PR do?

The model lists have grown a bit and so, like the doc navbar, I think we'll find our way better by alphabetizing them.
Adding new models will be easy in the README since Markdwon supports enumerated lists with 1. for all items. RestructuredText is more annoying but I'll make one script generate the proper part of index.rst automatically to make sure it's in sync with the README while I'm procrastinating something more important.

Comments:
Small QOL improvements to TrainingArguments
Author: sgugger
Pull request: yes

# What does this PR do?

Some small QOL improvements as discussed on the [forum](https://discuss.huggingface.co/t/seq2seqtrainer-questions/1276/):
- make `do_eval` defaults to `evaluation_strategy != "no"` so there is no need to pass the two
- make `run_name` defaults to `output_dir`.

Comments:
Thanks!
Seq2SeqDataset: avoid passing src_lang everywhere
Author: amanpreet692
Pull request: yes

Changed Constructor argument for AbstractSeq2SeqDataset to kwargs to avoid passing unwanted parameters to tokenizers, 
eg. src and tgt lang to T5 tokenizer.

# What does this PR do?
tokenization_util.py was generating unnecessary warning statements continuously for tokenizer arguments for seq2seq batch processing that shouldn't have been passed in the first place. Fixed this phenomenon and added a test case as suggested.

Fixes # (issue)
#7454 

## Before submitting
- This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
N/A
-Did you read the [contributor guideline]
Yes
- Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
Yes, #7454 
- Did you make sure to update the documentation with your changes? Here are the
  N/A
-Did you write any new necessary tests? 
Yes, added a relevant test in examples/seq2seq/test_datasets.py


## Who can review?
@sshleifer 

Comments:
Seq2seq example for T5 keeps on generating warning
Author: amanpreet692
Pull request: no

In the latest version from Master, on running finetune.sh for T5, was getting the following warning continuously:
**Keyword arguments {'src_lang':None,'tgt_lang':None,'add_prefix_space':False} not recognized.**

I found out this is because the translation and BART parameters are being passed to prepare_seq2seq_batch of T5Tokenizer which it cannot handle and the tokenizer in the end spits out warning for unused kwargs.
I made a small change in utils.py to the constructor at https://github.com/huggingface/transformers/blob/9e9a1fb8c75e2ef00fea9c4c0dc511fc0178081c/examples/seq2seq/utils.py#L100:
```python 
        **dataset_kwargs
    ):
        super().__init__()
        self.src_file = Path(data_dir).joinpath(type_path + ".source")
        self.tgt_file = Path(data_dir).joinpath(type_path + ".target")
        self.len_file = Path(data_dir).joinpath(type_path + ".len")
        if os.path.exists(self.len_file):
            self.src_lens = pickle_load(self.len_file)
            self.used_char_len = False
        else:
            self.src_lens = self.get_char_lens(self.src_file)
            self.used_char_len = True
        self.max_source_length = max_source_length
        self.max_target_length = max_target_length
        assert min(self.src_lens) > 0, f"found empty line in {self.src_file}"
        self.tokenizer = tokenizer
        self.prefix = prefix if prefix is not None else ""

        if n_obs is not None:
            self.src_lens = self.src_lens[:n_obs]
        self.pad_token_id = self.tokenizer.pad_token_id
        self.dataset_kwargs = dataset_kwargs
        dataset_kwargs.update({'add_prefix_space' : True} if isinstance(self.tokenizer, BartTokenizer) else {})
```

since src_lang and tgt_lang weren't being used anywhere else other than passing on to prepare_seq2seq_batch as parameters.

While calling the method I used dataset_kwargs as the paremeter which sorted out the issue:
```python
self.tokenizer.prepare_seq2seq_batch(
            [x["src_texts"] for x in batch],
            tgt_texts=[x["tgt_texts"] for x in batch],
            max_length=self.max_source_length,
            max_target_length=self.max_target_length,
            return_tensors="pt",
            **self.dataset_kwargs
        )

```

If this seems reasonable I can raise a PR and check it in?
@sshleifer @patil-suraj 


Comments:
Yes, great PR! Send it and tag me!
Bonus points for adding a test that used to break but now doesn't maybe to seq2seq/test_datasets.py

Here you go! #7470 Added a simple test case as well to test the arguments that would be sent to collate()
FileNotFoundError: File not found when running run_squad.py to fine-tune the BERT on  SQuAD v1.1.
Author: ghost
Pull request: no
Labels: name: wontfix

Hi, I am just following this tutorial https://github.com/huggingface/transformers/tree/master/examples/question-answering and created a folder named SQUAD_DIR under transformers. The train and test file were downloaded and put into the SQUAD_DIR folder. But the error is that such a file can not be found.

Comments:
This is a mismatch between the file location and what you're indicating to the script. Are you sure you're pointing to the correct directory? If so, can you try using absolute paths?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
[s2s] Fix t5 warning for distributed eval
Author: sshleifer
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests), 
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests? 


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik 
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao 
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao 
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->

Comments:
[s2s] fix kwargs style
Author: sshleifer
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests), 
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests? 


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik 
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao 
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao 
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->

Comments:
Seq2SeqTrainer Distributed: AttributeError and the RuntimeError
Author: sshleifer
Pull request: no

The following command (on 8 GPUS) fails with

```python
AttributeError: DistributedDataParallel has no attribute "config"
```


### Command
```bash
export WANDB_PROJECT=dmar
export BS=64
export GAS=1
export m=sshleifer/student_marian_en_ro_6_3
export MAX_LEN=128
python -m torch.distributed.launch --nproc_per_node=8 finetune_trainer.py \
    --tokenizer_name $m --model_name_or_path $m \
    --data_dir wmt_mar_pl \
    --output_dir marian_en_ro_6_3 --overwrite_output_dir --predict_with_generate \
    --learning_rate=3e-4 \
    --warmup_steps 500 --sortish_sampler \
    --fp16 \
    --gradient_accumulation_steps=$GAS \
    --per_device_train_batch_size=$BS --per_device_eval_batch_size=$BS \
    --freeze_encoder --freeze_embeds \
    --num_train_epochs=6 \
    --save_steps 3000 --eval_steps 3000 \
    --max_source_length $MAX_LEN --max_target_length $MAX_LEN --val_max_target_length $MAX_LEN --test_max_target_length $MAX_LEN \
    --do_train --do_eval --do_predict --evaluate_during_training\
    --predict_with_generate --logging_first_step \
    --task translation --label_smoothing 0.1 --n_gpu 8 \
    --run_name builtin_trainer_63_v8_pl \
    "$@"
```




Comments:
@sgugger After that bug fix, the next bug is:
```
RuntimeError: Precision loss when unpacking double
    tensorized_scalar = torch.Tensor(scalars).cuda()
RuntimeError: Precision loss when unpacking double
Traceback (most recent call last):
  File "finetune_trainer.py", line 442, in <module>
    main()
  File "finetune_trainer.py", line 383, in main
    model_path=model_args.model_name_or_path if os.path.isdir(model_args.model_name_or_path) else None
  File "/home/shleifer/transformers_fork/src/transformers/trainer.py", line 809, in train
    self.log(logs)
  File "/home/shleifer/transformers_fork/src/transformers/trainer.py", line 1031, in log
    total_flos = distributed_broadcast_scalars([self.total_flos]).sum().item()
  File "/home/shleifer/transformers_fork/src/transformers/trainer_utils.py", line 206, in distributed_broadcast_scalars
    tensorized_scalar = torch.Tensor(scalars).cuda()
RuntimeError: Precision loss when unpacking double
```
### Env

Apex installed.
```
- `transformers` version: 3.3.1
- Platform: Linux-4.9.0-11-amd64-x86_64-with-debian-9.12
- Python version: 3.7.4
- PyTorch version (GPU?): 1.5.1+cu101 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>
```

The next error is on @TevenLeScao 

I fixed it, will stuff into 1 PR when everything is working.
Distributed Trainer: 2 little fixes
Author: sshleifer
Pull request: yes

1) fix DDP access to `model.config`. We could also set `self.config = model.config` earlier in `__init__`
2) switch torch.Tensor -> torch.tensor. The latter "infers the dtype automatically"
After which the command in #7460 works.



CC @patil-suraj , @TevenLeScao 

Comments:
Can we see when the config is accessed (in your error message)? `model.config` should be accessed as sparsely as possible in `Trainer` to work with any kind of model and I'll probably remove the requirement entirely soon.

`Seq2SeqTrainer` uses model.config 8 times. Mostly `pad_token_id` to avoid counting padding in the loss func.

It should add an assert the model is a `PreTrainedModel` at __init__ just to be clean, then for your specific problem, it should use the function `self._actual_model()` to grab the config to avoid your error (e.g., `self.model.config` -> `self._actual_model().config`).

`Trainer` is on its way to fully handle models without config, see #7464.

OK. I reduced scope of this PR to just the `Tensor` -> `tensor`.