noamaanMulla-03/transformers-issues huggingface dataset:
Missing Import
Author: onaclov2000
Pull request: no

When trying to run this file:

https://github.com/huggingface/transformers/blob/3a10764574f252591eeaa5bbb10b778f623a4814/examples/language-modeling/run_language_modeling.py#L40

The following error occurs:
````
Traceback (most recent call last):
  File "run_language_modeling.py", line 32, in <module>
    from transformers import (
ImportError: cannot import name 'DataCollatorForWholeWordMask'
````
I attempted to run the following:
````
import transformers
for i in dir(transformers):
    if "data" in i.lower():
        print (i)
````
And I got the following:

````
CsvPipelineDataFormat
DataCollator
DataCollatorForLanguageModeling
DataCollatorForNextSentencePrediction
DataCollatorForPermutationLanguageModeling
DataCollatorForSOP
DataCollatorWithPadding
DataProcessor
GlueDataTrainingArguments                                                            
GlueDataset                                                                          
JsonPipelineDataFormat                                                               
LineByLineTextDataset
LineByLineWithSOPTextDataset                                                        
PipedPipelineDataFormat                                                              
PipelineDataFormat                                                                   
SquadDataTrainingArguments                                                           
SquadDataset                                                                         
TextDataset
TextDatasetForNextSentencePrediction                                                 
data                                                                                 
default_data_collator                                                               
is_datasets_available
````
It appears that the **DataCollatorForWholeWordMask** is not a part of transformers for some reason.

I commented it out and this one also appears to have an issue (I'll list all that complain at the bottom of this post and edit it as I find more)


At this point I commented the below imports out and it is running (well it's downloading one of the models I believe). I'll update if it fails/succeeds.

Missing Imports from transformers:
* DataCollatorForWholeWordMask
* LineByLineWithRefDataset





Comments:
Remember that the examples you pull from master need a [source install](https://huggingface.co/transformers/installation.html#installing-from-source). If you want the version that runs with the last release, you need to use that tag, here are the [examples for the last release](https://github.com/huggingface/transformers/releases/tag/v3.4.0) (v3.4.0).

You make a compelling argument :)
I did a checkout of the v3.4.0 release, and besides dying from a CUDA out of memory, it appears to be working :)
Mlflow integration callback
Author: noise-field
Pull request: yes

# What does this PR do?

This PR adds Trainer integration with [MLflow](https://mlflow.org/). 

It is implemented in roughly the same way as other integration callbacks (CometML, wandb) and gets added to the list of Trainer callbacks automatically when mlflow is installed. All the mlflow parameters are configured with env variables, as described in the library documentation. This PR adds an additional environment variable, `HF_MLFLOW_LOG_ARTIFACTS`, which controls whether to use mlflow artifact logging facility to save artifacts generated after training (it doesn't make much sense if mlflow is used locally).  

Fixes #7698


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@sgugger


Comments:
Doc styling
Author: sgugger
Pull request: yes

# What does this PR do?

This PR introduces a doc styling script and applies it to the repo. The styling script runs similarly to black, with a an option that fixes and overwrites the files (put inside `make style`) and an option that only checks if there should be a restyle, failing with an error if that's the case (put inside `make quality`).

The script is applied to all rst files inside `docs/source` and all py files inside `src/transformers`. It will look for paragraphs and always reorganize them to use the most of the `max_len` passed (set at 119 for the repo, like for the code). It will remove all duplicate or trailing whitespace, make all blank lines empty, ignore blocks of code/math and properly take care of the indentation.

A few extra things are performed:
- making the underline of the titles in rst to the `max_len` and always adding a blank line after those titles.
- unifying the format of the triple docstrings in the files
- always adding a new line before the beginning of a list (because sphinx sometimes complains otherwise)

To make the script ignore a string inside triple quotes (like warnings or long regex expressions), put a `# docstyle-ignore` somewhere before (it has to be between the previous triple quotes and the ones of the string you want to ignore).

In general, if the script reformats atrociously a docstring, it was because it was badly formatted. Adding a blank line to clearly mark paragraphs can make the script happier. Properly indenting list of arguments (see examples on any of the files of the lib) is also important to get good outputs.


Comments:
A few loose-ends to tie, but that will be for tomorrow!
Should update version requirement for scipy in 'examples\\distillation\\requirements.txt'?
Author: suliuzh
Pull request: no

There are inconsistent  version requirements for scipy in 'examples\\distillation\\requirements.txt' and 'examples\\movement-pruning\\requirements.txt'. Fixed version **1.3.1** in 'examples\\distillation\\requirements.txt' is not in the version range in **'>=1.4.1'** in  'examples\\movement-pruning\\requirements.txt'.
![image](https://user-images.githubusercontent.com/27858725/96845824-b1ba9b00-1483-11eb-9538-5c241ba3d00d.png)
**Solution**
I am wondering if it is necessary to update the version requirement in 'examples\\distillation\\requirements.txt' to be consistent. Fixed version can often cause conflict. 

Comments:
@VictorSanh what do you think?
[QOL] PretrainedConfig.to_diff_dict(other_config)
Author: sshleifer
Pull request: yes

This PR allows users to compare two configs in a backwards compatible way. 
### Solution
The following calls work:
```python
bart_large_config = BartConfig.from_pretrained('facebook/bart-large')
bart_base_config = BartConfig.from_pretrained('facebook/bart-base')
t5_config = T5Config.from_pretrained('t5-small')
bart_large_config.to_diff_dict() # unchanged
bart_large_config.to_diff_dict(bart_base_config) # compares configs
bart_large_config.to_diff_dict(t5_config) # can be across subtypes
bart_large_config.to_diff_dict(bart_base_config.to_dict()) # can be against dict
```

Adds test that outputs are reasonable.
### Problem
Current best way to compare configs is to define your own function. Here is the one I use. Also good for debugging conversion scripts:

```python
def dct_differences(dct_a, dct_b):
    SENTINEL = '__MissingKey'
    k1, k2 = set(dct_a), set(dct_b) # just the keys
    deltas = []
    for k in k1.union(k2):
        vala, valb = dct_a.get(k, SENTINEL), dct_b.get(k, SENTINEL)
        # TODO(SS): nested dicts? Maybe better to dump to json and compare (after sorting keys!)
        if vala == valb:
            if (vala == SENTINEL and valb == SENTINEL): raise AssertionError('Adversarial Sentinel Input!')
        else:
            deltas.append((k, vala, valb))
    return deltas
bart_large_config = BartConfig.from_pretrained('facebook/bart-large')
bart_base_config = BartConfig.from_pretrained('facebook/bart-base')
delta =  dct_differences(bart_large_config.to_dict(), bart_base_config.to_dict())
```
this implementation is almost as useful without breaking backwards compatibility.

Comments:
Didn't work. Will re-open when I have something better.
Load tuned model without downloading from huggingface
Author: sachinruk
Pull request: no

I have the following model which I have tuned for a classification task.
```python
BASE_MODEL = "distilbert-base-multilingual-cased"

class Model(nn.Module):
    def __init__(self, nc, p=0.1):
        super().__init__()
        self.base = AutoModel.from_pretrained(BASE_MODEL)
        in_features = 768 # self.base.pooler.dense.out_features
        self.dropout = nn.Dropout(p=p)
        self.fc = nn.Linear(in_features, nc, bias=False)
        
    def forward(self, x):
        out = self.base(**x)[0]
        out = out[:, 0, :]
        out = self.dropout(out)
        return self.fc(out)
```

However, the way that I load the model currently is by doing:
```python
model = Model(nc)
model.load_state_dict(torch.load(TUNED_MODEL_PATH))
```
The first line above causes `distibert` to be downloaded again and then my weights overwrite the model. I was hoping that there is a way of just getting the base architecture without downloading any weights.

I tried doing `self.base = DistilBertModel(DistilBertConfig())`. However, when loading the tuned model, it gives the error `size mismatch for base.embeddings.word_embeddings.weight: copying a param with shape torch.Size([119547, 768]) from checkpoint, the shape in current model is torch.Size([30522, 768]).`. I believe this is due to the fact that I am using **a multilingual** model.

Side questions: 
- Where does the AutoTokenizer/ AutoModel download the relevant files to?
- Also apologies for posting here instead of the forum. For some reason it won't let me login with the huggingface credentials.

Version: transformers==3.1.0


Comments:
 Instead of doing `self.base = DistilBertModel(DistilBertConfig())` you can do `self.base = DistilBertModel(DistilBertConfig(vocab_size=119547))`.
Hope more GPT Chinese pretrained model.
Author: guotong1988
Pull request: no

# 🚀 Feature request

Hope more GPT Chinese pretrained model.


![image](https://user-images.githubusercontent.com/4702353/97251450-e0998e00-1842-11eb-9268-9dfa837f6b29.png)

Thank you very much.

Comments:
Have you checked out the filtered list on the model hub? https://huggingface.co/models?filter=zh 

Thank you!
Fix comet_ml import and add ensure availability
Author: dsblank
Pull request: yes

# What does this PR do?

1. Adds a better check to make sure comet_ml is ready to use
2. Moves the integration imports above the ML imports. This is required to use comet_ml

The current version 3.4.0 is broken, and can not be used with comet_ml without a workaround. This PR fixes that.

## Before submitting
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?

## Who can review?

 Trainer: @sgugger


Comments:
FYI: @stas00 

> but I agree with @stas00: I wouldn't want to see warnings about cometml or wandb if I don't have the libraries installed.

This is not the case already. We were talking about the odd case where some other package installed one of these as its auto-dependencies. So the user now unwittingly needs to figure out why in the world she needs to get an API key for something she didn't ask for in first place.

Unfortunately I am forced to reset my conda env a lot recently, so I lost the one where this exact scenario has happened, so at the moment I can't point the guilty finger at which package installed `cometml` without me doing so intentionally/directly. If it happens again I will report back.

Otherwise all is good.

Resolved merge conflicts. Should be ready to go.

Thanks!
[WIP]  Create SequenceClassification MultipleChoice and TokenClassification for tf_longformer
Author: Groskilled
Pull request: yes

Tests for the new classes are not passing and there is no test yet for TFLongformerClassificationHead (and I am not sure it's needed).
@patrickvonplaten 

Resolves #6401  

Comments:
Hey @Groskilled - I added a couple of comments. Let's try to first make the torch tests pass. Let me know if you need more help after implementing the comments.

Closing PR due to inactivity
#7858 breaks IterableDataset with __len__ in Trainer
Author: changjonathanc
Pull request: no

https://github.com/huggingface/transformers/blob/08f534d2da47875a4b7eb1c125cfa7f0f3b79642/src/transformers/trainer.py#L381-L382

This used to be (before #7858)
```python
if isinstance(self.train_dataset, torch.utils.data.IterableDataset):
```

I am using IterableDataset with __len__ in Trainer. This change makes it return a sampler and results in an error later. `ValueError: DataLoader with IterableDataset: expected unspecified sampler option, but got sampler=<torch.utils.data.sampler.RandomSampler object at 0x7fa32c57b340>`

Maybe change to this?
```python
if (isinstance(self.train_dataset, torch.utils.data.IterableDataset) or
        not isinstance(self.train_dataset, collections.abc.Sized)):
```

@j-rossi-nl @sgugger 


Comments:
I'm sorry, but you could explain to me why an `IterableDataset` with a `__len__` is not a regular `Dataset`?

In my case, I wrap a `Dataset` using a class that inherits `IterableDataset`, and defines a `__len__()`.
The purpose is to implement smart batching[1]. I use `IterableDataset` so I can control how to iterate the data.

I don't know if it's possible/easier to `Dataset`+`Sampler`, if so please let me know.
Also note that (after the change) if I drop `__len__()` to suppress the bug, I would then need to specify `max_iter` (or something like that), which is inconvenient.
[1] (https://wandb.ai/pommedeterresautee/speed_training/reports/Train-HuggingFace-Models-Twice-As-Fast--VmlldzoxMDgzOTI)



It does seem a bit hacky but I guess we can add that test. Do you want to suggest a PR with the change?
Fix assertion error message for MLflowCallback
Author: harupy
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->


Comments:
Minor typo fixes to the preprocessing tutorial in the docs
Author: albanie
Pull request: yes

Minor typo fixes to the preprocessing tutorial in the docs

# What does this PR do?

Minor typo fixes to the tokenizer summary in the docs

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

## Who can review?
 documentation: @sgugger


Comments:
Move style_doc to extra_quality_checks
Author: sshleifer
Pull request: yes

Previously, if you have a doc style error in a .rst file,
```bash
python utils/style_doc.py $(modified_py_files) --max_len 119;
```
 wouldn't catch it

Comments:
Ideally there should be a modified_py_and_rst_files to speed up this check (and only apply it to modified files), but this works in the meantime. @stas00 if you want to do that last bit of optimization, let me know, otherwise I'll do that later.

Go for it, @sgugger. The modified files var is there, so it should be easy to apply it anywhere.

If you get stuck I'm here to help.
T5 Decoder Inputs
Author: alexorona
Pull request: no

# ❓ Questions & Help
Just confirming that my data preprocessing is perfect for T5. I added a print statement in `T5ConditionalGeneration` for the `decoder_input_ids` and `decoder_attention_mask` just before they're passed to the decoder. Which of these is right?

```
# pad_token prepended, eos_token is not in the sequence
decoder_input_ids: tensor([[0, 2018, 55, 0, 0, 0, 0]])
decoder_attention_mask: tensor([[1, 1, 1, 0, 0, 0, 0]])

# pad_token prepended, eos_token unmasked in attention
decoder_input_ids: tensor([[0, 2018, 55, 1, 0, 0, 0]])
decoder_attention_mask: tensor([[1, 1, 1, 1, 0, 0, 0]])

# pad_token prepended, eos_token masked in attention
decoder_input_ids: tensor([[0, 2018, 55, 1, 0, 0, 0]])
decoder_attention_mask: tensor([[1, 1, 1, 0, 0, 0, 0]])
```

Comments:
your 2nd option is correct here: 

```python
# pad_token prepended, eos_token unmasked in attention
decoder_input_ids: tensor([[0, 2018, 55, 1, 0, 0, 0]])
decoder_attention_mask: tensor([[1, 1, 1, 1, 0, 0, 0]])
```

1) You have to start with `decoder_start_token_id = pad_token_id` in T5 and 
2) the last EOS token should be attended to because the model "should learn" when the sentence is finished.

@patrickvonplaten Thanks, Patrick! That makes perfect sense. You're awesome!

@patrickvonplaten I noticed that when you pass this to the model:

```
decoder_input_ids: tensor([[2018, 55, 1, 0]])
decoder_attention_mask: tensor([[1, 1, 1, 0])
```
`T5ConditionalGeneration` changes it to this before passing it to the decoder:

```
# Masks the eos_token
# Correctly prepends an extra pad_id to inputs BUT appends a pad_token to attention_mask
decoder_input_ids: tensor([[0, 2018, 55, 1, 0]])
decoder_attention_mask: tensor([[1, 1, 1, 0, 0])
```
So that you have to actually pass this initially:

```
# Pass this to model
decoder_input_ids: tensor([[2018, 55, 1, 0]])
decoder_attention_mask: tensor([[1, 1, 1, 1])

# Which turns into this
decoder_input_ids: tensor([[0, 2018, 55, 1, 0]])
decoder_attention_mask: tensor([[1, 1, 1, 1, 0])
```

Hey @alexorona - sorry I don't quite follow here...could you provide a code example that I can run to see what you mean? :-)
[setup] update/add setup targets
Author: stas00
Pull request: yes

updating pip targets

* [x] adds `tokenizer`
* [x] adds `docs` to `dev`, since we need to have the tools to run `make docs`
* [x] adds `flax` to `dev`, since we need to have the libs to run flax tests - except when on windows - it skips it then
* [x] brings `all` up-to-date


@sgugger 

Comments:
Guys, since this is not code but a bunch of definitions please make direct suggestions that can be merged.  Thank you. 

I originally just wanted to add `docs` and `flax` to `dev` and had no idea about all the other needs.

I can't make a suggestion that moves stuff around in the file. I can push a commit to your branch if you want, but that's the best I can do.

I understand. I just don't know to work in this fashion. If you give me a spec I can work to implement it. Otherwise let's just merge this and subsequent PRs can do further improvements. 

Sure, I'll add my comments in a separate PR. Thanks for doing this one!
Fully remove codecov
Author: LysandreJik
Pull request: yes

Fully removes codecov as we don't have any full test suite in circleci, alongside https://github.com/huggingface/transformers/commit/829b9f8cc321aa28396e6203e0f21eed26b132f7

If we want to put it back up, we can merge the two slow tests (TF + PT) and run coverage on that, but we should first take care of the inconsistencies in coverage as explained in https://github.com/huggingface/transformers/issues/6317

cc @stas00 

Comments:
If we try to go back to using it, perhaps there is a way to merge reports from 2 half-full tests - otherwise it creates an unnecessary slowdown on CI  to run both together if we don't need to.