noamaanMulla-03/transformers-issues huggingface dataset:
Error when loading weights with a different 'projection_dim' in DPRQuestionEncoder
Author: gianlucabusatta
Pull request: no

## Environment info
```
- `transformers` version: 4.10.0
- Platform: Linux-5.4.104+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.11
- PyTorch version (GPU?): 1.9.0+cu102 (True)
- Tensorflow version (GPU?): 2.6.0 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No
```

### Who can help
research_projects/rag: @patrickvonplaten, @lhoestq

## To reproduce

```
from transformers import DPRQuestionEncoder, DPRConfig

model_name = "voidful/dpr-question_encoder-bert-base-multilingual"

config = DPRConfig.from_pretrained(model_name)
config.projection_dim = 512

model = DPRQuestionEncoder.from_pretrained(model_name, config=config)
```

# Error
```
NotImplementedError                       Traceback (most recent call last)
<ipython-input-16-0256969428e8> in <module>()
     11 config.projection_dim = 512
     12 
---> 13 model = DPRQuestionEncoder.from_pretrained(model_name, config=config)

2 frames
/usr/local/lib/python3.7/dist-packages/transformers/modeling_utils.py in _init_weights(self, module)
    577         Initialize the weights. This method should be overridden by derived class.
    578         """
--> 579         raise NotImplementedError(f"Make sure `_init_weigths` is implemented for {self.__class__}")
    580 
    581     def tie_weights(self):

NotImplementedError: Make sure `_init_weigths` is implemented for <class 'transformers.models.dpr.modeling_dpr.DPRQuestionEncoder'>
```

## Expected behavior

Be able to change `config.projection_dim` without errors.


Comments:
This is indeed an error and should be fixed by the PR linked above
[examples/flax] use Repository API for push_to_hub
Author: patil-suraj
Pull request: yes

# What does this PR do?

This PR updates all flax examples scripts to use the `Repository` API for `push_to_hub` and removes all instructions to manually create/clone a repo in the READMEs.

Comments:
> Thanks a lot for taking care of this - can we leave the symbolic links though so that people can continue following the steps like a recipe?

@patrickvonplaten the issue with this is that, we need to pass "./" or "." as the `output_dir` which does not work with `Repository`. 

What these changes do is that they create a directory for model/tokenize/config in `/tmp` and uses that path to save the `tokenizer` and `config` and use that as the `output_dir`.  So the users should still be able to follow the steps only difference is the script will nbe run from its own directory rather than creating a symlink.

Thanks, @LysandreJik for pointing that out, we could change the path from `/tmp` to something else. Some of our PyTorch examples also use `/tmp`, so we should also correct those no?

> Also Repository(".", clone_from="xxx") should work!

It does! Issue is 

```python
 if training_args.hub_model_id is None:
            repo_name = get_full_repo_name(Path(training_args.output_dir).name, token=training_args.hub_token)
        else:
            repo_name = training_args.hub_model_id
        repo = Repository(training_args.output_dir, clone_from=repo_name)
```

if `hub_model_id` is not passed, `repo_name` is derived from `output_dir`, and when the path is `"."`, then `path.name` returns an empty string, which then results in this error
`requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://huggingface.co/api/repos/create - Only regular characters and '-', '_', '.' accepted`

Maybe the `Path(training_args.output_dir).name` should be updated to `Path(training_args.output_dir).absolute().name` in that case?

Aah, yeah! Thanks Sylvain :)
[testing] auto-replay captured streams
Author: stas00
Pull request: yes

This is a sync with the BigScience project https://github.com/bigscience-workshop/Megatron-DeepSpeed/pull/120

tldr; `CaptureStd*` interferes with `pytest` flags like `-sv` that should normally prints the std streams to the output. This PR fixes that.

Detailed: We noticed that when `CaptureStd` was used on the whole sub-process run it'd make it very difficult to debug problematic tests, since well it eats the output. So this PR changes the default behavior to replay the captured streams and adds a config arg to disable that.

This is a testing feature so backward compatibility shouldn't be an issue, and it will make things easier for us developers, rather than more difficult. And by default `pytest` captures/hides the std streams anyway.

@sgugger, @LysandreJik 

Comments:
MultiBerts in Huggingface
Author: yanaiela
Pull request: no
Labels: name: New model

# 🚀 Feature request

It would be nice to have a script/converter of all the [multiberts](https://arxiv.org/pdf/2106.16163.pdf) checkpoints that were released about a month ago.
It was [released](https://github.com/google-research/language/tree/master/language/multiberts) by Google, thus using tensorflow, and it would be nice to have it under the huggingface library/hub.

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->

## Motivation

<!-- Please outline the motivation for the proposal. Is your feature request
     related to a problem? e.g., I'm always frustrated when [...]. If this is related
     to another GitHub issue, please link here too. -->
These checkpoints may be useful to study training dynamics and hypotheses over multiple seeds that were used to train the model.


Comments:
Pretrained Wav2Vec2 Model large robust does not load
Author: m-wiesner
Pull request: no

@patrickvonplaten

The behavior I notice is shown below. Any help is much appreciated!

Python 3.7.10
>>> from transformers import Wav2Vec2ForCTC
>>> mdl = Wav2Vec2ForCTC.from_pretrained('facebook/wav2vec2-large-robust')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/hltcoe/mwiesner/.conda/envs/nnet_pytorch_env/lib/python3.7/site-packages/transformers/modeling_utils.py", line 1325, in from_pretrained
    model = cls(config, *model_args, **model_kwargs)
  File "/home/hltcoe/mwiesner/.conda/envs/nnet_pytorch_env/lib/python3.7/site-packages/transformers/models/wav2vec2/modeling_wav2vec2.py", line 1412, in __init__
    self.lm_head = nn.Linear(config.hidden_size, config.vocab_size)
  File "/home/hltcoe/mwiesner/.conda/envs/nnet_pytorch_env/lib/python3.7/site-packages/torch/nn/modules/linear.py", line 78, in __init__
    self.weight = Parameter(torch.Tensor(out_features, in_features))
TypeError: new() received an invalid combination of arguments - got (NoneType, int), but expected one of:
 * (*, torch.device device)
      didn't match because some of the arguments have invalid types: (NoneType, int)
 * (torch.Storage storage)
 * (Tensor other)
 * (tuple of ints size, *, torch.device device)
 * (object data, *, torch.device device)

Comments:
Hey @m-wiesner, 

`'facebook/wav2vec2-large-robust'` is just a pretrained speech recognition model that does not have a lm head defined. So the model should be loaded as follows:

```python
from transformers import Wav2Vec2ForPreTraining
mdl = Wav2Vec2ForPreTraining.from_pretrained('facebook/wav2vec2-large-robust')
```

If you would like to use the model to fine-tune it on a downstream task you have to define the vocabulary size first, *e.g.*:

```python
from transformers import Wav2Vec2ForCTC
mdl = Wav2Vec2ForCTC.from_pretrained('facebook/wav2vec2-large-robust', vocab_size=32)
```

which will then initialize a random linear layer as the lm head that needs to be fine-tuned.

But in any way the error message should have been better here! Let me correct this :-)


🎉

Awesome. Most of the other models have the LM head attached and I've just been discarding that. Would it make sense to update the documentation to reflect the discrepancy between this and how the other wav2vec2 models are loaded?

Thanks for the rapid response!

Pretrained wav2vec2 models ideally should never have a vocab_size defined in the config. We can't change this anymore though because of backward compatibility.
[DPR] Correct init
Author: patrickvonplaten
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

This PR corrects the fast initialization scheme for DPR. DPR overwrote the `init_weights` function instead of implementing a `_init_weights`. This PR corrects this to align DPR more with the other models.

Comments:
modeling_fnet.py config option 
Author: huu4ontocord
Pull request: no

https://github.com/huggingface/transformers/blob/b7d264be0d70d5309dbf24a6c7ebba1e073bdda5/src/transformers/models/fnet/modeling_fnet.py#L177

Should refer to config.tpu_short_seq_length


Comments:
Maybe of interest to @gchhablani :)

So sorry about that @ontocord.

I will fix it immediately. Thanks a lot for pointing it out.

I made a few more fixes. See this notebook. Basically I moved the buffer for doing fft to the encoder class, and made it a parameter of the module. in this way you can fix problems with cuda vs non cuda. the parameter weren't converting to cuda. i also made one fix relating to dtype conversion so you can run in half mode. You can do a diff and feel free to accept and PR it if you wish!

https://colab.research.google.com/drive/19A-qrta2yqVcfSjUtjEv6w1HqOyYXuEo#scrollTo=olgITNUmyQxA


As an aside @gchhablani, did you run any speed tests. My small test didn't show much diff w/ bert. but I suppose the authors said that training is faster. I supposed I have to test it on something longer?

https://arxiv.org/abs/2105.03824

@ontocord I ran fine-tuning script on GLUE and the FNet model is faster than BERT. It takes 70% time (train + eval). The original ratios mentioned are only training and model is in Flax.

If you are talking about fourier transform only, then I didn't perform a test of speed for the three different ways they are implemented - GPU, TPU short sequence, TPU long sequence. I assume it will be slower because of PyTorch.

An interesting comparison could be creating a HuggingFace Flax model and comparing with the original. 

@patrickvonplaten what do you think?

Could you all share the fine-tuning script or a colab that shows the run? 70% of the time of BERT or 70% faster than BERT?


@ontocord It is 70% of the time of BERT in our case, but our script is different, and we used PyTorch. Torch does not have `vmap` as of now, because of which I expect slow down in the performance.

You can see the runs (go to Metrics) in the model cards: https://huggingface.co/models?other=fnet-bert-base-comparison

@ontocord I checked your changes, maybe you can open a PR and we can discuss? It'll be easier to involve others as well.

Sorry didn't see your message. Looks like you already did this... Great job :)
LengthGroupedSampler: why longest examples go first?
Author: ZhaofengWu
Pull request: no

Is there any reason for the longest, but not the shortest, examples to go first? I think in some curriculum learning literature people start with the easiest example. What's design decision behind starting with the longest ones?

Comments:
Correct me if I'm wrong @sgugger, but I believe this is because this way, if a memory error happens, it's at the absolute beginning - and not mid-training, which is much more annoying.

Exactly, this is to get the OOM as soon as possible if it should ever happen.

That makes a lot of sense -- thanks!
Support fp16 for inference
Author: urimerhav
Pull request: no
Labels: name: wontfix

# 🚀 Feature request - support fp16 inference

Right now most models support mixed precision for model training, but not for inference. Naively calling `model= model.haf()` makes the model generate junk instead of valid results for text generation, even though mixed-precision works fine in training. 

If there's a way to make the model produce stable behavior at 16-bit precision at inference, the throughput can potentially double on most modern GPUS. 

## Motivation

Double the speed is always attractive, especially since transformers are compute-intensive. 



Comments:
I imagine this only happens when the `generate()` method is used under the hood? 

Hi there!  It's true you can't just do `model.half()` for generation. There is nothing in Trainer/Seq2SeqTrainer right now for FP16-inference, only training, but we're looking at it right now through #8403. So stay tuned!

Thanks for the input @sgugger. Good to know we're not missing something here and it's actually unsupported somehow. 

Hi,
I've noticed the same issue of the model randomly generating junk when using autocast within a custom generate() method with the only change below (fp16 is a boolean). From the above comments I thought this approach should've worked.
```
if fp16:
	with torch.cuda.amp.autocast():
		outputs = self(**model_inputs)
else:
	outputs = self(**model_inputs)
```

The current model I've tested it on is a huggingface gpt2 model finetuned on a personal dataset. Without fp16 the generate works perfectly. The dataset is very specific and the model is supposed to generate symbols+numbers, so it's clear when it starts spitting out words during fp16 inference.


This issue has been automatically marked as stale and been closed because it has not had recent activity. Thank you for your contributions.

If you think this still needs to be addressed please comment on this thread.

has this been solved?
skip gptj slow generate tests
Author: patil-suraj
Pull request: yes

# What does this PR do?

As discussed offline, this PR skips the slow generations tests for GPT-J due to GPU OOM. These should be re-enabled with a bigger GPU on CI or when model parallelism #13726 is implemented and then run the tests on multi-GPU.

These slow tests should be run manually before merging anything related to GPTJ modeling.

Comments:
[optim] implement AdafactorSchedule
Author: stas00
Pull request: yes

Currently Adafactor doesn't use an external scheduler and doesn't expose its lr values, and especially as reported in https://github.com/huggingface/transformers/issues/11612 the Trainer can't work without a scheduler, so this PR:
- implements `AdafactorSchedule` which is a proxy to `Adafactor` and can pull the lr values out of it
- adds a basic test
- updates docs

The implementation is somewhat hackish, but it's good enough for now.

Fixes: https://github.com/huggingface/transformers/issues/11612

@sgugger, @LysandreJik 

Comments:
Correct me if I am wrong, but I see two flaws with the current solution:
1) Only the first learning rate is returned as float, the following ones are tensors of size 1 - gives error while trying to pickle the logging history,
2) Adafactor has separate learning rates for each of the network components (Linear layers, normalizations...). The current solution gives only the LR of the first component, usually the embedding matrix.


In other words you're saying this was a half-baked solution. It is very much so. The original workaround idea was just to return a dumb number, to make things work with HF Trainer, as Adafactor wasn't designed to share its LRs with other components.

@LukasStankevicius, would you like to enhance my initial hack to fully support the features you mentioned lacking/incomplete? It surely could use some TLC.

For my own use, I modified Adafactor scheduler as follows:
```python
from transformers.optimization import AdafactorSchedule

class MyAdafactorSchedule(AdafactorSchedule):
    def get_lr(self):
        opt = self.optimizer
        if "step" in opt.state[opt.param_groups[0]["params"][0]]:
            lrs = [opt._get_lr(group, opt.state[p]).item() for group in opt.param_groups for p in group["params"]]
        else:
            lrs = []
        return [lrs]
```

Now it does not give errors while pickling logging history and reports learning rates for all components. However, it pollutes the logs (a single logged step may contain a list of over 100 learning rates).

You may average, but then, that is a point of logging lr at all?
So, I do not know the optimal solution here. Maybe just warning in documentation about Adafactor learning rates.
Updating CITATION.cff to fix GitHub citation prompt BibTeX output.
Author: arfon
Pull request: yes

# What does this PR do?

This is a small tweak to the `CITATION.cff` file to fix the output from the GitHub 'cite this repository' prompt, as [this is now supported](https://github.com/citation-file-format/ruby-cff/commit/5407f8e207c1f362568bb21e22865a512be9841f) in the Gem GitHub uses, and that gem is now live on GitHub.

Old BibTeX output: 

```

@misc{Wolf_Transformers_StateoftheArt_Natural_2020,
    author = {Wolf, Thomas and Debut, Lysandre and Sanh, Victor and Chaumond, Julien and Delangue, Clement and Moi, Anthony and Cistac, Perric and Ma, Clara and Jernite, Yacine and Plu, Julien and Xu, Canwen and Le Scao, Teven and Gugger, Sylvain and Drame, Mariama and Lhoest, Quentin and Rush, Alexander M.},
    month = {10},
    pages = {38--45},
    title = {{Transformers: State-of-the-Art Natural Language Processing}},
    url = {https://www.aclweb.org/anthology/2020.emnlp-demos.6},
    year = {2020}
}
```

New BibTeX output:

```
@inproceedings{Wolf_Transformers_StateoftheArt_Natural_2020,
    author = {Wolf, Thomas and Debut, Lysandre and Sanh, Victor and Chaumond, Julien and Delangue, Clement and Moi, Anthony and Cistac, Perric and Ma, Clara and Jernite, Yacine and Plu, Julien and Xu, Canwen and Le Scao, Teven and Gugger, Sylvain and Drame, Mariama and Lhoest, Quentin and Rush, Alexander M.},
    month = {10},
    pages = {38--45},
    publisher = {Association for Computational Linguistics},
    title = {{Transformers: State-of-the-Art Natural Language Processing}},
    url = {https://www.aclweb.org/anthology/2020.emnlp-demos.6},
    year = {2020}
}
```

Tagging @sgugger who merged the original PR that added a `CITATION.cff` file in https://github.com/huggingface/transformers/pull/13214.


Comments:
Correct way to use pre-trained models - Any document on this?
Author: pratikchhapolika
Pull request: no

I want to do `Multiclass-Multilabel ( MLMC)` **classification** problem using `Conv-BERT` model.

**Steps that I have taken is:**

I downloaded the Conv-Bert model from this link: https://huggingface.co/YituTech/conv-bert-base
<< **YituTech/conv-bert-base**>>


```
from pytorch_pretrained_bert import BertTokenizer, BertForSequenceClassification, BertAdam
tokenizer = **BertTokenizer.from_pretrained**("path_to_Conv-Bert_model", do_lower_case = True)
model = **BertForSequenceClassification.from_pretrained**("path_to_Conv-Bert_model", num_labels = 240)
model.cuda()
```

I want to understand can we call any classification module from Hugging face and pass any pre-trained models to it like `Roberta, Conv-BERT.. so on`. ? << As in above example>> 
Is it mandatory to use Conv-Bert classification pre-trained model ?

Comments:
Hello! We have several documents that can help you get started! First of all, the [quicktour](https://huggingface.co/transformers/quicktour.html), and the [free course](https://huggingface.co/course/chapter1) of the HF ecosystem may help you out.

> Hello! We have several documents that can help you get started! First of all, the [quicktour](https://huggingface.co/transformers/quicktour.html), and the [free course](https://huggingface.co/course/chapter1) of the HF ecosystem may help you out.

What about my code above. Is this correct way of doing things?

@pratikchhapolika 
Hi Pratik, yes you can use most models for sequence classification. You can do the following


```
from transformers import AutoModelForSequenceClassification, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("name_of_base_model")
model = AutoModelForSequenceClassification("name_of_base_model")

//name_of_base_model can be bert-base-cased, albert-base-v2, roberta-large etc. 
```
The full list is [here](https://huggingface.co/transformers/pretrained_models.html)
You can then use the model & finetune it on the desired classification task (e.g. GLUE / SUPERGLUE)


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Bert Loses Patience - Batch Inference Doubt
Author: tanmaylaud
Pull request: no

In PABEE, with evaluate every layer output for a given patience threshold.

This can be done with if else when batch size is 1. 
How does this code work when batch_size > 1:
https://github.com/huggingface/transformers/blob/2772d3e79d66925cf4adeaffd8be610f0ab177b6/examples/research_projects/bert-loses-patience/pabee/modeling_pabee_bert.py#L224

When batch size is greater than 1, I believe every input in the batch will have its own patience outcome. But torch.all would wait for the longest patience outcome.
@JetRunner 
Won't the patience counter dimension be equal to batch size ?

Comments:
Thanks for your interest in our work Tanmay!

The code only considers the scenario when `batch_size=1`. Theoretically, we can do batch inference but as you noted we have to wait for the longest patience outcome anyway so it's not very useful. 

@JetRunner Thanks for the prompt response! 
Is there any better way to handle batch size greater than 1? 

I don't think so. If you really want to fully use the GPU, you can try multi-threads but I don't really recommend that. `bs=1` is fast enough to me.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.