noamaanMulla-03/transformers-issues huggingface dataset:
Fix PT TF ViTMAE
Author: ydshieh
Pull request: yes

# What does this PR do?

Fix PT TF ViTMAE: just use some settings both in PT/TF (instead of in only one model). Otherwise, the PT/TF equivalence tests for them won't use something like `std = 0.02` , and gets larger (init) weights --> larger diff in outputs. 

Also, **the `eps` for `layer norm` layers should be the same in PT/TF**.
(not a real big deal in practice, since here is `1e-5` v.s. `1e-12` -> but it also affects the tests)

Comments:
Let's merge since CI fails quite a lot on this one

> Let's merge since CI fails quite a lot on this one

Ok!
command line_by_line missing in https://github.com/huggingface/transformers/tree/master/examples/language-modeling
Author: TalitaGroetzinger
Pull request: no
Labels: name: wontfix

## Environment info
- `transformers` version: 3.5.0
- Platform: macOS-10.16-x86_64-i386-64bit
- Python version: 3.8.6
- PyTorch version (GPU?): 1.7.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help
@LysandreJik @patrickvonplaten @TevenLeScao


## Information

Model I am using (Bert, XLNet ...):

The problem arises when using:
* [ X] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)
https://github.com/huggingface/transformers/tree/master/examples/language-modeling


## To reproduce

In the old version of the script ``run_clm.py`` called ``run_language_modeling`` there was an argument ``line_by_line`` which allowed to read the data by putting each sequence on a line. This argument seems to be missing in the newer version 
``run_clm.py``. 

## Expected behavior
Perhaps there is an argument that has replaced ``line_by_line``  but I don't really see that. Sorry if I missed something. 



Comments:
Pinging @sgugger here - think he knows `run_clm.py` best

`run_clm` does not have the `line_by_line` option as it doesn't make sense for causal language modeling: pretraining for causal language modeling is done by concatenating all available texts separated by a special token, then building sequences of a certain `block_size` with them. Using `line_by_line` and keeping the sentences separate result in the model having to predict the padding token quite often (which pretrained causal models usually don't have) and without knowing when to stop predicting that padding token.

Only `run_mlm` keeps that option as it makes sense to have sentences of different lengths for masked language modeling. You can always copy the relevant bit of code in `run_mlm` to use it in `run_clm` but I would strongly advise against it.

Thank you for your reply, it makes sense. So that means that I need to give the script the concatenated data (and separate sequences by a special token)? Or does the script ``run_clm`` take care of that? 

The script takes care of that for you :-)

This issue has been automatically marked as stale and been closed because it has not had recent activity. Thank you for your contributions.

If you think this still needs to be addressed please comment on this thread.

@sgugger sorry to revive that thread.
I think a clm with two special tokens BOS / EOS would make sense to be trained in line by line mode, what do you think ?
(btw are you saying that all pre-trained gpt2 models are trained in fixed blocks ? if so do you confirm that original papers do the same when benchmarking with standards like 1BW ?)
thanks for your insight.
Blenderbot 1.0B Distilled eats up memory over many inferences
Author: alu13
Pull request: no

Hi, I've noticed that over the course of many inferences, the  Blenderbot 1.0B Distilled model continuously allocates GPU memory and eventually causes the GPU to crash. My project only uses single-turn inferences, and I was wondering how to prevent Blenderbot from continuously allocating memory. Thanks
<img width="1209" alt="Screen Shot 2022-03-06 at 11 27 45 PM" src="https://user-images.githubusercontent.com/12601917/156968067-425cf072-a182-4f85-87f7-64517ef2b93b.png">
!

Comments:
It's hard to say anything without looking at the code. Also for such general question, [forum](https://discuss.huggingface.co/) would be the best place to ask. Thank you!

Hi, thank you! I'll definitely post in the forum. Also some extra information, the inferences work for the first few hundred posts, but then I run into the error above and I can't get any more inferences after.

My code is below. It's just basic generation (with somewhat high volume). Thanks!
<img width="1244" alt="Screen Shot 2022-03-07 at 4 21 31 PM" src="https://user-images.githubusercontent.com/12601917/157120053-fa9b94dd-2931-4c03-9c8a-4bc747bebba0.png">




This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
error invoking create_optimizer from Jupyter lab
Author: bpraveenk
Pull request: no

- `transformers` version: 4.17.0
- Platform: Darwin-21.3.0-x86_64-i386-64bit
- Python version: 3.7.3
- PyTorch version (GPU?): 1.6.0 (False)
- Tensorflow version (GPU?): 2.8.0 (False)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help
@Rocketknight1, @patrickvonplaten 

## Information
Model I am using (Bert, XLNet ...): BART

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

import tensorflow as tf
import transformers
from transformers import BartConfig, TFBartForConditionalGeneration, BartTokenizerFast

create_optimizer(
        init_lr=2e-4,
        num_train_steps=18000,
        num_warmup_steps=100,
    )

** Error message **

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-100-7951434374ef> in <module>
      3         init_lr=2e-4,
      4         num_train_steps=18000,
----> 5         num_warmup_steps=100,
      6     )

~/anaconda3/lib/python3.7/site-packages/transformers/utils/dummy_tf_objects.py in create_optimizer(*args, **kwargs)
   2155 
   2156 def create_optimizer(*args, **kwargs):
-> 2157     requires_backends(create_optimizer, ["tf"])
   2158 
   2159 

~/anaconda3/lib/python3.7/site-packages/transformers/file_utils.py in requires_backends(obj, backends)
    846     failed = [msg.format(name) for available, msg in checks if not available()]
    847     if failed:
--> 848         raise ImportError("".join(failed))
    849 
    850 

ImportError: 
create_optimizer requires the TensorFlow library but it was not found in your environment. Checkout the instructions on the
installation page: https://www.tensorflow.org/install and follow the ones that match your environment.


## Expected behavior
Error free create_optimizer invocation


Comments:
I could fix the issue by running `pip install importlib`. Shouldn't importlib be installed automatically as part of transformers installation? I had to go through the code to identify the importlib dependency.

Hi, I can't reproduce this issue. I made a [Colab notebook](https://colab.research.google.com/drive/1BmkheJNsw9tOyirgW0CJujdluVI0sOnJ?usp=sharing) to test and `create_optimizer` worked correctly. Is it possibly a Conda issue?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
How to add uppercase letter in chinese-roberta-wwm-ext-large vocab?
Author: scaler2017
Pull request: no

I found chinese-roberta-wwm-ext-large vocab  without uppercase letter
?How to add uppercase letter in vocab?

chinese-roberta-wwm-ext-large:  https://huggingface.co/hfl/chinese-roberta-wwm-ext-large/tree/main


Comments:
Hi @scaler2017 👋  We try to reserve GitHub issues for bugs and unexpected code behavior. For further other questions, please use our forum here: https://discuss.huggingface.co/ :)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
The documentation of transformers.generation_utils.GenerationMixin.generate doesn't match the code of it
Author: zhaowei-wang-nlp
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.17.0
- Platform: Linux
- Python version: 3.8.12
- PyTorch version (GPU?): 1.7.1
- Tensorflow version (GPU?):
- Using GPU in script?:
- Using distributed or parallel set-up in script?:

### Who can help
 @sgugger@patrickvonplaten @narsil

## Information
The documentation of transformers.generation_utils.GenerationMixin.generate shows many parameters have default value, but the code of it shows all parameters are set to be None by default.

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce
It is about documentation, just look at it.

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->


Comments:
Hi @zhaowei-wang98 ,

The documentation is sort of correct I think.

The values will be defaulted from the config values of the model, which themselves have a default value which is the one provided in the docs: check here how those variables are initialized https://github.com/huggingface/transformers/blob/master/src/transformers/generation_utils.py#L1062

So even though, technically they are None by default, they will actually inherit the default of the config values (which can be overidden too).
It seems `max_length` does say the default values comes from the config, where other values don't necessarily.
@patrickvonplaten Should we harmonize?

So I think the doc reflects reality as much as possible. https://huggingface.co/docs/transformers/v4.17.0/en/main_classes/model#transformers.generation_utils.GenerationMixin.generate

But maybe some value is incorrectly reported in particular ?
Did you encounter a particular issue with the default values ?

We do write this in the docs:

> Apart from inputs, all the arguments below will default to the value of the attribute of the same name inside the [PretrainedConfig](https://huggingface.co/docs/transformers/v4.17.0/en/main_classes/configuration#transformers.PretrainedConfig) of the model. The default values indicated are the default values of those config.

Let's maybe wrap this message in a warning to make it clearer? Putting it in my generate doc refactor PR: https://github.com/huggingface/transformers/pull/15988

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
what's the difference between Megatron-gpt2 and GPT2 inside transformers?
Author: lucasjinreal
Pull request: no

what's the difference between Megatron-gpt2 and GPT2 inside transformers?

Comments:
Hi @jinfagang ! For such general questions please use the [forum](https://discuss.huggingface.co/). We use issues for bug reports and feature requests.

Megatron-gpt2  is the models trained using the megatron library. The arch is similar to the GPT2 arch. You can find more info in the doc https://huggingface.co/docs/transformers/model_doc/megatron_gpt2

@patil-suraj thanks. I'll move to forum for further discuss. hope you can give me a hand on understanding.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
CI: non-remote GH Actions now use a python venv
Author: gante
Pull request: yes

# What does this PR do?
 
This PR changes the cached python environment in non-remote GH actions to fresh virtual environments, that are rebuilt whenever `setup.py` gets updated. This ensures our test environment is closer to what a new user would see in the wild, and that we don't have package constraints due to other installed libraries (that come bundled in `ubuntu-latest`).

As a side effect, because the cached venv requires no installation of python packages on a cache hit, we also got slightly faster CI: 
- \>3 mins faster in `Add new model like template tests`
- \>3 mins faster in `Model templates runner`
(total of ~7 mins per full CI run when there is a cache hit)

Examples of runs:
cache miss -> create new venv: https://github.com/huggingface/transformers/runs/6027339356?check_suite_focus=true
cache hit -> load cached venv: https://github.com/huggingface/transformers/runs/6028381114?check_suite_focus=true

Comments:
_The documentation is not available anymore as the PR was closed or merged._

(cc @ydshieh )
TF generate refactor - XLA sample
Author: gante
Pull request: yes

# What does this PR do?

This PR brings XLA to `sample`, in `generate`. Four important details before reviewing:
1. The diff has the changes of https://github.com/huggingface/transformers/pull/16704, review that PR first plz :) It fixes a test from `beam_search`. I will rebase as soon as the other PR gets merged (the changes were bundled to confirm that it passes all generate tests).
2. The body is mostly copy/paste from `greedy_search`;
3. The sample step was changed from the previous implementation -- if we want to seed sampling with XLA, we need to use the `stateless` functions;
4. The XLA sample tests do not compare all generated tokens to their non-XLA sample counterparts, due to the numerical instabilities discussed on Slack. We do compare the first tokens, which are the same.

Finally, tests have been run for the usual models (`gpt2`, `t5`, `rag`, `speech2text`, `encoder_decoder`, `vision_encoder_decoder`, `bart`). 

____________________________

I've also run a quick sanity check on GPU. Using GPT2+sample, on an Nvidia T4:
- eager TF: ~1.7s
- XLA TF: ~54ms (~22s compile time) :point_right: 31x speedup

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@patrickvonplaten the `stateless` TF functions accept a `seed` argument that is a tuple of two integers 😅 Not very intuitive, I agree. They correspond to the `key` and `counter` used in the internal RNG algorithms ([source](https://www.tensorflow.org/api_docs/python/tf/random/Generator#from_key_counter)).

If you think it will be unintuitive for users, I can change it so that our `seed` argument corresponds to the `key` of the tuple (i.e. a single integer), and fix the `counter` to `0`. For practical purposes, it should be the same thing.

> @patrickvonplaten the `stateless` TF functions accept a `seed` argument that is a tuple of two integers sweat_smile Not very intuitive, I agree. They correspond to the `key` and `counter` used in the internal RNG algorithms ([source](https://www.tensorflow.org/api_docs/python/tf/random/Generator#from_key_counter)).
> 
> If you think it will be unintuitive for users, I can change it so that our `seed` argument corresponds to the `key` of the tuple (i.e. a single integer), and fix the `counter` to `0`. For practical purposes, it should be the same thing.

I see - ok maybe better to leave as is then to be aligned with TF

While running tests for T5 (as suggested by @Rocketknight1), I found out that our XLA code is not behaving properly for T5, for both `sample` and `greedy_search`. Because the problem is not exclusive to `sample`, I'm merging this PR and fixing the issue in a future one.

(example)
![image](https://user-images.githubusercontent.com/12240844/163792042-d23d01be-c2c9-40d4-bee2-30ea32c47a45.png)
Pin Jax to last working release
Author: sgugger
Pull request: yes

# What does this PR do?

Jax maintainers has apparently no problem breaking the `optax` library and every library depending on it. So this PR pins Jax and jaxlib for now until this gets sorted out.

cc @patil-suraj @patrickvonplaten for information, will merge as soon as CI is green (or only 500 errors coming from the Hub).

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thank you for fixing this @sgugger !
Raise error and suggestion when using custom optimizer with Fairscale or Deepspeed
Author: allanj
Pull request: yes

# What does this PR do?

1st Issues: OOM when saving the optimizer, https://github.com/huggingface/transformers/issues/14542
This issue happens in consolidating the optimizer, we add an argument `save_optimizer_state` to give an option on whether we want to save it.

2nd issue: Using a custom optimizer has a problem with Fairscale and Deepspeed, https://github.com/huggingface/transformers/issues/15784

We simply raise an error and warn the user to proceed with different solution.

Fixes # (issue)


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?


trainer: @sgugger


Comments:
I'm not really in favor of adding an argument to not save the optimizer as there is no point checkpointing if the optimizer is not saved. For the fairscale problem of OOM, there is an option that was detailed in #14542 to use ` force_broadcast_object=True` with the newest version of fairscale.

Oh, I just saw that solution. Maybe the second part where we raise the error is enough for this PR? Do you think it is necessary
```python3
if (self.sharded_ddp is not None or args.deepspeed) and (self.optimizer is not None or self.lr_scheduler is not None):
  raise RuntimeError(
      "Passing `optimizers` is not allowed if Fairscale or Deepspeed is enabled."
      "You should subclass `Trainer` and override the `create_optimizer_and_scheduler` method."
  )
```

This change I agree with :-) If you want to remove the others, we can merge the PR.
Make sure to run `make style` after so that the quality check passes.

_The documentation is not available anymore as the PR was closed or merged._

Yeah, I will remove the others and make a check

There is still one error of quality script, so you'll need to run `make style` on your branch :-)
Could you also rename the PR so that the title reflects the changes you actually did?

Thanks! You have to mark the PR as ready for review, as GitHub won't let me merge a draft PR :-)

Thanks. It is ready for review now.
[ViT, BEiT, DeiT, DPT] Improve code
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR cleans up some code of ViT, BEiT, DeiT and DPT.

Most importantly, it corrects the tuple outputs, in case no pooler was added (as shown by #16760).

Next to that, it cleans up the tests, by
- removing `to_2tuple`, and instead leveraging an `expected_seq_len` attribute of the `ModelTester`
-  removing `chunk_length` and `is_encoder_decoder` statements, which don't apply for these encoder-only models.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Any reasons not using `processors` or `pipelines` modules in example codes?
Author: nbqu
Pull request: no

Hi, I'm wondering that is there any reasons, or purposes not using `processors` or `pipelines` modules in example codes, such as [`run_glue.py`](https://github.com/huggingface/transformers/blob/master/examples/pytorch/text-classification/run_glue.py) or [`run_qa.py`](https://github.com/huggingface/transformers/blob/master/examples/pytorch/question-answering/run_qa.py).

First, I strongly appreciate for providing example codes, which is very helpful to understand not only `transformers` library, but also entire workflows of NLP. Moreover, example codes could introduce useful APIs in `transformers` to beginners like me.

So I suggest that it would be great to apply such APIs to example codes, unless there are specific reasons not using them.

Comments:
Hey @nbqu, `processors` are preprocessing components that are specifically used for complex situations where a feature extractor and a tokenizer are needed for preprocessing. Examples which require them therefore have them, for example see here: https://github.com/huggingface/transformers/blob/5664d2762260ddb9e01f0566ddd43cafd5cdd807/examples/pytorch/speech-recognition/run_speech_recognition_ctc.py#L270

Pipelines, on the other hand, support inference rather than training. The examples under the `examples` directory are for training specifically, so there's no need to leverage pipelines there.

However, if you take a look at your [documentation](https://huggingface.co/docs/transformers/quicktour), you'll see that pipelines are mentioned!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.