noamaanMulla-03/transformers-issues huggingface dataset:
added type hints for BART model
Author: robotjellyzone
Pull request: yes

# What does this PR do?

I have added type hints for BART model (both in PyTorch and Tensorflow) as described in #16059 
<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

@Rocketknight1 

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Comments:
@Rocketknight1 here is the PR which i've created since i am on Windows so i didnt run ``make fixup`` command

Hi @robotjellyzone, thanks for the PR! I see a lot of tests failing but I think that's our fault, not yours. Let me try running `make fixup` and then rerun them, and hopefully it all passes

_The documentation is not available anymore as the PR was closed or merged._

@robotjellyzone Tests pass now! Will merge and then update the other PRs depending on these copied models.
[modeling_utils] use less cpu memory with sharded checkpoint loading
Author: stas00
Pull request: yes

This PR lowers the peak cpu memory usage for sharded checkpoint loading

The following demonstration tells the full story. I'm using `/usr/bin/time -f %M` to report max rss = total cpu memory used by the process including peak memory.

This demo uses T0 which is 42GB big in fp32 https://huggingface.co/bigscience/T0/tree/main

So with the normal loading the program needs 87GB of CPU RAM (42x2 plus a few GBs for temps)

```
# full checkpoint
/usr/bin/time -f %M python -c "from transformers import AutoModelForSeq2SeqLM; \
model = AutoModelForSeq2SeqLM.from_pretrained('bigscience/T0')"
87286376

# shard it to 10GB / shard
python -c "from transformers import AutoModelForSeq2SeqLM; \
model = AutoModelForSeq2SeqLM.from_pretrained('bigscience/T0'); \
model.save_pretrained('t0-sharded')"

# before this PR
/usr/bin/time -f %M python -c "from transformers import AutoModelForSeq2SeqLM; \
model = AutoModelForSeq2SeqLM.from_pretrained('t0-sharded')"
68358000

# after this PR
/usr/bin/time -f %M python -c "from transformers import AutoModelForSeq2SeqLM; \
model = AutoModelForSeq2SeqLM.from_pretrained('t0-sharded')"
53529416
```

So after this PR the CPU memory usage is 1x model size (42GB here) + largest shard (10GB) + some temps = 53GB

Before this PR we were getting an additional 15GB (1.5x shard) of peak cpu memory.

@sgugger

Comments:
_The documentation is not available anymore as the PR was closed or merged._
cannot import name 'Text2TextGenerationPipeline
Author: aa327chenge
Pull request: no

In transformers 4.17，It will find error like 
："cannot import name 'Text2TextGenerationPipeline'"

My code is  "from transformers import Text2TextGenerationPipeline"

Comments:
The recommended way to load the `Text2TextGenerationPipeline` as stated in the [docs](https://huggingface.co/docs/transformers/v4.17.0/en/main_classes/pipelines#transformers.Text2TextGenerationPipeline) is:

```python
from transformers import pipeline
text2text = pipeline("text2text-generation") 
```

But if you must import the class you can import it as such:
```python
from transformers.pipelines.text2text_generation import Text2TextGenerationPipeline
```

If you're just trying to use the pipeline and not making something like a feature contribution for the class, you should use the first method to load the pipeline.

```python
class Text2TextGenerationPipeline(Pipeline):
    """
    Pipeline for text to text generation using seq2seq models.
    This Text2TextGenerationPipeline pipeline can currently be loaded from [`pipeline`] using the following task
    identifier: `"text2text-generation"`.
    The models that this pipeline can use are models that have been fine-tuned on a translation task. See the
    up-to-date list of available models on
    [huggingface.co/models](https://huggingface.co/models?filter=text2text-generation).
    Usage:
    ```python
    text2text_generator = pipeline("text2text-generation")
    text2text_generator("question: What is 42 ? context: 42 is the answer to life, the universe and everything")
    ```"""

    # Used in the return key of the pipeline.
    return_name = "generated"

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)

        self.check_model_type(
            TF_MODEL_FOR_SEQ_TO_SEQ_CAUSAL_LM_MAPPING
            if self.framework == "tf"
            else MODEL_FOR_SEQ_TO_SEQ_CAUSAL_LM_MAPPING
        )
```


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Changing the default branch from `master` to `main`.
Author: LysandreJik
Pull request: no

Over the next 24 hours, we'll change the default branch from `master` to `main`.

We'll leverage GitHub's renaming utility which should take care of the entire migration and handle redirection when the previous branch name was erroneously entered.

This will not update local environments, where running the following commands the first time will be necessary (will be mentioned on GitHub directly):

```bash
git branch -m master main
git fetch origin
git branch -u origin/main main
git remote set-head origin -a
```

Thanks for your understanding!

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

closing as completed
Docs link to deepspeed infinity redirect to a 404
Author: osanseviero
Pull request: no
Labels: name: bug

### System Info

```shell
Docs related
```


### Who can help?

@stas00 @sgugger 

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1. Go to In https://huggingface.co/docs/transformers/performance
2. Click link with "DeepSpeed-Infinity" text
3. Get site that says "The documentation page DEEPSPEED doesn’t exist in v4.18.0, but exists on the main version. Click here to redirect to the main version of the documentation."
4. Redirects to https://huggingface.co/docs/transformers/main/en/deepspeed, which is 404

### Expected behavior

```shell
Ideally this should not link to an error.
```


Comments:
Thanks a lot for the report, Omar - fixed here: https://github.com/huggingface/transformers/pull/16860
Fix multiproc metrics in no_trainer examples
Author: muellerzr
Pull request: yes
Labels: name: Examples

name: bug

# Fixes a bug in the `no_trainer` scripts involving metric evaluation

## What does this add?

Fixes up the metric evaluation in most of the `no_trainer` scripts when multi-processing is involved. The final batch gets duplicated in those situations, so the metrics are slightly higher (or lower) than their actual values.

## Who is it for?

Users of the `no_trainer` scripts

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Is there a way to evaluate GPT-2 model during fine-tuning process for accuracy and fluency?
Author: D-i-l-r-u-k-s-h-i
Pull request: no

# ❓ Questions & Help

I'm trying to evaluate GPT-2 model during fine tuning  process, and I'm able to calculate the loss at each epoch, but do not know how accuracy can be calculated or how to give a score to the model. Would like to get some suggestions as help.
## Details
<!-- Description of your issue -->

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**: https://stackoverflow.com/questions/60483956/how-to-perform-accuracy-testing-on-text-generation-task

Comments:
A common way of evaluating LMs is to measure their Perplexity.  
Say you want to finetune GPT2 on your dataset D.
Define train, val and test datasets (maybe something around 75%, 10%, 15%.
Measure the [perplexity](https://towardsdatascience.com/perplexity-intuition-and-derivation-105dd481c8f3) on train and val after each epoch. Compare train and eval curves for overfitting. 

There are a ton of other evaluation measures that might be better for your task - Google will be your best friend :-) 

@patrickvonplaten can you provide example/code implementation?
Support on Mixture of expert models
Author: fym0503
Pull request: no
Labels: name: New model

Hi, I find that there are emerging works in the field of NLP on **Mixture of experts** based models, such as Switch Transformers from Google. However, I do not find such mixture of expert models in huggingface transformers. Do you have the plan to support such models? Thanks !

Comments:
Hi,

It's true, but as long as there are no pretrained weights, chances are small models are added. There are some open-source implementations of MoE available, including:

- DeepSpeed: https://www.deepspeed.ai/tutorials/mixture-of-experts/
- Fairseq.

Once there are some pretrained weights available somewhere, be sure to let us know!

Hi, I find there is a very recent implementation of MoE with code and pretrained weights for your reference
https://github.com/pytorch/fairseq/tree/main/examples/moe_lm

Indeed, hot of the press! Let's add them!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

Any progress on this front ? Thanks...

cc'ing @patil-suraj here

Hey @cerisara ! We have planned to add the `moe_lm` in `Transformers` but I don't have much bandwidth to work on it.  If you or anyone else in the community is interested in adding it, I would be more than happy to help :) 

Hi @patil-suraj, thanks, I'm interested in these models and would like to contribute, but I'm afraid my bandwidth is too small as well, at least for now, sorry ;-)
tiny tweak to allow BatchEncoding.token_to_char when token doesn't correspond to chars
Author: garyhlai
Pull request: yes

Tagging @n1t0 @thomwolf @sgugger but this PR should be extremely quick to review for anyone.

# Problem

`BatchEncoding.token_to_char` is supposed to return the char spans in the original string; however, right now, for tokens such as "\
, \
, \<CLS>" that don't correspond to any chars in the original string, an error is raised `TypeError: type object argument after * must be an iterable, not NoneType`. 

Run the following snippet replicate: 
```
from transformers import AutoTokenizer, AutoModel

model_name = "bert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModel.from_pretrained(model_name)
text = "He is an absolutely amazing software developer"
tokenized_text = tokenizer(text)
tokenized_text.token_to_chars(0) # 0 corresponds to <CLS>
```


# Fix
The solution is to return `None` instead of raising an error for tokens not corresponding to any chars in the original string.


## P.S.

I am lost as to why `run_tests_torch` failed for `
if [ -f test_list.txt ]; then python -m pytest -n 3 --dist=loadfile -s --make-reports=tests_torch $(cat test_list.txt) | tee tests_output.txt fi`. Some help would be appreciated.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Can I follow up on this? Would love to get some feedback -- even if it's about the specific reasons why this PR is not up to  standard. Thanks!

Sorry for taking so long to review, could you just rebase on `main` so that the test suite passes? 

cc @SaulLu for knowledge

Done! @sgugger @LysandreJik 


Thanks again for your contribution!
Update for bs_msfp integration
Author: kmfnine
Pull request: yes

Some modification for calling Brainwave encoder library. 

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Hello! Sorry for not seeing this earlier. In the README of the examples folder, there is written the following:

> While we strive to present as many use cases as possible, the example scripts are just that - examples. It is expected that they won't work out-of-the box on your specific problem and that you will be required to change a few lines of code to adapt them to your needs. To help you with that, most of the examples fully expose the preprocessing of the data, allowing you to tweak and edit them as required.

To that end, we don't intend  to add additional features to existing scripts. However, feel free to upload a gist somewhere with the code that you have and share it on the [forum](https://discuss.huggingface.co), I'm sure some users would love to see it!

Thanks!
ViLT vs VIT Classifier heads question
Author: PrithivirajDamodaran
Pull request: no

Is there any specific reason for the classifier head on the ViLT model for tasks say ```ViltForImagesAndTextClassification```  or ```ViltForQuestionAnswering ``` has a ```LayerNorm and GELU``` and not just a Linear Input and output Layers  (Like below) whereas classifier head on the VIT for ```ViTForImageClassification``` Has only a linear layer ?

Please advice.

i.e This 
```python
# Classifier head
  self.classifier = nn.Sequential(
      nn.Linear(config.hidden_size, config.hidden_size * 2),
      nn.LayerNorm(config.hidden_size * 2),
      nn.GELU(),
      nn.Linear(config.hidden_size * 2, config.num_labels),
  )
```
and NOT this ?

```python
# Classifier head
 self.classifier = nn.Linear(config.hidden_size, config.num_labels) if config.num_labels > 0 else nn.Identity()
```



Comments:
Hi, thanks for your interest in ViLT. 

That was the decision of the authors. Maybe you can ask them 😉 

Ok :-)
The bare ViLT Model not working with CUDA ?
Author: PrithivirajDamodaran
Pull request: no


@NielsRogge  - Tried in the stock Colab notebook with the latest version of Transformers. Probably the ```to(device)``` is not handled uniformly for Text Embeddings or Image Embeddings?

What I am trying to do: I need to extract the raw hidden_state or pooler_ouput for large batches of image + text pairs. Use it as a feature for training my custom model.

```python
from transformers import ViltProcessor, ViltModel
import torch

device = "cuda" if torch.cuda.is_available() else "cpu"
vilt_model = ViltModel.from_pretrained("dandelin/vilt-b32-mlm")
vilt_processor = ViltProcessor.from_pretrained("dandelin/vilt-b32-mlm")
vilt_model.to(device)  # Added this ? As it is slow in CPU

from PIL import Image
import requests

url = "http://images.cocodataset.org/val2017/000000039769.jpg"
image = Image.open(requests.get(url, stream=True).raw)
text = "hello world"
inputs = vilt_processor(image, text, return_tensors="pt")
outputs = vilt_model(**inputs)

```

Throws

```
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (when checking argument for argument index in method wrapper__index_select)
```


Comments:
Hey! You're getting this error because your model is on GPU while your inputs are not. Could you try casting your inputs to GPU?

```py
inputs = {k:v.to('cuda') for k,v in inputs.items()}
```

Thanks, I was assuming ViltProcessor will do it for me. Closing
ValueError: DebertaV2Model does not support gradient checkpointing.
Author: shilida
Pull request: no
Labels: name: New model

ValueError: DebertaV2Model does not support gradient checkpointing!

Comments:
Hi,

DebertaV2 supports gradient checkpointing as per #14175.

Can you provide a code snippet to reproduce your issue?

`config = AutoConfig.from_pretrained(model_name)

        config.update(
            {
                "gradient_checkpointing":True,
                "output_hidden_states": True,
                "hidden_dropout_prob": hidden_dropout_prob,
                "layer_norm_eps": layer_norm_eps,
                "add_pooling_layer": False,
                "num_labels": self.num_labels,
            }
        )
        self.transformer = AutoModel.from_pretrained(model_name, config=config)`
Hello, the above is my code snippet

I tried to reproduce the error, but I couldn't. @shilida would you please mention the `model_name` and which transformer version you have currently installed? 

> I tried to reproduce the error, but I couldn't. @shilida would you please mention the `model_name` and which transformer version you have currently installed?

Thanks! The model_name is  debert-v2-xlarge. And the transformer version is 4.11.3

You should try something like that (worked for me with debertaV3):

```
import torch.utils.checkpoint

config = AutoConfig.from_pretrained(model_name)
config.update(
        {
            "output_hidden_states": True,
            "hidden_dropout_prob": hidden_dropout_prob,
            "layer_norm_eps": layer_norm_eps,
            "add_pooling_layer": False,
            "num_labels": self.num_labels,
        }
)
self.transformer = AutoModel.from_pretrained(model_name, config=config)
self.transformer.gradient_checkpointing_enable()
```
CI: Create empty venv on cache miss
Author: gante
Pull request: yes

# What does this PR do?

Fixes a CI error introduced by https://github.com/huggingface/transformers/pull/16789 -- when there was a cache miss, it attempted to initialize a virtual environment that didn't exist, in two CI workflows.

This PR fixes that by creating empty venvs on a cache miss.

(I wonder why the original CI run inside the PR didn't fail, the new cache names should have triggered the same error 🤔 )

Comments:
_The documentation is not available anymore as the PR was closed or merged._
CI: pip install now updates
Author: gante
Pull request: yes

# What does this PR do?

Follow up from https://github.com/huggingface/transformers/pull/16751: `pip install` in non-remote GHA workflows now updates packages, to allow us to override pre-installed versions.

Comments:
This PR actually needs further changes, the adding `-U` doesn't solve it (possibly because of other version requirements in the packages installed in the image). 

Should I update the dependencies of click (to `click>=8.0`, required by black) and protobuf (to `protobuf>=3.8.0`, required by tensorflow)? cc @LysandreJik @sgugger 

_The documentation is not available anymore as the PR was closed or merged._

It would be nice to find a general solution that won't have us needing to add a new dependency update in three weeks.

(closing the PR after some offline discussion -- going to attempt to change to cache fresh venvs instead)
Fix doctest list
Author: ydshieh
Pull request: yes

# What does this PR do?

We have `docs/source/en/model_doc/t5v1_1.mdx` in `documentation_tests.txt`, but the file is actually `docs/source/en/model_doc/t5v1.1.mdx`.

This cause doctest fail.

```
ERROR: file or directory not found: docs/source/en/model_doc/t5v1_1.mdx
```
(https://github.com/huggingface/transformers/runs/6104280137?check_suite_focus=true)

Comments:
_The documentation is not available anymore as the PR was closed or merged._
New features for CodeParrot training script
Author: loubnabnl
Pull request: yes

This PR adds some features to CodeParrot training script.

- Add TFLOPS to logging
- Use Accelerate checkpointing and tracking for Wandb and Tensorborad
- Fix gradient accumulation for DDP (https://github.com/huggingface/accelerate/pull/106)
- Scale loss approprietly for the last batch
- Fix typo in the README

cc @lvwerra @LysandreJik 

Comments:
_The documentation is not available anymore as the PR was closed or merged._