noamaanMulla-03/transformers-issues huggingface dataset:
[Time-Series] informer model
Author: elisim
Pull request: yes

# What does this PR do?

Adding Time Series Informer model https://arxiv.org/abs/2012.07436

Related issue: #20903 

@kashif :) 

Comments:
very cool! Having a look shortly!

> very cool! Having a look shortly!

Wow you saw it fast! right now it's just the template of the vanilla TS transformer. 

BTW I sent you an email :)

Hi, @NielsRogge  and @kashif 🙂

Maybe you have an example for a conversion script?
I'm following the [How to add a model to 🤗 Transformers?](https://huggingface.co/docs/transformers/add_new_model), section six "Write a conversion script":

> Don’t hesitate to ask the Hugging Face team to point you to a similar already existing conversion script for your model.

Thank you so much,
Eli

thanks! having a look!

> thanks! having a look!

Work is still in progress, but you might have a look if you have time :) 
And by the way, your implemention and the vanilla TS are helping me a lot!

Hi @kashif, I fixed the final attention output of ProbSparseAttention,  and added the ProbMask. In more detail:

# Major
1. Added calculation of the final `attn_output` using `v_aggregated`, meaning steps 7 & 8 in the following:

<img width="446" alt="image" src="https://user-images.githubusercontent.com/17675462/218316971-ce4dbe9e-d677-48e6-9184-0043ca179e6a.png">

**Reference:** [Informer paper](https://arxiv.org/abs/2012.07436), **Section:** "Implement of the ProbSparse self-attention"

2. Added ProbMask, function name `_prepare_decoder_prob_attention_mask`.


# Minor 
1. Comment-in attention dropout in `ProbSparseAttention` since the original impl didn't use it.
2. Removed `attention_mask` for the encoder, since the original impl don't apply it to the encoder, only for the decoder.
3. Added `self.attn = config.attn` in the decoder, to later check if to create ProbMask or the standard casual mask.
4. Removed unused code from the original impl.

Now the tests are falling mostly because assertion errors. Before continuing to fix them, I would appreciate if you can have a look :)

Thanks,
Eli

_The documentation is not available anymore as the PR was closed or merged._

thanks @sgugger will get it fixed!

@kashif fixed what I could from Sylvain comments.
The main thing is that some tests are breaking after this fix https://github.com/huggingface/transformers/pull/21099/commits/b4cbddfa05e3bd739b79569cd3c3b89e316f2451


@sgugger I have fixed all the assert issues in the PR #21846 and will fix the copies here when that gets merged
Update tiny model creation script and some others files
Author: ydshieh
Pull request: yes

# What does this PR do?

The original goal is to update tiny model creation script, so we can create tiny models for some (newly added) model classes. It turns out some files are needed to be updated too. See my own review comments.

Note: This PR doesn't imply we are able to create tiny models for all involved model classes in this PR. Some model classes require more work to be done (`speecht5, tvlt` for example), but let me do it in separate PR(s).

Comments:
_The documentation is not available anymore as the PR was closed or merged._
add 1 to cur_len to make up the new beam length
Author: jimmieliu
Pull request: yes

# What does this PR do?
cur_len is 1 token shorter comparing to the length of the sequence whose best_sum_logprobs is the numerator.

Fixes # (issue)
add 1 to cur_len

## Who can review?
@LysandreJik 



Comments:
_The documentation is not available anymore as the PR was closed or merged._

cc @gante

@gante Thx for the great advice. You are suggesting a better coding style.
VideoMAE doctest - use valid dummy pixel values
Author: amyeroberts
Pull request: yes

# What does this PR do?

This PR updates the raw input video frames to the expected data type. 

The pixel values passed into the image processor in the tests took values sampled from a standard normal distribution. For an image (or frame) this represents pixels which have been rescaled between [0 - 1] and normalized i.e. one which has already been passed to the image processor. 

After merging #21969, resizing the image throws an error as this image cannot be converted to a PIL.Image.Image without possibly unexpected behaviour from numpy and overflow issues. 


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

Comments:
_The documentation is not available anymore as the PR was closed or merged._
update: bertology paper
Author: QiushiSun
Pull request: yes

# What does this PR do?

Add additional reference papers for the documentation of BERTology.

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @sgugger

Integrations:

- deepspeed: HF Trainer: @stas00, Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam

Documentation: @sgugger, @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: @sgugger
- TensorFlow: @Rocketknight1

 -->


Comments:
_The documentation is not available anymore as the PR was closed or merged._
[`bnb`] Fix bnb error message
Author: younesbelkada
Pull request: yes

# What does this PR do?

Fixes https://github.com/huggingface/transformers/issues/22018

This PR introduces a clearer error message to users who wants to explore how to dispatch a model between CPU & GPU when loading a model in 8bit

cc @sgugger 


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Avoid `text_config_dict` and `vision_config_dict` being saved  for CLIP-like models
Author: ydshieh
Pull request: yes

# What does this PR do?

**Avoid `text_config_dict` and `vision_config_dict` being saved  for CLIP-like models.** So less confusion.

Currently, configuration classes for CLIP-like models will save both `text_config` and `text_config_dict`, if `text_config_dict` is provided (as `kwargs`). Similarly, for  `vision_config` and `vision_config_dict`. Many configuration files on the Hub have all these keys, and they look really confusing, see for example [openai/clip-vit-base-patch16](https://huggingface.co/openai/clip-vit-base-patch16/blob/main/config.json) or [laion/CLIP-ViT-H-14-laion2B-s32B-b79K](https://huggingface.co/laion/CLIP-ViT-H-14-laion2B-s32B-b79K/blob/main/config.json#L115)

This issue dates back before the PR #19954. The PR #19954 tried to avoid the usage of `text_config_dict` and `vision_config_dict` (while keeping back compatibility), it didn't prevent `text_config_dict` and `vision_config_dict` being saved.

This PR:
  -  avoid `text_config_dict` and `vision_config_dict` being saved,
  - make sure all the values provided in `text_config_dict` and `vision_config_dict` will be used to update `text_config` and `vision_config` (so backward compatibility), and only `text_config` and `vision_config` are saved
  - **therefore, we can load a existing configuration, save it and upload it again --> make it clean + less confusing, and not break anything**

 I will apply the same change to other CLIP-like models if the idea/approach is accepted.

Comments:
> Thanks! This may need to be copied over some CLIP-like models that also have some backward-compatibility code with the config dicts.

Sure, in the plan already. Quoted in the PR description

> I will apply the same change to other CLIP-like models if the idea/approach is accepted.

_The documentation is not available anymore as the PR was closed or merged._
Bug fix: token classification pipeline while passing offset_mapping
Author: cceyda
Pull request: yes

# What does this PR do?
Bug fix: add check so `AttributeError` isn't preventing using slow tokenizers with `offset_mapping`

On token-classification pipelines it threw an error (AttributeError None) if using a slow tokenizer & passing `offset_mapping`.

It is intended to work so (if you want) you can calculate offsets yourself while using a slow(or custom) tokenizer. otherwise "start"&"end" values returned from the pipeline are `None`

For example 'google/canine-c' (pretend it is finetuned)

```python
from transformers import pipeline

token_classifier = pipeline(
        "token-classification", model='google/canine-c',
        aggregation_strategy="simple", ignore_labels=[],
    )
offset_mapping=[(0,0)]+[(i,i+1) for i,t in enumerate(text)]+[(0,0)] # canine is an easy enough tokenizer to calculate offsets ourselves accounting for [cls],[sep].
ents=token_classifier(text)
print(ents) # without offset_mapping "start"&"end" is None
ents=token_classifier(text, offset_mapping=offset_mapping)
print(ents) # should return entities with "start"&"end" index values. 
```


## Before submitting
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. 

- pipelines: @Narsil


Comments:
_The documentation is not available anymore as the PR was closed or merged._

the current failing tests look unrelated to me 


LGTM @sgugger 

I'm confused by the failure, I'm guessing it's CircleCI running the wrong runner, but I don't remember the fix.
GitHub Private Vulnerability reporting 
Author: Sim4n6
Pull request: no

### Feature request

Enable Private vulnerability reporting in the GitHub repository, please.

https://docs.github.com/en/code-security/security-advisories/repository-security-advisories/configuring-private-vulnerability-reporting-for-a-repository

### Motivation

I may have identified something low-impact vulnerability, but I would like to report it privately via the usual channel and request a CVE in that case.

### Your contribution

The report that I will submit, when the feature is enabled, please.

Comments:
cc @Michellehbn 

Hi @Sim4n6, Thanks for reaching out to us! 🤗 We have a bug bounty program with HackerOne and would love for you to submit security vulnerability reports to https://hackerone.com/hugging_face. This is a private program so we will need to invite you. Do you happen to have an H1 username? Or you can send security@huggingface.co an email and we'll send you an invite! 

it is [Sim4n6](https://hackerone.com/sim4n6?type=user). No problem.

Invite sent! Thanks again! 

Done.
Docs Improvement - In ZSH, not using ' ' around pip install fails, fix it
Author: shaun-scale
Pull request: yes

# What does this PR do?

Running 
```
pip install transformers[torch]
```
in the default ZSH terminal will fail with the error `zsh: no matches found: transformers[torch]`

The solution is to wrap the installation path in ' ' like 
```
pip install 'transformers[torch]'
```

Relevant StackOverflow: https://stackoverflow.com/questions/30539798/zsh-no-matches-found-requestssecurity

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@sgugger, @stevhliu and @MKhalusova

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Can't install tf2 on M1 Chip by default
Author: shaun-scale
Pull request: yes

# What does this PR do?

Trying to 
```
pip install 'transformers[tf-cpu]'
```

will give you a confusing error like below
```
Collecting sentencepiece==0.1.91
  Using cached sentencepiece-0.1.91.tar.gz (500 kB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [5 lines of output]
      Package sentencepiece was not found in the pkg-config search path.
      Perhaps you should add the directory containing `sentencepiece.pc'
      to the PKG_CONFIG_PATH environment variable
      No package 'sentencepiece' found
      Failed to find sentencepiece pkgconfig
      [end of output]

```

The answer is to install `cmake` and `pkg-config` based on the reply here: 
https://github.com/google/sentencepiece/issues/378#issuecomment-969896519

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@sgugger, @stevhliu and @MKhalusova

Comments:
This actually got worse... if you open a new Laptop, you also have to do the following
```
brew install cmake
brew install pkg-config
brew install sentencepiece
pip install sentencepiece
```

and then I had to also install Rust next...
```
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source "$HOME/.cargo/env"
```

After that, it finally works
```
pip install 'transformers[tf-cpu]'
```

Not sure how thorough we want to be in the docs of getting people fully up to speed vs. making certain assumptions. The `sentencepiece` part was fairly brutal to work through

_The documentation is not available anymore as the PR was closed or merged._
Remove set_access_token usage + fail tests if FutureWarning
Author: Wauplin
Pull request: yes

`set_access_token` is deprecated and will be removed in `huggingface_hub>=0.14`.

This PR removes it from the tests (it was not used in `transformers` source code itself). In the future, use `set_git_credential` if needed. It is a git-credential-agnostic helper, i.e. you can store your git token in `git-credential-cache`, `git-credential-store`, `osxkeychain`, etc. The legacy `set_access_token` was only able to set in `git-credential-store` no matter the user preference.

(for context, I found out about this while working on https://github.com/huggingface/huggingface_hub/pull/1381)

---

In addition to this, I have added

```
filterwarnings =
    error::FutureWarning:huggingface_hub*
```

to the `setup.cfg` config file to fail on future warnings from `huggingface_hub`. In `hfh`'s CI we trigger on FutureWarning from any package but it's less robust (any package update leads can lead to a failure). No obligation to keep it like that (I can remove it if you prefer) but I think it's a good idea in order to track future FutureWarnings.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Show the number of `huggingface_hub` warnings in CI report
Author: ydshieh
Pull request: yes

# What does this PR do?

Show the number of `huggingface_hub` warnings in CI report, as discussed in #22051

Will be shown like below

<img width="488" alt="Screenshot 2023-03-09 134910" src="https://user-images.githubusercontent.com/2521628/224050100-90c1477a-c33f-485e-85e1-ec648cbb4f91.png">


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Trainer hyperparameter_search only returns the best trial config
Author: anruijian
Pull request: no

### Feature request

Allow `hyperparameter_search` method of `Trainer` to return the entire `ExperimentAnalysis` object instead of  a single `best_run`.

### Motivation

The `hyperparameter_search` method of the `Trainer` currently only returns the best configuration `best_run`, instead of the more comprehensive `ExperimentAnalysis` object `analysis`. However, I believe that `analysis` would be more valuable than just the single best run configuration since it offers additional attributes and methods that can provide more useful information about tuning (see [doc](https://docs.ray.io/en/releases-1.11.0/tune/api_docs/analysis.html#analysis-tune-analysis)). Therefore, I suggest modifying the`hyperparameter_search` method to return the `ExperimentAnalysis` object so that users can do more analysis.

```python
    analysis = ray.tune.run(
        dynamic_modules_import_trainable,
        config=trainer.hp_space(None),
        num_samples=n_trials,
        **kwargs,
    )
    best_trial = analysis.get_best_trial(metric="objective", mode=direction[:3], scope=trainer.args.ray_scope)
    best_run = BestRun(best_trial.trial_id, best_trial.last_result["objective"], best_trial.config)
    if _tb_writer is not None:
        trainer.add_callback(_tb_writer)
    return best_run
```

### Your contribution

I can submit a PR for this feature. 

Comments:
We'd be happy to look at a PR!

@sgugger I have a concern regarding the different backends we use (ray, optuna, sigopt, wandb) and their varying return objects. I wonder if we should consider modifying all backends to return a more comprehensive object, such as the `analysis` object used in ray, to ensure consistency across all the backends. 

While I am familiar with the ray tune backend, I am unsure about how to proceed with the other backends. I checked the code briefly to find the object that acts as `analysis` for ray:

1. [study](https://github.com/huggingface/transformers/blob/main/src/transformers/integrations.py#L196) for optuna
2. [entire list of experiments](https://github.com/huggingface/transformers/blob/main/src/transformers/integrations.py#L433) for sigopt
3. [dictionary](https://github.com/huggingface/transformers/blob/main/src/transformers/integrations.py#L433) for wandb (need to modify the dictionary to record results of all experiments instead of the best one.)

Let me know if I understand it correctly.

I think it's okay if the object is backend specific.

PR #22040 submitted.
pt-to-tf model architecture override
Author: Rocketknight1
Pull request: yes

This PR adds an extra arg to the `pt-to-tf` conversion script. We've seen a few uploaded models where the `config.json` doesn't specify the model class and the script autodetects the wrong one, which means some weights are not converted. This argument lets you override the autodetection and specify a model class to use for the conversion.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
[deepspeed] offload + non-cpuadam optimizer exception
Author: stas00
Pull request: yes

adapting to https://github.com/microsoft/DeepSpeed/pull/2971 - as our deepspeed tests will fail without that new flag when that deepspeed PR will get merged.

Will add the new config `zero_force_ds_cpu_optimizer` to the integration docs and require `deepspeed>=0.8.3`, but can't do it here w/o breaking DS' CI.  Will do it here post new release https://github.com/huggingface/transformers/pull/22044

@jeffra 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Add setters by type of args to TrainingArguments
Author: sgugger
Pull request: yes

# What does this PR do?

This PR introduces setters that group some of the `TrainingArguments` by type, to make this class easier to use. This was suggested in #20831 

If the idea is of interest, I'll create more of those to cover all training arguments.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@MKhalusova this is a way to split the documentation of training arguments (currently 96 :grimacing: ) by splitting them in groups, could you have a quick look?

I really like the idea of splitting 96 arguments into reasonable chunks. Looking at the examples, I am not sure how it would look when you use more than one group. 
Is it going to look something like this? 

```
args = TrainingArguments("working_dir")
args = args.set_training(learning_rate=1e-4, batch_size=32)
args = args.set_logging(strategy="steps", steps=100)
args = args.set_evaluate(strategy="steps", steps=100)
```


Yup, or chained directly if that's more your style:
```py
args = (
    TrainingArguments("working_dir")
    .set_training(learning_rate=1e-4, batch_size=32)
    .set_logging(strategy="steps", steps=100)
    .set_evaluate(strategy="steps", steps=100)
)
```

I think this should make the docs much more readable and might make the code more readable as well.