noamaanMulla-03/transformers-issues huggingface dataset:
[i18n-<languageCode>] Translating docs to <languageName>
Author: sachiweb
Pull request: no
Labels: name: WIP

<!--
Note: Please search to see if an issue already exists for the language you are trying to translate.
-->

Hi!

Let's bring the documentation to all the <languageName>-speaking community 🌐 (currently 0 out of 267 complete)

Who would want to translate? Please follow the 🤗 [TRANSLATING guide](https://github.com/huggingface/transformers/blob/main/docs/TRANSLATING.md). Here is a list of the files ready for translation. Let us know in this issue if you'd like to translate any, and we'll add your name to the list.

Some notes:

* Please translate using an informal tone (imagine you are talking with a friend about transformers 🤗).
* Please translate in a gender-neutral way.
* Add your translations to the folder called `<languageCode>` inside the [source folder](https://github.com/huggingface/transformers/tree/main/docs/source).
* Register your translation in `<languageCode>/_toctree.yml`; please follow the order of the [English version](https://github.com/huggingface/transformers/blob/main/docs/source/en/_toctree.yml).
* Once you're finished, open a pull request and tag this issue by including #issue-number in the description, where issue-number is the number of this issue. Please ping @ArthurZucker, @sgugger for review.
* 🙋 If you'd like others to help you with the translation, you can also post in the 🤗 [forums](https://discuss.huggingface.co/).

## Get Started section

- [ ] [index.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/index.mdx) https://github.com/huggingface/transformers/pull/20180
- [ ] [quicktour.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/quicktour.mdx) (waiting for initial PR to go through)
- [ ] [installation.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/installation.mdx).

## Tutorial section
- [ ] [pipeline_tutorial.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/pipeline_tutorial.mdx)
- [ ]  [autoclass_tutorial.mdx](https://github.com/huggingface/transformers/blob/master/docs/source/autoclass_tutorial.mdx)
- [ ]  [preprocessing.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/preprocessing.mdx)
- [ ]  [training.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/training.mdx)
- [ ]  [accelerate.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/accelerate.mdx)
- [ ]  [model_sharing.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/model_sharing.mdx)
- [ ]  [multilingual.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/multilingual.mdx)

<!--
Keep on adding more as you go 🔥
-->


Comments:
Tokenizer call function gives an error when using the "target_text "argument without using "text" argument. 
Author: shamanez
Pull request: no

### System Info

Name: transformers
Version: 4.21.2
Summary: State-of-the-art Machine Learning for JAX, PyTorch and TensorFlow
Home-page: https://github.com/huggingface/transformers
Author: The Hugging Face team (past and future) with the help of all our contributors (https://github.com/huggingface/transformers/graphs/contributors)
Author-email: transformers@huggingface.co
License: Apache
Location: /databricks/python3/lib/python3.9/site-packages
Requires: packaging, pyyaml, filelock, numpy, regex, tokenizers, tqdm, huggingface-hub, requests

@ArthurZucker

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

1. tokenizer_label = AutoTokenizer.from_pretrained(base_model)
2. labels = tokenizer_label(text_target=targets, padding=False, truncation=True)

### Expected behavior

TypeError: __call__() missing 1 required positional argument: 'text'



Comments:
You version of Transformers is too low, you need to upgrade it as `target_text` is a somewhat recent feature.

Should I install tranformers from source ? Because I also tried PIP and
didn’t work.

On Mon, 27 Feb 2023 at 9:02 PM, Sylvain Gugger ***@***.***>
wrote:

> You version of Transformers is too low, you need to upgrade it as
> target_text is a somewhat recent feature.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/21806#issuecomment-1445870119>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AEA4FGUREYYFIHEIQHKINCDWZRNSHANCNFSM6AAAAAAVIIOMAI>
> .
> You are receiving this because you authored the thread.Message ID:
> ***@***.***>
>


This change was introduced in `transformers==v4.22.0`. Try `pip install --upgrade transformers` as `pip install transformers` will do nothing if you already have the library. 

it worked. Thanks
Confusing documentation in T5
Author: seanmor5
Pull request: no

### System Info

latest

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Not sure if this is a bug exactly, but the way the documentation reads for T5 doesn't seem correct in the context of Flan-T5. Specifically, the configuration parameter `d_kv` it states: 

```
d_kv (int, optional, defaults to 64) — Size of the key, query, value projections per attention head. d_kv has to be equal to d_model // num_heads.
```

However if you look at `flan-t5-small`, d_kv is 64 while the hidden size is 512 and the number of heads is 6, which obviously doesn't hold up with the statement in the docs.

Is the T5 behavior correct as is? Are the docs just wrong?

### Expected behavior

Looking at the implementation of T5 it seems generic w.r.t to `d_kv`, `num_heads`, and `d_model` and `inner_dim` is actually the size of the k/v/q projection

Comments:
cc @ArthurZucker and @younesbelkada 

The T5 behaviour is correct, and as pointed out the doc is probably not! I'll open a PR to fix this 👍🏻 thanks
[T5 doc] Fix confusing documentation about `d_kv`
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Fixes #21641. The documentation mentioned that `d_kv` must be equal to `d_model//num_heads` while this does not hold in the code. The code states that `d_kv = inner_dim//num_heads`, but inner dim can be different from the d_model. 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Deprecating `position_ids` in GPTJ
Author: KaijuML
Pull request: no

### System Info

latest transformers version, pytorch

### Who can help?

Hi @ArthurZucker and @younesbelkada, let me know if someone else should be tagged in this, esp. considering this is a harmless "bug", not something really urgent.

Basically what the title says: I think the position_ids in the [GPT-J code](https://github.com/huggingface/transformers/blob/main/src/transformers/models/gptj/modeling_gptj.py) should be deprecated, since they are not used anywhere as far as I can tell. Something along the line of what bloom does at:
https://github.com/huggingface/transformers/blob/main/src/transformers/models/bloom/modeling_bloom.py#L891

I can also submit a PR if you want, just wanted to make sure I didn't overlook anything.

Let me know,
Clément

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

N/A

### Expected behavior

Cleaner code, no useless no-op

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Friendly ping @ArthurZucker and @younesbelkada 

I agree with you, opened a PR for this. Thanks for reporting.
fsdp bf16 enable autocast
Author: pacman100
Pull request: yes

# What does this PR do?

1. Fixes #21560 wrt FSDP integration. 


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Hello @sgugger, officially, there is not much info on `mixed_precision` support of FSDP. In the official docs here https://pytorch.org/docs/stable/fsdp.html, it doesn't mention anything regarding `bf16` and `fp16` nuances. 

In the  official tutorial: https://github.com/lessw2020/transformer_central/tree/main/mixed_precision, for `bf16` they don't specify the need for autocasting. Here, it is mentioned the need for `ShardedGradScaler` for `fp16`. And the issue https://github.com/pytorch/pytorch/issues/75676 is still open. 

However, `bf16` support with FSDP does work for few models, notably T5 as mentioned in this github comment from my experiments: https://github.com/pytorch/pytorch/issues/79605#issuecomment-1184410231

In T5, attention probs are casted back to `bf16` explicitly by this line of code: https://github.com/huggingface/transformers/blob/main/src/transformers/models/t5/modeling_t5.py#L560-L562 . 

This avoids the error that happens for BERT in #21560. With this observation, I just tried enabling autocast and observed no errors and expected performance. Hence, this PR. 




using this PR when I run:
```bash
torchrun --nproc_per_node=2 run_glue.py   --model_name_or_path bert-base-cased   --task_name $TASK_NAME   --overwrite_output_dir   --do_train   --do_eval   --max_seq_length 128   --per_device_train_batch_size 16   --learning_rate 2e-5   --num_train_epochs 3   --output_dir $TASK_NAME/   --fsdp "full_shard auto_wrap"   --fsdp_config "fsdp_config.json"   --bf16
```

wirh fsdp_config.json contents being:
```
{
	"fsdp_transformer_layer_cls_to_wrap": "BertLayer",
	"fsdp_backward_prefetch": "backward_pre",
	"fsdp_forward_prefetch": true,
	"limit_all_gathers": true
}
```

output logs:
```
wandb: Run summary:
wandb:                  eval/accuracy 0.84804
wandb:            eval/combined_score 0.87039
wandb:                        eval/f1 0.89273
wandb:                      eval/loss 0.36461
```

without fsdp run gave below results:
```
torchrun --nproc_per_node=2 run_glue.py   --model_name_or_path bert-base-cased   --task_name $TASK_NAME   --overwrite_output_dir   --do_train   --do_eval   --max_seq_length 128   --per_device_train_batch_size 16   --learning_rate 2e-5   --num_train_epochs 3   --output_dir $TASK_NAME/   --bf16
```

```
wandb: Run summary:
wandb:                  eval/accuracy 0.84804
wandb:            eval/combined_score 0.87057
wandb:                        eval/f1 0.8931
wandb:                      eval/loss 0.36857
```

So, similar performance between them.
add interface or integration to provide interpretability/explainability of hugging face models
Author: aahmadai
Pull request: no

### Feature request

require model answer requests on training biases and model transparency for interpretability and explainability within workflow of such model processes, currently there is no support for this on the platform

e.g.
a form of blackbox testing on the models
a form of ui interface
a way to evaluate
a way to visualize
a linege of data changes in learning process
a publically available benchmarking of models
a way to retune the models - debasing bias

perhaps, linkage to captum/lime, or other such tooling

### Motivation

regulation requirements for trustworthy ai (to be able to answer how the model learned this for correctness, transparency, and fairness)
to be able to correct the biases in training datasets

this is important because you have an array of models which support zero transparency.
it is also important for progressing ai.
to build model lineage
to provide for continued compliance and data governance across different geographic regulations in use of such models.

fundamentally, to answer these questions:
how the result was produced
whether the model was correct in producing such a result based on the implementation

[https://ec.europa.eu/futurium/en/ai-alliance-consultation.1.html]

### Your contribution

not sure how I can help if the developers have yet to even add such feature and make themselves unapproachable, this is something that is constantly overlooked by the ML/DL community with a lot of marketing hype where they cannot fully explain outside of a research paper how the model process reached that result.


Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Mark pipeline tests to skip them easily
Author: sgugger
Pull request: yes

# What does this PR do?

This PR re-introduces the `is_pipeline_test` marker to easily flag pipeline tests. The problem is that since #21516 pipeline tests aren't isolated in the pipelines folder anymore (it looks like it but there is an inheritance of the pipeline tester in all model test classes). Thus the `is_pipeline_test` will allow us to flag those tests.

By setting the RUN_PIPELINE_TESTS environment variable to False, we can skip all pipeline tests. Contrarily to other similar env variables, this one defaults to True. This is because it's very annoying to have to remember to add a `RUN_PIPELINE_TESTS=yes` before the pytest command when trying to debug things locally. I'll actually propose to switch the defaults of all other env variables in a follow-up PR. The main thing is to set it to False when running test jobs unrelated to pipelines (which this PR does).



Comments:
_The documentation is not available anymore as the PR was closed or merged._
Clean up auto mapping names
Author: ydshieh
Pull request: yes

# What does this PR do?

Clean up auto mapping names + add a check 🐕‍🦺 

Comments:
so far on `main`
```bash
`layoutxlm` appears in the mapping `PROCESSOR_MAPPING_NAMES` but it is not defined  in the keys of `CONFIG_MAPPING_NAMES`.
`wav2vec2_with_lm` appears in the mapping `PROCESSOR_MAPPING_NAMES` but it is not defined  in the keys of `CONFIG_MAPPING_NAMES`.
`blip_2` appears in the mapping `MODEL_MAPPING_NAMES` but it is not defined in the keys of `CONFIG_MAPPING_NAMES`.
`decision_transformer_gpt2` appears in the mapping `MODEL_MAPPING_NAMES` but it is not defined in the keys of `CONFIG_MAPPING_NAMES`.
`nllb` appears in the mapping `MODEL_MAPPING_NAMES` but it is not defined in the keys of `CONFIG_MAPPING_NAMES`.

```

_The documentation is not available anymore as the PR was closed or merged._
Add Blip and Blip2 for pipeline tests
Author: ydshieh
Pull request: yes

# What does this PR do?

A continuation of (not merged) #21802.

@NielsRogge I will add you as the contributor.
@Narsil Just in case you want to take a look too, as you reviewed #21904, and asked for adding the tests.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thank you ! !!!
Add: task guide for zero shot object detection
Author: MKhalusova
Pull request: yes

This PR adds a task guide for zero-shot object detection. Unlike other task guides, there is no fine-tuning and complex preprocessing of custom data. However, the task illustrates different ways of inferencing with OWL-ViT, such as using the pipeline, manual inference with text queries, manual inference for a batch of examples, and image-guided object detection. The task guide is based on [this notebook](https://github.com/huggingface/notebooks/blob/main/examples/zeroshot_object_detection_with_owlvit.ipynb) with some additions (e.g. pipeline example) and some modifications.

Comments:
Images are in this PR https://huggingface.co/datasets/huggingface/documentation-images/discussions/49

_The documentation is not available anymore as the PR was closed or merged._

> Nice work, thanks for adding this! I especially like the brief intro to the OWL-ViT model. Maybe we can embed one of the OWL-ViT demos (like this [one](https://huggingface.co/spaces/adirik/OWL-ViT)) directly on the page so users can play with it?

Thanks for the suggestion! I embedded demo :)
Avoid modeling tests run in pipeline CI jobs
Author: ydshieh
Pull request: yes

# What does this PR do?

Avoid modeling tests run in pipeline CI jobs.


PR #21887 applied
```
@is_pipeline_test
class PipelineTesterMixin:
```
Together the changes in #21516
```
class BertModelTest(ModelTesterMixin, PipelineTesterMixin, unittest.TestCase):
```
these 2 make all modeling test methods become pipeline tests, therefore CircleCI pipeline jobs run all pipeline tests + all model tests. This causes OOM as pipeline jobs are run with `-n 8`.

This PR applies `@is_pipeline_test` to each test methods instead of the test class to avoid such issue.


### Effect

Run
```python
python -m pytest -m is_pipeline_test -v tests/models/bridgetower/test_modeling_bridgetower.py::BridgeTowerModelTest::test_model
```

#### On main
test pass

#### on this PR
test deselected

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_21911). All of your documentation changes will be reflected on that endpoint.
Add BLIP and BLIP-2 to image-to-text pipeline
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR adds BLIP and BLIP-2 to the image-to-text pipeline.

Usage is as follows:

```
from transformers import pipeline
from transformers import AutoProcessor, BlipForConditionalGeneration, Blip2ForConditionalGeneration

# BLIPv1
# processor = AutoProcessor.from_pretrained("Salesforce/blip-image-captioning-base")
# model = BlipForConditionalGeneration.from_pretrained("Salesforce/blip-image-captioning-base")

# BLIPv2
processor = AutoProcessor.from_pretrained("Salesforce/blip2-opt-2.7b")
model = Blip2ForConditionalGeneration.from_pretrained("Salesforce/blip2-opt-2.7b")

pipe = pipeline("image-to-text", model=model, image_processor=processor.image_processor, tokenizer=processor.tokenizer)

print(pipe("http://images.cocodataset.org/val2017/000000039769.jpg"))
```

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@ydshieh Are these tested automatically now ? Or should we add something to make sure they are tested too ?

Hey @Narsil !

My PR #21516 is not merged yet, and on `main` it is still using `metaclass`, so the pipeline tests (in theory) are generated on the fly (just as you did before).

But the problem is that we don't generate the tiny models offline yet for newly add models in `transformers`, and therefore those model classes are not tested in pipeline testing on current `main`.

I plan to re-run the tiny model generation ASAP. If it's urgent for this PR, I can do it!

@NielsRogge Do you mind adding a tiny model and a small test in the `tests/pipelines/` directly for this PR maybe ? (The PR looks good, I'd just like to make sure it's not breaking on small models within tests if possible).

FYI: I tried to create tiny models for `blip` and `blip-2` using the existing script, but they both failed to create.

- `blip-2`: there is no `Blip2ModelTest` class
  - (there is `Blip2ForConditionalGenerationTest`, as well as `Blip2VisionModelTest`)
  - but the creation script needs to know from `Blip2ModelTest`
  - ~~(I am not really in favor to further complicating the creation script - it's super complex already)~~
  - it's better if we can manage to have some naming convention in modeling test files
    - (I agree that the current test names in `blip-2` make sense however)

- `blip`: processor fails to be created (due to `feature_extractor` attribute)
  - I will try to fix this and create tiny model for `blip`


I will try to work on enhancing the script. But if you somehow manage to create them manually (model/tokenizer/processor), go ahead.

Close this one as the task is completed in #21904

Thank you @NielsRogge for take the initiative.
Fix doctests for TFVisionTextDualEncoder
Author: Rocketknight1
Pull request: yes

So I might have just copy-pasted all the PyTorch doctests into the TF class and made the CI angry. But it's fixed now!

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_21910). All of your documentation changes will be reflected on that endpoint.
faster forward following what is done for images
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Follow up of #21897

Keeps batch_size active, but restricts it to the imagepart.
Calculates candidate_labels features only once (there's no way with this current pipeline to send different candidate_labels so we can take the optimization.
Ultra narrowed for CLIP but the pipeline has existed for 4 months now with no new models. Given the popularity of CLIP for diffusion models, I think this is ok to overspecify. We can always relax later when new models come in.
This allows to downgrade from ChunkPipeline to regular PIpeline

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Failure seems unrelated
update model_split_percents
Author: ydshieh
Pull request: yes

# What does this PR do?

#21883 changed `WhisperModelTest`'s `model_split_percents ` from `[0.5, 0.7, 0.9]` to `= [0.8, 0.9]` to make `test_model_parallelism` work. But it fails `test_disk_offload` which uses `self.model_split_percents[0]`.

This PR adds back `0.5`. With `model_split_percents = [0.5, 0.8, 0.9]`, the relevant Whisper tests using `model_split_percents` all pass.



Comments:
_The documentation is not available anymore as the PR was closed or merged._
Allow for try/except imports for custom code
Author: zanussbaum
Pull request: no

### Feature request

When uploading a model with custom code, I wanted to try and use Flash Attention in one of the modules. However, to get around a case where people might use it on CPU, I added `try/except` block for the import. 

However, I get an error when downloading locally like `This modeling file requires the following packages that were not found in your environment` which seems to come from [this file](https://github.com/huggingface/transformers/blob/main/src/transformers/dynamic_module_utils.py#L112). 

### Motivation

I want to be able to write custom model code that allows for optional imports if possible (like FlashAttention)

### Your contribution

I could try a crack at a PR?

Comments:
Indeed that sounds like a nice feature to have. I think the easiest way to deal with it would be to remove all try/except blocks from the content [here](https://github.com/huggingface/transformers/blob/37e0974afcbccdc85da59d51b44e1437b6b3caea/src/transformers/dynamic_module_utils.py#L117) before the tests of the imports. If you want to take a stab at it, happy to review a PR!

Yes I'll give it a go and send a PR out in a few!
Use large VM for `repo_utils_job`
Author: ydshieh
Pull request: yes

# What does this PR do?

Use `large` VM for `repo_utils_job`, as #21856 add `torch` for that job which requires more memory.

Comments:
Can you just try to do a fake code modification in one of the repo utils so that the tests are run and we can check it works?

Should be fine with 2 runs

https://app.circleci.com/pipelines/github/huggingface/transformers/58978/workflows/d2090147-1c3f-4b21-8840-36409ab3424b

https://app.circleci.com/pipelines/github/huggingface/transformers/58978/workflows/09bc67f7-a079-4865-9ec2-9094820afa0e/jobs/719891

and it shows docker/large

_The documentation is not available anymore as the PR was closed or merged._