noamaanMulla-03/transformers-issues huggingface dataset:
Possible error in text classification example code for multi-label classification
Author: subhalingamd
Pull request: no

### System Info

N/A *(system config independent)*

### Who can help?

@ArthurZucker and @younesbelkada 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

N/A

### Expected behavior

Shouldn't the default condition be `p > 0` in the following lines for multi-label classification? The sequence classification model returns the logits and sigmoid has not been applied anywhere.

https://github.com/huggingface/transformers/blob/5c081e29930466ecf9a478727039d980131076d9/examples/pytorch/text-classification/run_classification.py#L658

https://github.com/huggingface/transformers/blob/5c081e29930466ecf9a478727039d980131076d9/examples/pytorch/text-classification/run_classification.py#L724




Comments:
True yes, this seems to be a good point - cc @ranchlai who added this script

Indeed. This will cause higher precision and lower recall in multi-label classification. Thanks @subhalingamd for pointing out. I can give a quick fix. what do you think? @younesbelkada 

That would be great @ranchlai ! Thanks a lot !
Limit to inferior fsspec version
Author: LysandreJik
Pull request: yes

The newly released fsspec version breaks the implementation in transformers' CI, see the following error:

```
"/home/circleci/.pyenv/versions/3.8.12/lib/python3.8/site-packages/datasets/builder.py", line 1173, in as_dataset
    raise NotImplementedError(f"Loading a dataset cached in a {type(self._fs).__name__} is not supported.")
NotImplementedError: Loading a dataset cached in a LocalFileSystem is not supported.
/home/circleci/transformers/src/transformers/models/beit/modeling_beit.py:737: UnexpectedException
```

Additionally:
https://github.com/huggingface/datasets/pull/6331 and https://github.com/huggingface/huggingface_hub/pull/1773

Comments:
_The documentation is not available anymore as the PR was closed or merged._
python falcon doc-string example typo
Author: SoyGema
Pull request: yes

# What does this PR do?

<!--

-->

<!-- Remove if not applicable -->

fix small typo .


## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@amyeroberts @ArthurZucker 



Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26995). All of your documentation changes will be reflected on that endpoint.
Change default `max_shard_size` to smaller value
Author: younesbelkada
Pull request: yes

# What does this PR do?

As per title, we can also change that value dynamically with respect to model size - with this change large models will end up having many shards.


cc @ArthurZucker 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26942). All of your documentation changes will be reflected on that endpoint.
[`NLLB-MoE`] Fix NLLB MoE 4bit inference
Author: younesbelkada
Pull request: yes

# What does this PR do?

Fixes: https://github.com/huggingface/transformers/issues/26898

The hidden states gets silenty casted in `unint8` leading to the error described in #26898

The check `and self.fc2.weight.dtype != torch.int8` is not sufficient in order to cover 4bit models, for these models the weights are in `uint8`, hence adding an extra condition to cover 4bit models fixes the inference issue

cc @ArthurZucker 


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thanks!
'Owlv2Processor' object has no attribute 'post_process'
Author: dinhanhx
Pull request: no

### System Info

- `transformers` version: 4.35.0.dev0
- Platform: Linux-5.15.133+-x86_64-with-glibc2.35
- Python version: 3.10.12
- Huggingface_hub version: 0.16.4
- Safetensors version: 0.3.3
- Accelerate version: 0.22.0
- Accelerate config: 	not found
- PyTorch version (GPU?): 2.0.0 (True)
- Tensorflow version (GPU?): 2.12.0 (True)
- Flax version (CPU?/GPU?/TPU?): 0.7.2 (gpu)
- Jax version: 0.4.13
- JaxLib version: 0.4.13
- Using GPU in script?: NVIDIA P-100
- Using distributed or parallel set-up in script?: Non

### Who can help?

@NielsRogge 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Run the example code from huggingface model repo, https://huggingface.co/google/owlv2-base-patch16,

Kaggle notebook, https://www.kaggle.com/inhanhv/google-owlv2 

### Expected behavior

'Owlv2Processor' object has attribute 'post_process'

Comments:
Hi,

Thanks for flagging (note that you can also open a discussion on the huggingface model repo). The model cards are now updated, one needs to use `post_process_object_detection` instead of `post_process`.

I understand. Thanks for the help.
[DOCS] updated docstring for tokenizer 
Author: KMJ-007
Pull request: yes

# What does this PR do?

updated docstring for the following :

`BartTokenizerFast`,` BarthezTokenizerFast`, `BertTokenizerFast`, `AlbertTokenizerFast`, `BigBirdTokenizerFast`, `BlenderbotSmallTokenizerFast`, `BlenderbotTokenizerFast`

Comments:
@ydshieh Can you please review this?


Hi @KMJ-007 

Thank you for your contribution.

Let's not take a huge number of entries. Either a single entry, or the entries from the same model architecture. So everyone can have the chance to contribute 🙏 

understood!

closing this PR and creating new PR for same model architecture
small typos found
Author: rafaelpadilla
Pull request: yes

just very small typos found

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@ArthurZucker , @amyeroberts 


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Remove token_type_ids from default TF GPT-2 signature
Author: Rocketknight1
Pull request: yes

Although GPT-2 supports `token_type_ids`, the implementation is very weird (token embeddings are also used as token type embeddings!) and in practice `token_type_ids=None` is used almost exclusively.

For most models, you could mimic the effects of `token_type_ids=None` by just passing an all-zeros array, but this does not work for GPT-2 because it completely skips the embeddings when `token_type_ids=None`. This means that a model exported with a `token_type_ids` input cannot be coerced to behave correctly.

To stop this tripping up other users, we remove `token_type_ids` from the GPT-2 input sig. This issue is specific to GPT-2 - most other models have more reasonable ways of handling `token_type_ids` and shouldn't be affected.

Fixes #26783

Comments:
_The documentation is not available anymore as the PR was closed or merged._
How to perform batch inference? 
Author: ryanshrott
Pull request: no

### Feature request

I want to pass a list of tests to model.generate. 

text = "hey there"
inputs = tokenizer(text, return_tensors="pt").to(0)

out = model.generate(**inputs, max_new_tokens=184)
print(tokenizer.decode(out[0], skip_special_tokens=True))




### Motivation

I want to do batch inference. 

### Your contribution

Testing

Comments:
I opened a PR for this in #24432 to illustrate it for GPT-2, but it will be incorporated in a bigger PR.

cc @gante @stevhliu

See #24575

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Hey there @ryanshrott @NielsRogge 👋 

I've added a short section in our basic LLM tutorial page on how to do batched generation in [this PR](https://github.com/huggingface/transformers/pull/26937). 

Taken from the updated guide, here's an example:
```py
>>> from transformers import AutoTokenizer, AutoModelForCausalLM

>>> model = AutoModelForCausalLM.from_pretrained(
...     "mistralai/Mistral-7B-v0.1", device_map="auto", load_in_4bit=True
... )
>>> tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-v0.1", padding_side="left")
>>> tokenizer.pad_token = tokenizer.eos_token  # Most LLMs don't have a pad token by default
>>> model_inputs = tokenizer(
...     ["A list of colors: red, blue", "Portugal is"], return_tensors="pt", padding=True
... ).to("cuda")
>>> generated_ids = model.generate(**model_inputs)
>>> tokenizer.batch_decode(generated_ids, skip_special_tokens=True)
['A list of colors: red, blue, green, yellow, orange, purple, pink,',
'Portugal is a country in southwestern Europe, on the Iber']
```

@gante Thanks. Is this faster than running them in a loop?

@ryanshrott yes, much faster when measured in thorughput! The caveat is that it requires slightly more memory from your hardware, and it will have a slightly higher latency
🌐 [i18n-ZH] Translate multilingual into Chinese
Author: yyLeaves
Pull request: yes

# What does this PR do?
Add zh (Chinese) translation for multilingual.md. #20095 
## Who can review?
Documentation: @stevhliu 
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26935). All of your documentation changes will be reflected on that endpoint.
Bugfix device map detr model
Author: pedrogengo
Pull request: yes

# What does this PR do?

Fixes #26700 #23145


## Who can review?

@LysandreJik @SunMarc 

Comments:
Hi @SunMarc :) I don't have 2 GPUs unfortunately. Can you run on your side? In the meantime I can remove the tests I added + fix the style issue

@SunMarc addressed your comment on my side. Could you test?
I also noticed a bad pattern maybe on `check_copies.py`. At first, I tried this:

`# Copied from transformers.models.deformable_detr.modeling_deformable_detr.DeformableDetrPreTrainedModel with DeformableDetr->Deta,DeformableDetrConvEncoder->DetaBackboneWithPositionalEncodings`

And it was breaking the CI, because it first replace `DeformableDetr` to `Deta`, and after search for the next pair, which was `DeformableDetrConvEncoder->DetaBackboneWithPositionalEncodings`. However, as we first replaced, it turns that in the code `DeformableDetrConvEncoder` was changed to `DetaConvEncoder`. Maybe we can sort by length and start replacing from the largest string, and also replace with a placeholder in both codes to avoid replacing more than one time. Just something that I saw haha I can propose a PR for that, but I don't think it is that much important.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26849). All of your documentation changes will be reflected on that endpoint.
[docstring] Fix docstring for ErnieConfig, ErnieMConfig
Author: Sparty
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @muellerzr and @pacman100

Integrations:

- deepspeed: HF Trainer/Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc and @younesbelkada

Documentation: @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->

Comments:
🌐 [i18n-ZH] Translate create_a_model.md into Chinese
Author: yyLeaves
Pull request: yes

# What does this PR do?
Translate create_a_model.md into Chinese
part of #20095  
## Who can review?
Documentation: @stevhliu 
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_27026). All of your documentation changes will be reflected on that endpoint.
Fix key dtype in GPTJ and CodeGen
Author: fxmarty
Pull request: yes

As per title.

The issue is that 

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "hf-internal-testing/tiny-random-GPTJForCausalLM"
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float16)
tokenizer = AutoTokenizer.from_pretrained(model_id)

inp = tokenizer("this is me", return_tensors="pt").to("cuda")
model = model.to("cuda")

model = model.eval()
with torch.no_grad():
    res = model(**inp, use_cache=True)

print("res.past_key_values[0][0].dtype", res.past_key_values[0][0].dtype)
print("res.past_key_values[0][1].dtype", res.past_key_values[0][1].dtype)
```
gives
```
res.past_key_values[0][0].dtype torch.float32
res.past_key_values[0][1].dtype torch.float16
```

the reason being that `sin` and `cos` are not properly casted to the correct dtype contrarily to the implementation in e.g. llama.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thank you @ArthurZucker for the notice. Indeed in the original codebase of GPTJ and codegen `sincos` is kept as fp32, and the ROPE computation as well as `_attn` computation is done with the `key` always in fp32. The looks very much like a bug in the original implementation of those models.

Anyway, in the interest of keeping the compute similar to the original implementation, I simply cast the key cache to the correct dtype.

It is still float16 x float32. We simply store the KV cache as float16. This `key = torch.cat((past_key, key), dim=-2)` basically casts everything to float32 (while `past_key` is fp16, `key` is fp32)

(edit: But I see the point. Maybe casting the key cache to fp16 can bring instabilities. I would be surprised though given that it is not an issue for similar decoder models).

it is for llama 🙄
[`GPTNeoX`] Faster rotary embedding for GPTNeoX (based on llama changes)
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Fixes #25813 which indicates that ROPE is slow. It's a follow up of #22785, were ROPE was improved for Llama model. 


Comments:
_The documentation is not available anymore as the PR was closed or merged._

cc @StellaAthena FYI, this PR should greatly speed up the ROPE embeddings of the GPTNeoX model, similarly to how it was done for the LLaMa model.

Let us know if you want to review/have any comments!

cc @Narsil as this touches buffers that will no longer be persistent, will wait for you in case this is conflicting with TGI?
Perplexity docs have broken gifs
Author: hamelsmu
Pull request: no

See https://huggingface.co/docs/transformers/perplexity

Looks like raw html is showing up

![image](https://github.com/huggingface/transformers/assets/1483922/8fe15d12-5f43-4eb2-8704-a04928c80199)

Also looks like that asset is not at that location. 

cc: @muellerzr 

Comments:
Thank you so much for reporting, we will take a look 🤗 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Add `default_to_square_for_size` to `CLIPImageProcessor`
Author: ydshieh
Pull request: yes

# What does this PR do?

Add `default_to_square_for_size` to `CLIPImageProcessor`.

As in the file, we also have `crop_size = get_size_dict(crop_size, default_to_square=True, param_name="crop_size")`, so I give the name `default_to_square_for_size` for `get_size_dict(size, default_to_square=default_to_square_for_size)` to avoid confusion.

Otherwise, I can give it `default_to_square`, and we may add `default_to_square_for_crop_size` later (or in this PR).

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thanks a lot !

I forgot to change this line

```
        output_size = get_resize_output_image_size(
            image, size=size["shortest_edge"], default_to_square=False, input_data_format=input_data_format
        )
```
to 
```
        output_size = get_resize_output_image_size(
            image, size=size["shortest_edge"], default_to_square=self.use_square_size, input_data_format=input_data_format
        )
```
will do it but wait on next Monday to merge
[docstring] Fix docstrings for `CodeGenConfig`, `CodeGenTokenizer`, `CodeGenTokenizerFast`
Author: daniilgaltsev
Pull request: yes

# What does this PR do?

Fixes the docstrings for CodeGen following #26638 

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @muellerzr and @pacman100

Integrations:

- deepspeed: HF Trainer/Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc and @younesbelkada

Documentation: @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->


Comments:
@ydshieh Hi, can you review this?

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26821). All of your documentation changes will be reflected on that endpoint.
Microsoft's GLIP Grounding Language Image Pretraining
Author: ethansmith2000
Pull request: no
Labels: name: New model

### Model description

Combines best practices of CLIP and object detectors.
Allows for localization of text and image content.

### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

github https://github.com/microsoft/GLIP
weights on hf https://huggingface.co/GLIPModel/GLIP/tree/main
colab demo https://colab.research.google.com/drive/12x7v-_miN7-SRiziK3Cx4ffJzstBJNqb?usp=sharing

Comments:
Hey, is there a reason why you opened this one as well as #27031 ?