noamaanMulla-03/transformers-issues huggingface dataset:
sample code in for mobilevitv2 is not working
Author: darwinharianto
Pull request: no

### System Info

- `transformers` version: 4.31.0.dev0
- Platform: macOS-10.16-x86_64-i386-64bit
- Python version: 3.10.11
- Huggingface_hub version: 0.14.1
- Safetensors version: 0.3.1
- PyTorch version (GPU?): 2.0.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: no
- Using distributed or parallel set-up in script?:no

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

run sample code from [MobileViTV2ForImageClassification](https://huggingface.co/docs/transformers/v4.30.0/en/model_doc/mobilevitv2#transformers.MobileViTV2ForImageClassification)

throws an error

ValueError: Unrecognized image processor in apple/mobilevitv2-1.0-imagenet1k-256. Should have a `image_processor_type` key in its preprocessor_config.json of config.json, or one of the following `model_type` keys in its config.json: align, beit, bit, blip, blip-2, bridgetower, chinese_clip, clip, clipseg, conditional_detr, convnext, convnextv2, cvt, data2vec-vision, deformable_detr, deit, deta, detr, dinat, donut-swin, dpt, efficientformer, efficientnet, flava, focalnet, git, glpn, groupvit, imagegpt, instructblip, layoutlmv2, layoutlmv3, levit, mask2former, maskformer, mgp-str, mobilenet_v1, mobilenet_v2, mobilevit, mobilevitv2, nat, oneformer, owlvit, perceiver, pix2struct, poolformer, regnet, resnet, sam, segformer, swiftformer, swin, swin2sr, swinv2, table-transformer, timesformer, tvlt, upernet, van, videomae, vilt, vit, vit_hybrid, vit_mae, vit_msn, xclip, yolos


### Expected behavior

sample code should not throw an error

Comments:
Hi @darwinharianto 

Thank you for opening the issue.

Could you specify which code sample in the link is the exact one that has problem to run? Thank a lot!

@darwinharianto Thanks for reporting this! It seems the issue is coming from the preprocessor config files on the hub for this checkpoint: [it points to a class which doesn't exist](https://huggingface.co/apple/mobilevitv2-1.0-imagenet1k-256/blob/6229cf24f57fe7210db6c6f1ad872a616b802679/preprocessor_config.json#L10). If I clone and modify the config file, the image processor will load correctly.

I'll also add this modeling file to our doctests.
Add MobileVitV2 to doctests
Author: amyeroberts
Pull request: yes

# What does this PR do?

Adds MobileVitV2 to the doctests. The example snippet wasn't working because the model's config files pointed to an image processor that doesn't exist. This adds the models to the doctests so that this is caught. 

Also removes a duplicate line in image_processing_auto.py

Fixes #24763 (partially)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

Comments:
_The documentation is not available anymore as the PR was closed or merged._

> ValueError: Files in `utils/documentation_tests.txt` are not in alphabetical order.

@amyeroberts  

You are the one creating this check 😆 

![image](https://github.com/huggingface/transformers/assets/22614925/6636d6fa-8da3-4305-b69f-a7925ecde3a8)


Oh, I am wrong! The PR doctest is not triggered as this PR doesn't change modeling file. Great!
Docs: Update logit processors __call__ docs
Author: gante
Pull request: yes

# What does this PR do?

PR done as part of https://github.com/huggingface/transformers/issues/24575

This PR polishes the inexistent `__call__` method docs for the logit processors (before this PR, only the base classes had docs). 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Replacement of 20 asserts with exceptions
Author: Baukebrenninkmeijer
Pull request: yes

# What does this PR do?

Replaces 20 assertions with relevant errors, mostly ValueError. 

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes part of #12789 

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

## Who can review?

@JuheonChu  
@sgugger 

I saw both of you tagged in above issue. Please have a look when you have time! :)

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @sgugger

Integrations:

- deepspeed: HF Trainer/Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam

Documentation: @sgugger, @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: @sgugger
- TensorFlow: @Rocketknight1

 -->


Comments:
You will need to put your PR out of draft mode for us to be able to merge it :-)

_The documentation is not available anymore as the PR was closed or merged._
Update default values of bos/eos token ids in `CLIPTextConfig`
Author: ydshieh
Pull request: yes

# What does this PR do?

Currently the default values are not the ones from the corresponding tokenizers.

See discussion in #24650

However, we can't use the `config.eos_token_id` in the modeling file (which is the ultimate goal in #24650) with only the change in this PR. We will have to update all the Hub repo. config files first 😢 . (Probably there is something easier to do)

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Regarding the padding token:

(copy past from (partial) internal discussion given @patil-suraj)

> When we added CLIP I tested for the text_projection , logits_per_image and logits_per_text. For the text_projection the model pulls the embeddings of the last token i.e the eos token. The rest of the tokens i.e the padding tokens are ignored. We can see in this [colab](https://colab.research.google.com/drive/1kgGMnFpkc4TP7otlhAOngp9Wlke4tKJw?usp=sharing) that text_projection ,  logits_per_image and logits_per_text match with the OAI model because we only take the pooled embeddings. And when CLIP was released it was intended for these features  which are needed for contrastive tasks. Hence I didn't test against all token embeddings.

> IMO the wrong padding token will only affect inference when using all token emebeddings i.e Stable Diffusion. For training even if the padding token is wrong it shouldn't affect because

  > - Because CLIP did not use attention_mask during training.


  > - CLIPTextEncoder uses casual mask, so the tokens to the right don't influence the hidden states of tokens to the left.
  > - CLIP is trained with contrastive loss which is computed using the projections, and as I said above the text_projection is computed by pooling the eos token embeddings, which will be always similar no matter what the padding token is, because CLIPTextEncoder is causal, so the eos embeddings  won't be affected by tokens on the right.
  > - Hence, for downstream training (like SD) as long as a consistent token is used for padding it shouldn't severely affect the training. But for inference we will need to use the same token as Patrick explained. 
This could also be the reason that we didn't have any issue related to this.

> As far as I can understand, it'll only affect the inference if a different token (compared to the padding token used for training) is used for padding. (edited)
Fix pad across processes dim in trainer and not being able to set the timeout
Author: muellerzr
Pull request: yes

# What does this PR do?

Reverts tiny regression where `dim=1` is needed during `pad_across_processes`, and `ddp_timeout` wasn't trickled down through `PartialState`

Fixes # (issue)

Solves https://github.com/huggingface/transformers/issues/24751

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sgugger


Comments:
Yep, sorry 😅 Right one is used now

_The documentation is not available anymore as the PR was closed or merged._

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_24775). All of your documentation changes will be reflected on that endpoint.
gpt-bigcode: avoid `zero_` to support Core ML
Author: pcuenca
Pull request: yes

# What does this PR do?

In-place `zero_` is not supported by the Core ML conversion process. This PR replaces it with `zeros_like` so conversion can proceed.

The change only affects a workaround for a PyTorch bug on the `cpu` device.

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@younesbelkada, @loubnabnl, @jlamypoirier 


Comments:
Note: to fully test conversion of `gpt-bigcode` models, the following `coremltools` PRs (or equivalent workarounds) need to be applied as well: https://github.com/apple/coremltools/pull/1910, https://github.com/apple/coremltools/pull/1911.

_The documentation is not available anymore as the PR was closed or merged._

@younesbelkada I think this is already fixed in PT.
Should we just drop this logic?
Opened a PR: https://github.com/huggingface/transformers/pull/24768 which supercedes this one

We support versions of PyTorch from 1.10 and onward, so we need to keep the workaround for the bug.

Merging to unblock @pcuenca , let's maybe address @jlamypoirier 's comments in a follow up PR !
How can I using deepspeed along with LION optimizer/.?
Author: luohao123
Pull request: no

How can I using deepspeed along with LION optimizer/.?

Comments:
Please use the [forums](https://discuss.huggingface.co/) for such questions.

@sgugger I have post one: https://discuss.huggingface.co/t/how-to-using-lion-optimizer/42270 Hope you guys could give any help.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Improve PreTrainedModel.from_pretrained return type
Author: ringohoffman
Pull request: no

### System Info

transformers version: `main`
Platform: macOS 13.1 (x86_64)
Python version: 3.7.15
PyTorch version (GPU?): 1.13.0 (False)
Tensorflow version (GPU?): N/A
Using GPU in script?: No
Using distributed or parallel set-up in script?: No

### Who can help?

Documentation: @sgugger, @stevhliu and @MKhalusova

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```python
import transformers

bert_model = transformers.BertForSequenceClassification.from_pretrained("...")

reveal_type(bert_model)
# expected: BertForSequenceClassification
# actual: tuple[Unknown | BertForSequenceClassification, dict[str, Unbound | Unknown] | dict[str, Unknown | list[Unknown]] | Unknown] | Unknown | BertForSequenceClassification
```

### Expected behavior

The hinted type of the return from `from_pretrained` should be the same type as the class.

I know we could just annotate `from_pretained` to return `typing.Self`... but IMO this points to a lack of typing in a core part of `transformers`, and I think it would make this core code more robust if the static type checkers agreed that that is the correct return type (since there may be type errors lurking).

Comments:
Transformers does not support any type-checker and we do not want to bloat the code to add support for them: we accept PRs adding simple annotations but we will refuse anything more complex than that.

> Transformers does not support any type-checker and we do not want to bloat the code to add support for them: we accept PRs adding simple annotations but we will refuse anything more complex than that.

I can understand your position of not wanting to bloat the code. It definitely seems reasonable if that is an established philosophy for huggingface internal APIs. I think for user-facing APIs, though, it could be beneficial to have more advanced type hints so that the average user's IDE will tell them what type of object they are dealing with when loading their model, rather than seeing a confusing, gigantic type union.

I was originally planning to submit all the work I had done to fix the type issues in `from_pretrained`, but I would be happy to leave it at just improving the signature of `PretrainedModel.from_pretrained` if you will be willing to consider it: https://github.com/huggingface/transformers/pull/24035

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Rm duplicate pad_across_processes
Author: muellerzr
Pull request: yes

# What does this PR do?

Accelerate now handles `pad_across_processes` directly, so removes code copied from Accelerate. As it's internal, no need for a deprecation cycle

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sgugger 


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Add text-mesh models inside Hugginfaces
Author: math-sasso
Pull request: no
Labels: name: New model

name: Feature request

### Feature request

Text to 3D models are really getting the scene in some industries but actually the state of the art techiniques are very hard to integrate in production code. Some examples are:

https://www.nasir.lol/clipmesh
https://github.com/openai/shap-e


Would be awesome to the community if HF has that integrated.

### Motivation

Text-3D models can have a big space in multiple types of industry

### Your contribution

If I have some guidance I can help working on this side. But I will need HF developers help.

Comments:
Hi @math-sasso, 

This would be a great addition to the library! 

I don't know the paper's in depth, but I believe that both of these models - CLIPMesh and Shap-e - are diffusion models and so might be better suggestions for the diffusers library: https://github.com/huggingface/diffusers
Does max_seq_length specify the maxium number of words
Author: artemlos
Pull request: no

I'm trying to figure out how the `--max_seq_length` parameter works in [run_classifier](https://github.com/huggingface/pytorch-pretrained-BERT/blob/master/examples/run_classifier.py). Based on the source, it seems like it represents the number of words? Is that correct?

Comments:
`max_seq_length` specifies the maximum number of tokens of the input. The number of token is superior or equal to the number of words of an input. 

For example, the following sentence:

```
The man hits the saxophone and demonstrates how to properly use the racquet.
```

is tokenized as follows:

```
the man hits the saxophone and demonstrates how to properly use the ra ##c ##quet .
```

And depending on the task 2 to 3 additional special tokens (`[CLS]` and `[SEP]`) are added to the input to format it.

@rodgzilla thanks!

could we make it smaller? 

So what if there are sentences where the maximum number of tokens is greater than max_seq_length?

Does that mean extra tokens beyond max_seq_length will get cut off?

@tsungruihon yes, just use smaller sentences

@echan00 no automatic cut off but there is a warning from the tokenizer that your inputs are too long and the model will throw an error. You have to limit the size manually.

Hi All,

Does that mean we cannot use BERT for classifying long documents. The documents having 5-6 Paragraphs and each paragraph having 10-15 mins with about 10-12 words in each line ?

@SaurabhBhatia0211 
You can try splitting a document to smaller chunks (e.g. paragraphs or even lines), computing embedding for each of those chunks, and average those vectors to get the document representation. 

@rodgzilla is this true?

> HuggingFace's Trainer API, including the SFTrainer, by default pads all sequences to the maximum length within the batch, not to the max_seq_length argument. The max_seq_length argument serves as a hard limit to the sequence length, truncating any examples that are longer than that. The API was designed this way because padding to the maximum sequence length in the batch improves computational efficiency.

?