noamaanMulla-03/transformers-issues huggingface dataset:
Handle ValueError in model_utils (generation config)
Author: dbuos
Pull request: yes

# What does this PR do?

Add error handling clause.

<!-- Remove if not applicable -->

Fixes #25388


## Who can review?

@gante 





Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_25389). All of your documentation changes will be reflected on that endpoint.

yes, sounds good to do it at the warning level - important to fix soon as this is breaking all transformers usage of llama 2 (and perhaps other models).

> Thanks for your quick PR! It seems weird to intercept this error here. The problem seems to stem from an invalid generation config (or the exception is badly chosen) so maybe do another except with a different log: probably at warning level, telling the user the generation config is invalid so we go back to a default config.
> 
> What do you think!

@sgugger Absolutely, that makes sense. I've made the necessary changes. I've added another except block with a warning level.

Uhmmm there are way more models out there with generation config issues than I thought 💔  

It seems validation needs more thought, namely:
1. We should be able to load incorrect files, but perhaps keep strictness at save time (to avoid perpetuating bad practices)
2. At least during a transition phase, we must downgrade these exceptions to warnings.

@dbuos This change is something I'd like to avoid -- if we are not throwing exceptions, we should keep doing things the way we were using before, and not reset the generation config. I'm working on validation this week, so I'd like to take this one :) As such, I'm closing this PR.

(cc @sgugger)

ok @gante , will you roll back the latest update then to reverse the breaking changes for models like Llama 2? I was just running the model and same issue. I'm moving to use transformers 4.31 but it's not ideal having to hard code that.

@RonanKMcGovern Yeah, you will be able to run 

```py
from transformers import AutoModelForCausalLM
chk = 'meta-llama/Llama-2-7b-hf'

if __name__ == '__main__':
    model = AutoModelForCausalLM.from_pretrained(chk, load_in_4bit=True, device_map='auto')
    print("Loaded Ok")
```

and use the model without modifications or changes in behavior. You may see a bunch of new warnings guiding you towards correct `generate` parameterization, though ;)

@RonanKMcGovern apologies if my reaction was perceived as abrupt when closing this PR! Your prompt reaction to fix this issue was appreciated 🤗
Llama2 models not loading (Using main branch)
Author: dbuos
Pull request: no

### System Info

Error when loading the model "meta-llama/Llama-2-7b-chat-hf" using the following code:
 
```python
from transformers import AutoModelForCausalLM
chk = 'meta-llama/Llama-2-7b-chat-hf'

if __name__ == '__main__':
    model = AutoModelForCausalLM.from_pretrained(chk, load_in_4bit=True, device_map='auto')
    print("Loaded Ok")
``` 

The error message was:
```shell
`do_sample` is set to `False`. However, temperature is set to 0.9 -- this flag is only used in sample-based generation modes. Set `do_sample=True` or unset temperature to continue.
``` 

This is because the method GenerationConfig.validate() raises a ValueError and that Error is not controlled in modeling_utils.py file.
One possible solution is to add the the ValueError to the except clause in that file:

![image](https://github.com/huggingface/transformers/assets/68216/d359e135-12e0-4730-ae02-8be24361667d)




### Who can help?

@gante

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Using the main branch (install from source code)

```python
from transformers import AutoModelForCausalLM
chk = 'meta-llama/Llama-2-7b-chat-hf'

if __name__ == '__main__':
    model = AutoModelForCausalLM.from_pretrained(chk, load_in_4bit=True, device_map='auto')
    print("Loaded Ok")
``` 

### Expected behavior

To be able to load the model

Comments:
I have same issue with another Model. I think this is urgent to fix.

Same here. Meanwhile downgrading to transfomers 4.31 version solves the problem

I found this issue in the dev branch of transformers-4.32.0.dev0. By the way, I made a PR that could solve that. https://github.com/huggingface/transformers/pull/25389

Same here!

Hey everyone 👋 If you're hitting this exception, it means that there is something wrong with your model's config file 💔  

Meanwhile, we are deciding internally how to massage this question into a more user-friendly solution.

After the PR above gets merged, you will be able to do everything as before. 

The only difference from before is that you will see new warnings, related to poor `generate` parameterization (which may come from the generation config file, as in the case of llama 2) :)
Generate: lower severity of parameterization checks
Author: gante
Pull request: yes

# What does this PR do?

Fixes #25388
Related to #25389 

We are clearly not ready for strict generate parameter checks, as there are several config files on the hub that fail these basic checks (like on Llama 2). 

This PR lowers the severity of the exceptions added in #25381 to warnings and, if the issues are detected at init-time, the warning message suggests fixing the config file itself.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
VQA task guide
Author: MKhalusova
Pull request: yes

This PR adds a new Visual Question Answering task guide to the transformers docs: 
fine-tuning ViLT, based on @NielsRogge 's [notebook](https://github.com/NielsRogge/Transformers-Tutorials/blob/master/ViLT/Fine_tuning_ViLT_for_VQA.ipynb)

 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Addressed @NielsRogge's and @Steven's feedback. @amyeroberts please take a look :)
KnowBert
Author: rishibommasani
Pull request: no
Labels: name: wontfix

As has amazingly and remarkably become a standard response to a new model announcement, will this new transformer model be implemented:
https://arxiv.org/pdf/1909.04164.pdf - KnowBert at EMNLP19. 

Comments:
Hi,
We only add models when there are pretrained weights released.
This doesn't seem to be the case for KnowBert, or maybe I missed them?

Yes, looking into if they are releasing pretrained weights, I incorrectly assumed they were. 

Wow! Really looking forward to this.
I really feel that models that combine text and unstructured data are not getting enough attention.
This is so relevant for creating great AI products because I dare to say that in most real life applications you do not deal with text only data. Metadata is crucial!

The authors released the model. Any update on integrating it into huggingface?

We'd welcome a community or author-contributed implementation! 

(Also might look into integrating it ourselves at some point, but bandwidth is low)

[Update: [link to the implem + weights](https://github.com/allenai/kb)]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Hello Community, Any update on integrating KnowBert into huggingface?
🌐 [i18n-KO] Translated `perf_train_tpu_tf.md` to Korean
Author: 0525hhgus
Pull request: yes

<!-- PR의 제목은 "🌐 [i18n-KO] Translated `<your_file>.md` to Korean" 으로 부탁드립니다 -->
# What does this PR do?

Translated the `perf_train_tpu_tf.md` file of the documentation to Korean. 
Thank you in advance for your review!

Part of https://github.com/huggingface/transformers/issues/20179
<!-- 메인 이슈에 기록이 남아요! 가짜연구소 리포를 사용해 연습하실때는 제거해주시면 감사하겠습니다! :smile: -->

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [ ] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [ ] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에만 가짜연구소 팀원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
<!-- Team PseudoLab, may you please review this PR?  -->
@kihoon71, @0525hhgus, @54data, @Sunmin0520, @seank021, @augustinLib

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. 가짜연구소 팀원들과 리뷰가 끝난 후에만 허깅페이스 직원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
<!-- May you please review this PR? -->
<!-- @sgugger, @ArthurZucker, @eunseojo -->

Comments:
_The documentation is not available anymore as the PR was closed or merged._

I closed the PR because the document was not generated correctly.
🌐 [i18n-KO] Translated `model_sharing.mdx` to Korean
Author: 0525hhgus
Pull request: yes

<!-- PR의 제목은 "🌐 [i18n-KO] Translated `<your_file>.mdx` to Korean" 으로 부탁드립니다 -->
# What does this PR do?

Translated the `model_sharing.mdx` file of the documentation to Korean.
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179
<!-- 메인 이슈에 기록이 남아요! 가짜연구소 리포를 사용해 연습하실때는 제거해주시면 감사하겠습니다! :smile: -->

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [x] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에만 가짜연구소 팀원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
Team PseudoLab, may you please review this PR? 
@0525hhgus, @KIHOON71, @sim-so, @gabrielwithappy, @HanNayeoniee, @wonhyeongseo, @jungnerd

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. 가짜연구소 팀원들과 리뷰가 끝난 후에만 허깅페이스 직원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
@sgugger, @ArthurZucker, @eunseojo May you please review this PR? 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

> In general, additional proofreading is necessary. It is recommended to carefully review the machine-translations and revise any sections that appear unclear or inaccurate.

Thank you for the advice. I will proceed with more detailed proofreading in machine translation!

May you please review this PR? 🙂
@sgugger, @ArthurZucker, @eunseojo
Cannot reproduce results for Pix2struct on InfographicVQA
Author: Lizw14
Pull request: no

I am using the `pix2struct-infographics-vqa-base` and `pix2struct-infographics-vqa-large` model here and doing inference on InfographicsVQA. However, I get 29.53 ANLS for base and 34.31 ANLS for large, which do not match with the 38.2 and 40.0 results as in the original paper. Could anyone help with this?

Here is my inference code:
```
import requests
from PIL import Image
import torch
from transformers import Pix2StructForConditionalGeneration, Pix2StructProcessor

model = Pix2StructForConditionalGeneration.from_pretrained("google/pix2struct-infographics-vqa-base").to("cuda")
processor = Pix2StructProcessor.from_pretrained("google/pix2struct-infographics-vqa-base")

image_url = "https://blogs.constantcontact.com/wp-content/uploads/2019/03/Social-Media-Infographic.png"
image = Image.open(requests.get(image_url, stream=True).raw)
question = "Which social platform has heavy female audience?"
inputs = processor(images=image, text=question, return_tensors="pt").to("cuda")

predictions = model.generate(**inputs)
pred = processor.decode(predictions[0], skip_special_tokens=True)
gt = 'pinterest'

print(pred)
```

Comments:
cc @younesbelkada 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

gentle ping @younesbelkada 

Hi everyone, 
Sadly I won't have the bandwidth to properly dig into this right now, @Lizw14 do you still face the same issue when using the main branch of `transformers`?
```
pip install git+https://github.com/huggingface/transformers.git
```

@Lizw14 
quickly going back to the issue, can you double check you used the same hyper parameters than the ones presented on the paper? for example what is the sequence length you are using? in what precision do you load the model (fp32, fp16, bf16, int8)?
Ideally can you share the full script you use to reproduce the results of the paper
Thanks!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
[DOCS] Add example for `TopPLogitsWarper` 
Author: chiral-carbon
Pull request: yes

# What does this PR do?

Adds an example to the docstring of `TopPLogitsWarper` class definition in [this file](https://github.com/huggingface/transformers/blob/main/src/transformers/generation/logits_process.py).

Fixes one of the cases in #24783 

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.

## Who can review?
@gante 


Comments:
Thanks for the review, @gante!

Quick question:

> * we don't need to set any pad token, as we are not adding padding

that is correct, but without the two lines for padding, I get this response when I run `model.generate()` in my own python terminal:
```
The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.
Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.
```
If this is fine, then I will go ahead and remove the line setting the pad token.

@chiral-carbon yes, that is fine to illustrate the example :)

Thanks @gante, I have addressed the nits. I re-ran the commands in terminal for good measure so you will see different outputs from the previous commit.

_The documentation is not available anymore as the PR was closed or merged._
Support pushing of NF4 to hub
Author: RonanKMcGovern
Pull request: no

What would it take to allow NF4 models from bitsandbytes to be pushed to the hub?

Comments:
cc @younesbelkada ans @SunMarc 

Hi @RonanKMcGovern, in order to allow NF4 models to be pushed to the hub, we need to be able to serialize them just like for 8-bit model. Feel free to open an issue on bitsandbytes library to request this feature. In our side, we can't do much. 

Thanks - submitted an [issue here](https://github.com/TimDettmers/bitsandbytes/issues/695).
🌐 [i18n-KO] Translated `add_new_model.md` to Korean
Author: mjk0618
Pull request: yes

<!-- PR의 제목은 "🌐 [i18n-KO] Translated `<your_file>.md` to Korean" 으로 부탁드립니다! -->
# What does this PR do?

Translated the `add_new_model.md` file of the documentation to Korean.
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [x] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)
<!-- 1. 위 체크가 모두 완료된 뒤에, 이 아래에 리뷰를 요청할 팀원들을 멘션해주세요! -->
<!-- May you please review this PR? @member1 @member2 ... -->
May you please review this PR? @nuatmochoi, @bolizabeth, @hyunhp, @mjk0618, @jungnerd

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)
 May you please review this PR? @sgugger, @ArthurZucker, @eunseojo
<!-- 2. 팀원들과 리뷰가 끝난 후에만 허깅페이스 직원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->

Comments:
Hello @sgugger , may you please approve the workflow for this PR?

I ran doc-builder locally, and it looks OK. We just want to see the live-preview to double check.

Thank you so much for your support.
Hope you have a great day! 👍 

_The documentation is not available anymore as the PR was closed or merged._

May you please review this PR? @sgugger, @ArthurZucker, @eunseojo
🌐 [i18n-KO] Translated `model_summary.md` to Korean
Author: 0525hhgus
Pull request: yes

<!-- PR의 제목은 "🌐 [i18n-KO] Translated `<your_file>.md` to Korean" 으로 부탁드립니다 -->
# What does this PR do?

Translated the `model_summary.md` file of the documentation to Korean 😄 
Thank you in advance for your review!

Part of https://github.com/huggingface/transformers/issues/20179
<!-- 메인 이슈에 기록이 남아요! 가짜연구소 리포를 사용해 연습하실때는 제거해주시면 감사하겠습니다! :smile: -->

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [x] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에만 가짜연구소 팀원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
<!-- Team PseudoLab, may you please review this PR?  -->
@0525hhgus, @KIHOON71, @sim-so, @gabrielwithappy, @HanNayeoniee, @wonhyeongseo, @jungnerd

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. 가짜연구소 팀원들과 리뷰가 끝난 후에만 허깅페이스 직원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
May you please review this PR?
@sgugger, @ArthurZucker, @eunseojo

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thanks for translating the `model_summary` file! Feel free to mark as ready for review whenever you're ready 😄 

> Thanks for translating the `model_summary` file! Feel free to mark as ready for review whenever you're ready 😄

Thank you for mentioning it! I changed the status to ready for review. 

May you please review this PR? 😄 
@sgugger, @ArthurZucker, @eunseojo