noamaanMulla-03/transformers-issues huggingface dataset:
[`logging.py`] set default `stderr`  path if `None`
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Attempt to fix #24047

Thanks to contributors, seems like the issue [is known](https://github.com/pyinstaller/pyinstaller/issues/7334#issuecomment-1357447176). 
Though it is not entirely related to `transformers`, adding a safeguard seems like a good practice

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Monkey patching globals like `sys.stdout` and `sys.stderr` is more of an emergency hack than a fix that libraries should be using. The proper way to handle Python windowed mode is to guard any direct usage of `sys.stderr` in `if sys.stderr is not None` blocks.

```python
sys.stderr.write("hello")  # This is broken

# Any of these are fine
if sys.stderr:
    sys.stderr.write("hello")
print("hello", file=sys.stderr)
```

>  Though it is not entirely related to transformers, adding a safeguard seems like a good practice

It is related to any library that uses `sys.stderr` directly.
Bug in autotokenizer some open source models such as falcon. 
Author: Alkahwaji
Pull request: no

### System Info

I want to get the sentence difference and use autotokenizer a falcon model 40B and 7B but usually, I receive an attribute error bug: model 'signal' has no attribute 'SIGALRM'.


![image](https://github.com/huggingface/transformers/assets/23569519/75d671dd-8b57-4447-acf2-5c093b79eddd)


@younesbelkada  @ArthurZucker @Rocketknight1

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

'''
from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("tiiuae/falcon-40b-instruct")
model = AutoModelForCausalLM.from_pretrained("tiiuae/falcon-40b-instruct")

'''


![image](https://github.com/huggingface/transformers/assets/23569519/e53a5a08-459e-4dde-bb95-ac151a018a33)


### Expected behavior

Give me a method to get the word embeddings using sentence transformers or autotokenizer open source models such as falcon or LLMA2.

Comments:
Hey! Could you provide a full trace to know where the error come from?
I cannot reproduce the error with the script you provided. Make sure you are using the latest version of transformers too! 

Thanks. 

![image](https://github.com/huggingface/transformers/assets/23569519/4921397e-8ae2-4cfe-925e-8415b5ab66af)



It seems that the [SIGALARM is not available on Windows](https://stackoverflow.com/questions/52779920/why-is-signal-sigalrm-not-working-in-python-on-windows). But I am not very familiar with this, will let @sgugger answer! 

As per the[ contribution guidelines](https://github.com/huggingface/transformers/blob/c9a82be592ca305180a7ab6a36e884bca1d426b8/CONTRIBUTING.md), could you provide the platform you are running this on? 
- which version of python are you using? 
- which version of transformers are you using? 
- are you on linux or windows? 

You can avoid this error by passing `trust_remote_code=True` (which is needed for this model). We will make the error message clearer when signal is not available.
Better error message when signal is not supported on OS
Author: sgugger
Pull request: yes

# What does this PR do?

This PR adds a `try`/`except` block around the logic asking the user whether they want to execute the code on a distant repo when they don't set `trust_remote_code=True` to give a helpful error message.

Fixes #25029

Comments:
_The documentation is not available anymore as the PR was closed or merged._
[`RWKV`] Add note in doc on `RwkvStoppingCriteria`
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Given the usage of RWKV, decide to go with a tip in the doc rather than changing the `generate` and adding `RwkvStoppingCriteria` to the library. Adresses #23852

Comments:
_The documentation is not available anymore as the PR was closed or merged._
New mode in `model.generate`
Author: huangjy-pku
Pull request: no

Hello, is there any support for combining `contrastive_search` and `beam_search` in `model.generate` method?

My motivation is that when using `beam_search` I observe the outputs of model sometimes tend to produce repetitive text. I learned that `contrastive_search` was proposed to address the repetition issue. But when changing to `contrastive_mode`, the performance drops a lot, probably due to a lack of the beams. However, I checked the source code and found `contrastive_mode` triggers only when `num_beams == 1`.

Therefore, I wonder if I can combine the advantages of the two generation modes for better result?

Comments:
cc @gante 

Hey @huangjy-pku I believe the authors intended its use to be as an alternative to beam search, but I agree the two could be used together (beam search explores beam diversity, and contrastive search explores repetition avoidance).

Building it takes a significant time, and I haven't seen demand for it outside this issue -- we won't devote resources to it for now, as our bandwidth is limited. I would also oppose a PR, as it would add up to our long-term maintenance budget. If demand increases, I will revisit this decision :)

However, I have a short-term suggestion: have you tried the `repetition_penalty` argument in `generate`? ([docs](https://huggingface.co/docs/transformers/main/en/main_classes/text_generation#transformers.GenerationConfig.repetition_penalty))

Thanks. By the way, I tried increasing `repetition_penalty` in `generate` and it did help :)
beam_indices = None 
Author: Dongximing
Pull request: no

### System Info

Hi dear officer 

I find a bug, that is, if I use the 'force_words_id'  parameter in generate() function and set output_score = True. then  I want to get beam_indices, it will return None. but if I remove force_words_id. it will work. 
`
prompt = """Tell me some about Canada"""
input_tokenized_info = tokenizer(prompt, return_tensors="pt")
input_ids, attention_mask = input_tokenized_info['input_ids'], input_tokenized_info[
    'attention_mask']
input_ids = input_ids.to('cuda') 
attention_mask = attention_mask.to('cuda')
force_words = ["Canada"]
force_words_ids = tokenizer(force_words, add_special_tokens=False).input_ids
outputs = model.generate(input_ids=input_ids, attention_mask=attention_mask,num_beams =4,max_new_tokens=10,\
                         return_dict_in_generate=True,output_scores=True)
`
`print(outputs.beam_indices) tensor([[ 0,  0,  0,  0,  1,  0,  0,  3,  1,  0, -1, -1, -1, -1, -1]],
       device='cuda:0')`
But if add 'force_words_id' `outputs = model.generate(input_ids=input_ids, attention_mask=attention_mask,num_beams =4,max_new_tokens=10,\
                         return_dict_in_generate=True,output_scores=True,force_words_ids=force_words_ids)
print(outputs.beam_indices) None`



### Who can help?

@ArthurZucker , please give me some guidance, thanks 🙏 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

it should return indices. 

### Expected behavior

it should return indices. 

Comments:
cc @gante who is gonna be more familiar with this! 

Hey @Dongximing 👋 

The PR linked above (#25042) should fix it :) In a nutshell, the values were not being piped all the way to the output.
Generate - add beam indices output in contrained beam search
Author: gante
Pull request: yes

# What does this PR do?

Fixes #25000

This PR adds the `beam_indices` output to constrained beam search, just like in the other beam methods. The changes are (mostly) copy-paste from beam search, and they pipe `beam_indices` all the way to the output.

Script for reproducibility (didn't work before, works after these changes)
```py
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("gpt2")
model = AutoModelForCausalLM.from_pretrained("gpt2").to('cuda')

prompt = """Tell me some about Canada"""
input_tokenized_info = tokenizer(prompt, return_tensors="pt")
input_ids, attention_mask = input_tokenized_info['input_ids'], input_tokenized_info[ 'attention_mask']
input_ids = input_ids.to('cuda')
attention_mask = attention_mask.to('cuda')
force_words = ["Canada"]
force_words_ids = tokenizer(force_words, add_special_tokens=False).input_ids
outputs = model.generate(input_ids=input_ids, attention_mask=attention_mask,num_beams =4,max_new_tokens=10, return_dict_in_generate=True,output_scores=True, force_words_ids=force_words_ids)
print(outputs.beam_indices)
# Before: `None`
# After: `tensor([[ 0,  1,  1,  1,  0,  0,  2,  3,  3,  1, -1, -1, -1, -1, -1]], device='cuda:0')`
```

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix last models for common tests that are too big.
Author: sgugger
Pull request: yes

# What does this PR do?

This PR fixes the last batch of models that are too big for common tests. After it, only two classes override the common test to skip it:
- timm-backbone
- layoutlmv2
In the first case, it's because there is no timm model small enough to work (I did switch to the smallest resnet though) and in the second detectron2 does not let us configure a smaller backbone (I did try with the constant exposed by they don't seem to have any effect).

Comments:
_The documentation is not available anymore as the PR was closed or merged._
[docs] duplicate table of contents in perf_infer_gpu_one.mdx
Author: eenzeenee
Pull request: no

## Description
There are duplicate titles [**Requirements**] in `perf_infer_gpu_one.md` from line 51 and 117 causing an error occurs when moving the table of contents.

## Document / Language

`perf_infer_gpu_one.md` / en

## Suggestion

line 51

As is : 
```### Requirements```
To be : 
```### Requirements [[requirements-for-fp4-mixedprecision-inference]]```

line 117

As is : 
```### Requirements```
To be : 
```### Requirements [[requirements-for-int8-mixedprecision-matrix-decomposition]]```

Please let me know if I missed something in guideilnes.
Thank you in advance for your attention to it!

Comments:
I can't see this on the main branch, maybe this was fixed by the recent refactor of this guide?

Good catch! I don't think this [doc](https://huggingface.co/docs/transformers/main/en/perf_infer_gpu_one) was refactored in the recent PR (it was just moved), and I'm getting the same error on the `main` branch.

Would you like to open a PR with your proposed fix? 🤗 

Thanks for your feedbacks. I opened PR https://github.com/huggingface/transformers/pull/25066!
model.generate does not work when using a AlbertModel
Author: anujsahani01
Pull request: no

### System Info

- `transformers` version: 4.31.0
- Platform: Linux-5.15.109+-x86_64-with-glibc2.35
- Python version: 3.10.6
- Huggingface_hub version: 0.16.4
- Safetensors version: 0.3.1
- Accelerate version: not installed
- Accelerate config: not found
- PyTorch version (GPU?): 2.0.1+cu118 (False)
- Tensorflow version (GPU?): 2.12.0 (False)
- Flax version (CPU?/GPU?/TPU?): 0.7.0 (cpu)
- Jax version: 0.4.13
- JaxLib version: 0.4.13
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>


### Who can help?

@sgugger 

Apologies in advance if this is not a bug and instead a fault of my code.

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

## Step 1: Loading AI4Bharat/indic-bert model and tokenizer
```
# Load model directly
tokenizer = AutoTokenizer.from_pretrained("ai4bharat/indic-bert", src_lang="en_XX",tgt_lang="mr_IN")
model = AlbertModel.from_pretrained("ai4bharat/indic-bert", max_length = context_length)
```
## Step 2: Loading the dataset using datasets

```
from datasets import load_dataset
dataset = load_dataset("anujsahani01/English-Marathi", split = 'test[:20]')
```

## Step 3: Tokenizing the input data
```
# generate output
 model_inputs = tokenizer(data['english'], text_target =  data['marathi'],  max_length= max_length, padding = 'max_length', 
 truncation=True, return_tensors = 'pt').to(device)
 labels = model_inputs['labels']
 source = model_inputs['input_ids']
```

## Step 4:  Using .generate to make predictions
```
 preds = model.generate(**model_inputs, max_new_tokens = max_length)
```


### Expected behavior

Hey folks,
I was comparing different multilingual language models on basis of different evaluation metrics. Was not able to generate outputs using indic-bert

## Model
```ai4bharat/indic-bert```

## Error Message 
```
TypeError: The current model class (AlbertModel) is not compatible with `.generate()`, as it doesn't have a language model head.
```

Any of your inputs will be highly appreciated.
Thank You !

Comments:
I don't know what more you want me to say. As the error clearly states, this is not a model that supports `.generate`. ALBERT was trained on a masked language modeling objective so none of its variants are compatible with `.generate()`.
[`generate`]  Only warn users if the `generation_config`'s `max_length` is set to the default value
Author: ArthurZucker
Pull request: yes

# What does this PR do?

Updates the condition for raising a warning in generate to warn users if the `generation_config`'s `max_length` is set to the default value. 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@sgugger as discussed on Slack with @ArthurZucker: `max_new_tokens` can't be used to set a default `generation_config` that works well out of the box -- `max_length` does. As such, let's adapt the warning to enable us (and the users) to set good default arguments.
🌐 [i18n-KO] Translated `hpo_train.md` to Korean
Author: harheem
Pull request: yes

# What does this PR do?

Translated the `hpo_train.md` file of the documentation to Korean 😄 
Thank you in advance for your review!

Part of https://github.com/huggingface/transformers/issues/20179

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [x] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)
@wonhyeongseo, @keonju2, @harheem, @HongB1, @junejae

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)
 May you please review this PR?
@sgugger, @ArthurZucker, @eunseojo

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Can you just solve the conflict so we can merge this PR?
MPT
Author: zphang
Pull request: no
Labels: name: New model

### Model description

New LLM from MosaicML, 7B parameters. See: https://www.mosaicml.com/blog/mpt-7b

### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

https://huggingface.co/mosaicml/mpt-7b/tree/main

The model is already implemented in a HF/T compatible way, but has multiple source files for its model implementation, some components that aren't used in this current model, and most importantly, dependencies that aren't normally included in HF (e.g. einops, flash_attn).

Do the HF folks have a view on whether we would want to add those dependencies, or implement a vanilla version based only on existing requirements (in which case, it would arguably be easier to modify an existing LM implementation instead, rather than use MosaicML's implemenetation)?

Comments:
Can this model be implemented without the need for executing custom python files? I believe it's a huge security risk to allow these models to run their own python code. Yes it is up to the users to verify it's safety and take responsibility for their own actions; yet in the real world, most people won't scan through the python files that come with the models and the next majority might not understand code enough to verify it's safety.


Disregarding HF, does anyone have inference code to run the MPT with FasterTransformer?

We are currently talking about this integration with mosaic ML, hope to have updates on this soon! 

> Disregarding HF, does anyone have inference code to run the MPT with FasterTransformer?

Here is a script for converting a HuggingFace MPT checkpoint to FasterTransformer https://github.com/mosaicml/llm-foundry/blob/main/scripts/inference/convert_hf_mpt_to_ft.py

> We are currently talking about this integration with mosaic ML, hope to have updates on this soon!

@ArthurZucker Curious to know how the talks with MosaicML are going. :-)

Probably need to update the ticket description citing MPT-30B as well :-)

> We are currently talking about this integration with mosaic ML, hope to have updates on this soon!

@ArthurZucker Any updates on this issue?

Yes! We didn't receive a proper answer, I'll be taking this over! Will open a pr by tomorrow! 😉
[`MPT`] Add MosaicML's `MPT` model to transformers
Author: ArthurZucker
Pull request: yes

# What does this PR do?

Fixes #23174

First questions:
- [ ] Should we keep the nested config for attention and init configs? Pros: backward, Cons: not what we usually do, can't modify on the fly, harder to maintain
- [ ] Should we keep flash attention or go with better transformers 
- [ ] Do we want 100% backward compatibility 

# TODOS :
- [ ] Properly setup the config 
- [ ] Write a mapping to go from original mosaicml config to new config (since attribute names have to be changed)
- [ ] Design tests, clone the repo to `hf-internal-testing` since at the end we intend to remove the code from the hub. Test attention patterns , flash and trition
- [x] One model on file. 

# Notes : 
Tokenizer is the same as GPTNeoX, only has a fast version, adds sentinel tokens. We don't really need a custom config for this and should just always have these in the tokenizer config.


Comments:
_The documentation is not available anymore as the PR was closed or merged._

@sgugger we think the PR is ready for a review 🙏  ! The logits tests pass with tolerance `1e-12` between the model on the Hub and ours. There is nothing to do on the Hub as the current code is perfectly backward compatible with their config and weights.

```python
import torch
from transformers import AutoModelForCausalLM, MptForCausalLM, AutoTokenizer, MptForCausalLM

model_id = "mosaicml/mpt-7b"
tok = AutoTokenizer.from_pretrained(model_id)

model = MptForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, device_map={"":1}, load_in_4bit=True)
model_trust_remote_code = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, device_map={"":0}, load_in_4bit=True, trust_remote_code=True)

outputs_transformers = model(torch.LongTensor([[1, 2, 3, 4, 5]]).to(1), output_hidden_states=True)
outputs_trust_remote_code = model_trust_remote_code(torch.LongTensor([[1, 2, 3, 4, 5]]).to(0))

print(torch.allclose(outputs_transformers.logits, outputs_trust_remote_code.logits.to(1), atol=1e-12, rtol=1e-12))
>>> True
```

Currently we don't support advanced features such as triton attention or custom init, hence we advise super users that want to use this feature to load the trust_remote_code model if they want to benefit from these features

cc also @Narsil and @OlivierDehaene for TGI - I think things should work smoothly on your side

> cc also @Narsil and @OlivierDehaene for TGI - I think things should work smoothly on your side

MPT is already supported actually. (No triton, nor flash either, because of alibi)
🌐 [i18n-KO] Translated `perf_infer_cpu.md` to Korean
Author: junejae
Pull request: yes

<!-- PR의 제목은 "🌐 [i18n-KO] Translated `<your_file>.md` to Korean" 으로 부탁드립니다! -->
# What does this PR do?

Translated the `<your_file>.md` file of the documentation to Korean.
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [x] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에만 OSSCA 팀원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
<!-- Team OSSCA, may you please review this PR?  -->
@wonhyeongseo, @keonju2, @harheem, @HongB1, @junejae

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. 팀원들과 리뷰가 끝난 후에만 허깅페이스 직원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
May you please review this PR? @sgugger, @ArthurZucker, @eunseojo

Comments:
_The documentation is not available anymore as the PR was closed or merged._
[ `ForSequenceClassification`] Support `left` padding
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Update the computation of `sequence_lengths` that determines the index of the outpus to be pooled on models that use pooled inputs in such a way.
Adresses #24265

- [ ] add a common test to make sure whether you pad left or right, sequence outputs are the same

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_24979). All of your documentation changes will be reflected on that endpoint.

Test seems flaky:
```python 
FAILED tests/models/gptj/test_modeling_gptj.py::GPTJModelTest::test_pt_tf_model_equivalence - AssertionError: 0.14537059 not less than or equal to 1e-05 : outputs.logits: Difference between PyTorch and TF is 0.14537058770656586 (>= 1e-05).
FAILED tests/models/gptj/test_modeling_tf_gptj.py::TFGPTJModelTest::test_pt_tf_model_equivalence - AssertionError: 0.21959408 not less than or equal to 1e-05 : outputs.logits: Difference between torch and tf is 0.2195940762758255 (>= 1e-05).
```