noamaanMulla-03/transformers-issues huggingface dataset:
isses with converting codellama model with convert_llama_weights_to_hf.py
Author: ilang6
Pull request: no

### System Info

Hi .
I'm trying to convert the new codellama 7B model with convert_llama_weights_to_hf.py 
but I'm getting the following error:
`ValueError: Trying to set a tensor of shape torch.Size([32016, 4096]) in "weight" (which has shape torch.Size([32000, 4096])), this look incorrect.`
I'm running the script on macos 13.0 (22A380) and python 3.9 with Transformers version 4.32.0.

Thank you in advance. 

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

This was the code I ran.
`python src/transformers/models/llama/convert_llama_weights_to_hf.py --input_dir /Volumes/Extreme_SSD/llama2_code/CodeLlama-7b --output_dir /Volumes/Extreme_SSD/llama2_code/CodeLlama-7b/converted --model_size 7B`

### Expected behavior

At the end of the process, I should have the model in a bin format.
With config.json and tokenizer file.

Comments:
Hey ! This is expected, the conversion script was updated to support code llama see #25740.
Make sure you are running the script on main 

Thank you for your help.
I tried the updated script again and everything thing worked just fine.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
[docs] removed MaskFormerSwin and TimmBackbone from the table on index.md
Author: MKhalusova
Pull request: yes

# What does this PR do?

The PR addresses  #25967:

`MaskFormerSwin` and `TimmBackbone` should not be listed in the framework support table on the index.md page because these models are backbones and so not meant to be loaded and used on their own. Instead, they define architectures which can be loaded using the AutoBackbone API. See the [comment](https://github.com/huggingface/transformers/issues/25967#issuecomment-1717818649) in the issue. 

This PR updates the table, and the script that builds the table so that these models are excluded when building the table. 


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix MusicGen logging error
Author: osanseviero
Pull request: yes

# What does this PR do?

Issue: #26369 

Loading

```python
pipe = pipeline("text-to-audio", model="facebook/musicgen-small")
data = pipe("latin salsa with violins")
```

at the moment, throws a large error


## Who can review?

@ylacombe @sanchit-gandhi 



Comments:
_The documentation is not available anymore as the PR was closed or merged._
Add Russian localization for README
Author: qweme32
Pull request: yes

# What does this PR do?

Add Russian localization for README.

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests)

Comments:
cc @stevhliu 

awesome 

any comments?

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26208). All of your documentation changes will be reflected on that endpoint.

thx, added @stevhliu
chorse: change numpy to jax numpy
Author: pphuc25
Pull request: yes

Hi,
I just do a simple remove numpy and change to jax numpy, seen at now jax is more robust and can replace to numpy.
I would like to cc @stevhliu to review my PR.

Comments:
Makes sense to me to use the built-in NumPy module, but I'm not too familiar with how robust JAX is compared to NumPy. What do you think @LysandreJik ?

From a quick look this looks OK, ok for you @sanchit-gandhi ?

This is not such a great idea! Let me explain why: any operations run as `numpy` code will be executed on the CPU. Any operation run as `jax.numpy` code will be executed on the **accelerator device** (GPU or TPU). Note that this behaviour in JAX is different to PyTorch: by default, JAX code will be run on the accelerator device (if available), whereas in PyTorch it will be the CPU (unless specified otherwise).

This is significant because JAX uses [*asynchronous dispatch*](https://jax.readthedocs.io/en/latest/async_dispatch.html) to coordinate code between the CPU and accelerator device. If the accelerator device is running `jax.numpy` code, the CPU is free to carry on and run subsequent `numpy` code. It is not blocked by the accelerator running `jax` code, and can in fact already execute any 'future' code. This is useful in ML applications, since the accelerator can stay busy running the modelling code (forward and backward pass), whilst the CPU can go ahead and prepare the next batch of data ahead of time.

If we change our data preparation to use `jax.numpy` code, then we execute it on the accelerator device. This means we have to wait until the accelerator is free before running the data preparation, so we loose our asynchronous dispatch advantage.

An easy rule to keep in mind when writing JAX code is that **only the modelling code should be in JAX**. Everything else should be in NumPy. This way, we ensure only the modelling code is executed on the accelerator device, and everything else on CPU, thus maximising the async dispatch potential.

Let me know if you have any questions @pphuc25 - more than happy to answer!

Hmm, that's sound great insight, this is the first time I hear about it, very helpful information for me, thank you so much
🌐 [i18n-KO] Translated  `audio_classification.mdx` to Korean
Author: gabrielwithappy
Pull request: yes

<!-- PR의 제목은 "🌐 [i18n-KO] Translated `<your_file>.mdx` to Korean" 으로 부탁드립니다 -->
# What does this PR do?

Translated the `audio_classification.mdx` file of the documentation to Korean.
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179
<!-- 메인 이슈에 기록이 남아요! 가짜연구소 리포를 사용해 연습하실때는 제거해주시면 감사하겠습니다! :smile: -->

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [ ] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에만 가짜연구소 팀원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
Team PseudoLab, may you please review this PR? @0525hhgus, @KIHOON71, @sim-so, @gabrielwithappy, @HanNayeoniee, @wonhyeongseo, @jungnerd

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. 가짜연구소 팀원들과 리뷰가 끝난 후에만 허깅페이스 직원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
<!-- @sgugger, @ArthurZucker, @eunseojo May you please review this PR? -->

Comments:
Team PseudoLab, may you please review this PR? @0525hhgus, @kihoon71, @sim-so, @gabrielwithappy, @HanNayeoniee, @wonhyeongseo, @jungnerd

@wonhyeongseo 
I updated your suggestion!
Thank you for your feedback. :-)

> 좋은 번역 감사합니다 😀
> 오디오 관련 용어를 꼼꼼하게 번역해주셔서 수월하게 이해했습니다 👍
> 아래 리뷰를 남기니, 참고 부탁드립니다!

꼼꼼한 리뷰 너무감사 드립니다!!🤗

@sgugger, @ArthurZucker, @eunseojo 
May you please review this PR? 

cc @stevhliu for review :)

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26200). All of your documentation changes will be reflected on that endpoint.
[LLM Tokenizer] Tokenizer loads too slowly
Author: Sakurakdx
Pull request: no

When I use LLM such as LLAMA's tokenzier, its loading speed is very slow, even taking half an hour. I would like to ask how long is the normal loading time and is there any way to speed up the loading?

Comments:
Hey @Sakurakdx, by "loading speed", are you talking about the model or the tokenizer? Would you have a reproducible code snippet I could try to see what you mean?

Sorry, my statement may not be accurate. The following is a clarification: When I use `AutoTokenizer` to load the tokenzier of LLAMA2, I need to wait for a long time.

```python
tokenizer = AutoTokenizer.from_pretrained("chinese-llama-2-7b", fast=True, trust_remote_code=True)
```

Do you mind trying to install from main to see if it fixes your issue? You can do so with:
```
pip uninstall transformers
pip install git+https://github.com/huggingface/transformers
```

Thank you this is super helpful!
Add Nougat
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR adds Nougat, a [Donut](https://huggingface.co/docs/transformers/model_doc/donut) model trained to transcribe scientific PDFs into an easy to read Markdown format.

To do:

- [x] remove cv2 dependency of `NougatImageProcessor` - or use `is_cv2_available`
- [x] make sure code that relies on cv2, levensthein, nltk is tested (cc @ydshieh) 
- [x] add image processor tests
- [x] add slow tokenizer => not needed
- [x] add tokenizer tests
- [x] add integration test in tests/models/vision_encoder_decoder

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Remaining to do's:

- [x] remove cv2 dependency
- [x] add init tokenizer
- [x] fix (*args, **kwargs) of processor to explicitly define the arguments
- [x] transfer checkpoints

Pinging @ArthurZucker for final review :) models are on the hub now: https://huggingface.co/models?other=nougat

Sure! Will review asap 

@ArthurZucker FYI when rebasing on main, it looks like https://github.com/huggingface/transformers/pull/23909 made this test fail:

```
tests/models/nougat/test_tokenization_nougat.py::NougatTokenizationTest::test_encode_decode_with_spaces - ValueError: This tokenizer class has no tokenizer to be tested.
```
This is because Nougat only has a fast tokenizer. Should [this line](https://github.com/huggingface/transformers/blob/914771cbfe02c423b8361f341dbd6d6203889060/tests/test_tokenization_common.py#L980) be updated to test a fast one if it's available, or should be skip the test?

Should be skipped as we do with bloom I think! 

@NielsRogge has no merge permission. Please ping one of us to merge :-)

Done, feel free to merge

Thanks @NielsRogge and @molbap for this new model 🔥
feat: add enable_gradient_checkpoint for roberta
Author: pphuc25
Pull request: yes

# What does this PR do?

While reviewing the "run_mlm_flax.py" file, I encountered an issue when attempting to execute the "model.enable_gradient_checkpointing()" function. Upon further investigation, I observed that there was no "enable_gradient_checkpointing" available. As a solution, I configured FlaxRobertaForMasked to include the "enable_gradient_checkpoint" feature.

I would like to cc @sanchit-gandhi to review my code

Comments:
Hey @pphuc25 - I believe that the `enable_gradient_checkpointing()` method is working for Flax Roberta. See code snippet below that demonstrates:
```python
from transformers import FlaxRobertaForMaskedLM

model = FlaxRobertaForMaskedLM.from_pretrained("hf-internal-testing/tiny-random-roberta", from_pt=True)
print("Gradient checkpointing before enabling: ", model.module.gradient_checkpointing)

model.enable_gradient_checkpointing()
print("Gradient checkpointing after enabling: ", model.module.gradient_checkpointing)
```
**Print Output:**
```
Gradient checkpointing before enabling:  False
Gradient checkpointing after enabling:  True
```

What behaviour are you observing that makes you think otherwise?

Oh, it my mistake, I have download with not the lastest version of transformers, new update have fix this, sorry very much.
docs: change dataset name
Author: pphuc25
Pull request: yes

# What does this PR do?

In the sample code for running a language model in the Flax framework, I believe that "unshuffled_deduplicated_no" is not a suitable example due to its size, which consists of 3.18M samples. Loading such a large dataset can be time-consuming, especially for beginners. Therefore, I suggest changing the dataset to something smaller, like 1M samples, for testing purposes. As a result, I have modified the data source to "unshuffled_deduplicated_vi"


Comments:
Tokenizer and unicode problems
Author: bombel28
Pull request: no

### System Info

UnicodeEncodeError: 'charmap' codec can't encode character '\u016b' in position 13: character maps to <undefined>

This is new...

Next new issue: when using hardcut in oobabooga to use eos token, tokenizer can't handle two eos (or bos)-tokens.

All issues belongs to llama model (llama-2-13B here)

EDIT: seems like wrong encoding is used. Earlier version uses correctly utf-8
EDIT again: converting training data to UTF-8 BOM solves the problem with unicode.
### Who can help?

@ArthurZucker 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

1. Use a raw text training file in UTF8 in different languages like czech, french
2. use many linefeeds at once (six or more)
3. activate Add EOS token in oobabooga and set Hard Cut String to \n\n\n (default)

Start training on an early llama-2 model with basic / original tokenizer

Two problems will cause to don't start training.

I solved the eos-token-problem meanwhile this way for me:
in tokenization_llama.py, i added /changed this in     def tokenize(self, text: "TextInput", **kwargs):

        if len(tokens) > 1 and tokens[0] == SPIECE_UNDERLINE and tokens[1] in self.all_special_tokens:

            # Log the text to a file
            with open("error_log.txt", "a") as log_file:
                log_file.write(text + "\n")
			
            tokens = tokens[1:]
        return tokens

So the last unusable token is removed, and the line is logged for further error handling in training data.



### Expected behavior

as in old version before, this problems did not occure on same training data

Comments:
Sorry could you explain what « obabooga » is? 

text generation web UI, sorry for typo, should be oobabooga. For error log, I just had to change one more time....

```
        if len(tokens) > 1 and tokens[0] == SPIECE_UNDERLINE and tokens[1] in self.all_special_tokens:

            # Log the text to a file
            with open("error_log.txt", "a", encoding='utf-8') as log_file:
                log_file.write(text + "\n")
			
            tokens = tokens[1:]
        return tokens

```
But as using UTF-8 BOM, encoding seems to work.

seems to be solved.
Add image-to-image pipeline
Author: NielsRogge
Pull request: no
Labels: name: Good First Issue

### Feature request

Would be great to add an `image-to-image` pipeline, to handle tasks like image super resolution.

We do support Swin2SR in the library for this purpose: https://huggingface.co/docs/transformers/main/model_doc/swin2sr#transformers.Swin2SRForImageSuperResolution

This can be implemented similar to [other pipelines](https://github.com/huggingface/transformers/tree/main/src/transformers/pipelines). For an example PR that added a pipeline, see https://github.com/huggingface/transformers/pull/11598.

### Motivation

Would be great to do image-to-image tasks in 2 lines of code.

### Your contribution

I can help in assisting a contributor (cc @Narsil)

Comments:
Hi @NielsRogge, i would like to help with that, if that's ok.

Great, feel free to start opening a PR

on it!

Hi @NielsRogge I would want to work on this.

Hi, @NielsRogge I want to contribute to this.

I'd love to contribute!

/attempt

Is this issue resolved?

Yes, will close.
Update llama error message (decoder_input_ids --> input_ids)
Author: tmabraham
Pull request: yes

Change:
```python 
if input_ids is not None and inputs_embeds is not None:
            raise ValueError("You cannot specify both decoder_input_ids and decoder_inputs_embeds at the same time")
```        
to
```python 
if input_ids is not None and inputs_embeds is not None:
            raise ValueError("You cannot specify both input_ids and inputs_embeds at the same time")
```        
Similarly change:
```python
else:
    raise ValueError("You have to specify either python_input_ids or python_inputs_embeds")
```
to 

```python
else:
    raise ValueError("You have to specify either input_ids or inputs_embeds")
```

Comments:
oops somehow old commits got pulled in, closing and will fix
Model to be adjusted
Author: L4EPITTMSU
Pull request: no

https://github.com/huggingface/transformers/blob/6b82d936d49956ba7b43c5ee590f4868de373b65/src/transformers/models/big_bird/modeling_big_bird.py#L977

getting this issue: 
/opt/software/Python/3.6.4-foss-2018a/lib/python3.6/site-packages/transformers/models/big_bird/modeling_big_bird.py:978: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
  * num_indices_to_pick_from

Comments:
If you're using a recent version of the transformers library, it should eventually be updated to address this issue.
This warning is indicating that the behavior of floordiv is deprecated and will change in a future version of PyTorch.
import torch

# Instead of this
result = torch.floordiv(a, b)

# Use this
result = torch.div(a, b, rounding_mode='floor')

It's not an issue but a warning, and since you do not have a reproducer I can't reproduce it. Given the warning, I am not entirely sur there anything to do here, we are already using :
> for actual floor division, use torch.div(a, b, rounding_mode='floor').

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Safety check low_cpu_mem_usage when in 4bit or 8bit
Author: BramVanroy
Pull request: yes

Currently, the check is only done for `is None` after which `low_cpu_mem_usage ` is set to True. But that can lead to unexpected behavior, especially if it is set to False. The result would be a hard-to-debug error trace (cf. related issue). This PR makes sure that when in 4bit or 8bit, `low_cpu_mem_usage` is set to True regardless of its current value and it will log a warning to the user to notify them of the change.

closes https://github.com/huggingface/accelerate/issues/1858

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_25603). All of your documentation changes will be reflected on that endpoint.

cc @younesbelkada 

Running `make style` should fix the CI 😉 

The history of the PR is a bit messed up 😓 you are gonna need to rebase / force push to make it work

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
fixing bad docstring example
Author: lucas-spangher-1
Pull request: yes

# What does this PR do?

Removed input arguments from the docstring example of AutoformerForPrediction.forward() in order to get it working out of the box. 

Detailed in this issue: https://github.com/huggingface/blog/issues/1382

Fixes # (issue)

## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [X] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @sgugger

Integrations:

- deepspeed: HF Trainer/Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam

Documentation: @sgugger, @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: @sgugger
- TensorFlow: @Rocketknight1

 -->


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_25600). All of your documentation changes will be reflected on that endpoint.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Fix DeepSpeed issue with Idefics
Author: HugoLaurencon
Pull request: yes

# What does this PR do?

This PR fixes a bug when trying to do inference with Idefics with DeepSpeed Zero-3.
The model was loaded correctly, but the shapes were found to be incorrect using DS at a step in `IdeficsDecoupledLinear` during inference.

I checked on different prompts that the output is the same without using DeepSpeed.

## Who can review?

@pacman100


Comments:
_The documentation is not available anymore as the PR was closed or merged._