noamaanMulla-03/transformers-issues huggingface dataset:
Switch from `cached_download` to `hf_hub_download` in remaining occurrences
Author: Wauplin
Pull request: yes

(somehow related to https://github.com/huggingface/huggingface_hub/pull/2317)

This PR get rids of the remaining `cached_download` in favor of the more robust `hf_hub_download`. Cached_download is quite legacy (replaced since Aug 2022) and I'd like to completely drop it at some point. The core part of `transformers`'s logic made the switch quite some time ago already.

Main change is basically to go from 

```py
json.load(open(cached_download(hf_hub_url(repo_id, filename, repo_type="dataset")), "r"))
```

to 

```py
json.loads(Path(hf_hub_download(repo_id, filename, repo_type="dataset")).read_text())
```

I used `Path(...).read_text()` to avoid opening the file without closing it.

**Note:** there is still one occurrence of `cached_download` in [./src/transformers/models/deprecated/van/convert_van_to_pytorch.py](https://github.com/huggingface/transformers/blob/main/src/transformers/models/deprecated/van/convert_van_to_pytorch.py) but since it is in a `deprecated/` folder, I did not bother maintaining it.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_31284). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

That was quick! Thanks for the review :hugs:
Having a function to verify if checkpoint is valid
Author: Bfault
Pull request: no
Labels: name: Feature request

### Feature request

Have a boolean function that returns true if a given checkpoint is valid and false if not

### Motivation

In case a machine can be down at any moment while training. If it saves the checkpoint when the machine goes down, last checkpoint can be corrupted. So we would like to check if the last checkpoint is valid and be deleted (or not) by the end user.

### Your contribution

I can work on this

Comments:
Thanks for opening this feature request, @Bfault! 

It might be difficult to have a boolean check for this, as there's only one way a checkpoint can be valid but an infinite number of ways it could be invalid. If a machine goes down, it's also unlikely we're be able to anticipate and run this in time. Perhaps we could have the option to run a clean up, when the machine has recovered, with a kind of try / except pattern on loading the checkpoints, working from the most recent, WDYT @muellerzr @SunMarc? 

> Perhaps we could have the option to run a clean up, when the machine has recovered, with a kind of try / except pattern on loading the checkpoints

@amyeroberts 
Do you mean something like this (#31287) ?
fix accelerate tests for roberta xl
Author: SunMarc
Pull request: yes

# What does this do ? 
This PR fixes the `no_split_modules` for roberta xl model introduced in this PR #31223. Also, we need to change model_split_percents, so that we made sure to have enough space on the first gpu for the embeddings. The tests are passing: 

```
tests/models/xlm_roberta_xl/test_modeling_xlm_roberta_xl.py::XLMRobertaXLModelTest::test_cpu_offload If you want to use `RobertaLMHeadModel` as a standalone, add `is_decoder=True.`
[2024-06-06 14:56:19,427] [INFO] [real_accelerator.py:191:get_accelerator] Setting ds_accelerator to cuda (auto detect)
If you want to use `RobertaLMHeadModel` as a standalone, add `is_decoder=True.`
If you want to use `RobertaLMHeadModel` as a standalone, add `is_decoder=True.`
PASSED
tests/models/xlm_roberta_xl/test_modeling_xlm_roberta_xl.py::XLMRobertaXLModelTest::test_disk_offload_bin If you want to use `RobertaLMHeadModel` as a standalone, add `is_decoder=True.`
If you want to use `RobertaLMHeadModel` as a standalone, add `is_decoder=True.`
If you want to use `RobertaLMHeadModel` as a standalone, add `is_decoder=True.`
PASSED
tests/models/xlm_roberta_xl/test_modeling_xlm_roberta_xl.py::XLMRobertaXLModelTest::test_disk_offload_safetensors If you want to use `RobertaLMHeadModel` as a standalone, add `is_decoder=True.`
If you want to use `RobertaLMHeadModel` as a standalone, add `is_decoder=True.`
PASSED
tests/models/xlm_roberta_xl/test_modeling_xlm_roberta_xl.py::XLMRobertaXLModelTest::test_model_parallelism If you want to use `RobertaLMHeadModel` as a standalone, add `is_decoder=True.`
If you want to use `RobertaLMHeadModel` as a standalone, add `is_decoder=True.`
If you want to use `RobertaLMHeadModel` as a standalone, add `is_decoder=True.`
PASSED
```

cc @ydshieh since you pinged me on this one

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_31288). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Enable HF pretrained backbones
Author: amyeroberts
Pull request: yes

# What does this PR do?

Enables loading HF pretrained model weights for backbones. 

```py
from transformers import MaskFormerConfig, MaskFormerForInstanceSegmentation

config = MaskFormerConfig(backbone="microsoft/resnet-50", use_pretrained_backbone=True)
model = MaskFormerForInstanceSegmentation(config)
```

* Updates documentation to show correct initialization 
* Removes check in `verify_backbone_config_arguments`
* Adds test for loading HF checkpoints 

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_31145). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Mark MobileNetV1ModelTest::test_batching_equivalence as flaky
Author: amyeroberts
Pull request: yes

# What does this PR do?

This test repeatedly fails. Marking as flaky to try and keep the CI more healthy. 

Opened issue for tracking: https://github.com/huggingface/transformers/issues/31257

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_31258). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Remove ConversationalPipeline and Conversation object
Author: Rocketknight1
Pull request: yes

Both the `ConversationalPipeline` and the `Conversation` object have been deprecated for a while, and are due for removal in `4.42`, which is the upcoming version. This PR removes them and every reference to them that I could find.

Fixes #31158

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_31165). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Please make sure there are very solid examples of how to do conversational from now on!

@osanseviero has been updating some docs in model repos, and I added the [sidebar tutorial](https://huggingface.co/docs/transformers/conversations) as well. If there's anywhere else you think it would make sense to add these examples, let me know!

Merging this - I think it's reasonably safe, but let's see if any more issues turn up before the next release.
interpolation added for TVP.
Author: bhuvanmdev
Pull request: yes

# What does this PR do?

Adds interpolation to the tvp model for the position and pad embeddings.

Addresses #30579 

## Who can review?
@amyeroberts


Comments:
@bhuvanmdev For the quality checks could you: 
* Rebase on main
* Make sure the most recent formatting libraries are installed: `pip install -e '.[quality]'`
* Format with `make fixup`

Done. Did I miss any changes to make? @amyeroberts
Rename test_model_common_attributes -> test_model_get_set_embeddings
Author: amyeroberts
Pull request: yes

# What does this PR do?

The method name is misleading - it is testing being able to get and set embeddings, not common attributes to all models

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_31321). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
i cannot find the code that transformers trainer model_wrapped by deepspeed , i can find the theory about  model_wrapped was wraped by DDP(Deepspeed(transformer model )) ,but i only find the code transformers model wrapped by ddp, where is the deepspeed wrapped ? thanks ^-^ 
Author: ldh127
Pull request: no

### Feature request

i cannot find the code that transformers trainer model_wrapped by deepspeed , i can find the theory about  model_wrapped was wraped by DDP(Deepspeed(transformer model )) ,but i only find the code transformers model wrapped by ddp, where is the deepspeed wrapped ? thanks ^-^ 

### Motivation

x

### Your contribution

x

Comments:
Hi @ldh127, thanks for raising an issue! 

This is a question best placed in our [forums](https://discuss.huggingface.co/). We try to reserve the github issues for feature requests and bug reports.

> 你好@ldh127，感谢您提出问题！
> 
> 这个问题最好放在我们的[论坛](https://discuss.huggingface.co/)中。我们尝试保留 github 问题以用于功能请求和错误报告。

thanks for your reply , but i can not create topic in that [forums](https://discuss.huggingface.co/),  can you give me the answer directly , thanks , i am so sorry to create the duplicate question  $_$ 

Hi @ldh127, what is the problem you're having posting on the forum?


last time it seems that i have no access to create the new question as a new user , but just i try it again, it is success . i had created my question ,  thanks @_@

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
KeyError Issue Reason
Author: nitinhariharan
Pull request: no

### System Info

Hi Team
I am trying to predict for a fine tuned roberta base model for text classification task

from torch.utils.data import Dataset
from tqdm.auto import tqdm
from transformers import AutoTokenizer, pipeline
import torch.multiprocessing as mp
from fastfit import FastFit

tokenizer = AutoTokenizer.from_pretrained("location/checkpoint-9375/", truncation=True, max_length=512)
model = FastFit.from_pretrained("location/checkpoint-9375/").to('cuda')
pipe = pipeline("text-classification", model=model, tokenizer=tokenizer, device=0, truncation=True, max_length=512)

for out in pipe(dataset, batch_size=18):
print(out)


this is the prediction code and if i give batches it throws out key error but with one batch size it is running fine

![image](https://github.com/huggingface/transformers/assets/85015974/578ab626-2bf5-4f9f-9804-9886e1520237)
can you help me with it why it is so

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

from torch.utils.data import Dataset
from tqdm.auto import tqdm
from transformers import AutoTokenizer, pipeline
import torch.multiprocessing as mp
from fastfit import FastFit

tokenizer = AutoTokenizer.from_pretrained("location/checkpoint-9375/", truncation=True, max_length=512)
model = FastFit.from_pretrained("location/checkpoint-9375/").to('cuda')
pipe = pipeline("text-classification", model=model, tokenizer=tokenizer, device=0, truncation=True, max_length=512)

for out in pipe(dataset, batch_size=18):
print(out)



### Expected behavior

It should give the correspodent highest score label that is fine tuned with

Comments:
Hi @nitinhariharan, without access to the model or dataset, it's very hard for us to know what the issue is, as we can't replicate on our side. 

On the shared screenshot, there's an error message saying this model type is not supported by the text-classification pipeline 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Paligemma: fix causal mask
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

Currently on `main` Paligemma fails when I try to do inference, because the causal mask is not inverted. This PR inverts the 4D causal mask. 


Comments:
Hi @zucchini-nlp, thanks for working on this!
I guess it is a duplicate of #31207

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_31312). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Ah oke, great. Closing this one :)
Cannot convert llama 3 model to hf
Author: Bedoshady
Pull request: no

### System Info

I have an intel based computer with 32 gb of ram

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

I have downloaded llama 3 model from meta and have .pth file, I tried running convert_llama_weights_to_hf.py but I get `Fetching all parameters from the checkpoint at H:\Downloads\llama3-main\Meta-Llama-3-8B.
Loading the checkpoint in a Llama model.
Loading checkpoint shards: 100%|███████████████████████████████████████████████████████| 33/33 [00:04<00:00,  7.07it/s]
Saving in the Transformers format.
Traceback (most recent call last):
  File "H:\Downloads\transformers\src\transformers\models\llama\convert_llama_weights_to_hf.py", line 407, in <module>
    main()
  File "H:\Downloads\transformers\src\transformers\models\llama\convert_llama_weights_to_hf.py", line 396, in main
    write_model(
  File "H:\Downloads\transformers\src\transformers\models\llama\convert_llama_weights_to_hf.py", line 313, in write_model
    shutil.rmtree(tmp_model_path)
  File "H:\tools\MiniConda2\envs\tg\lib\shutil.py", line 750, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "H:\tools\MiniConda2\envs\tg\lib\shutil.py", line 620, in _rmtree_unsafe
    onerror(os.unlink, fullname, sys.exc_info())
  File "H:\tools\MiniConda2\envs\tg\lib\shutil.py", line 618, in _rmtree_unsafe
    os.unlink(fullname)
PermissionError: [WinError 5] Access is denied: 'H:\\Downloads\\llama3-main\\Meta-Llama-3-8B\\tmp\\pytorch_model-1-of-33.bin'` what should I do?

### Expected behavior

The model should be converted to hf without errors

Comments:
check on your permissions to your file . might work as it says Access is denied . use CHMOD and provide read and write access for the specif model path.

I am running this on windows 10 using python in miniconda.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
AutoModal how to enable TP for extremly large models?
Author: MonolithFoundation
Pull request: no

Hi, I have 8V100s, but a single one can not fit InternVL1.5 model which has 28B parameters.

So that, I just wonder if I can fit all of them into 8 V100 with TP?

I found that Deepspeed can be used to do tensor parallel like this:

```
# create the model
if args.pre_load_checkpoint:
    model = model_class.from_pretrained(args.model_name_or_path)
else:
    model = model_class()
...

import deepspeed

# Initialize the DeepSpeed-Inference engine
ds_engine = deepspeed.init_inference(model,
                                 tensor_parallel={"tp_size": 2},
                                 dtype=torch.half,
                                 checkpoint=None if args.pre_load_checkpoint else args.checkpoint_json,
                                 replace_with_kernel_inject=True)
model = ds_engine.module
output = model('Input String')
```

I didn't succeed because of it just support built in model which can be imported, but for custom model which have to `fromPretrained` it does support.

But as I mentioned at start, my V100 will OOM when load model.

Does there any convenient way to loading hf model which is customized with tp enable ?

Comments:
Hi @MonolithFoundation, thanks for raising an issue! 

This is a question best placed in our [forums](https://discuss.huggingface.co/). We try to reserve the github issues for feature requests and bug reports.

This could be treated as feature, as I didn't found anywhere has doc about this, as such it can be treated as unspported

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
ChatGLM3-6b测试模型时报错AttributeError: can't set attribute
Author: padsasdasd
Pull request: no

### System Info

- `transformers` version: 4.36.0
- Platform: Linux-5.15.0-101-generic-x86_64-with-glibc2.17
- Python version: 3.8.10
- Huggingface_hub version: 0.22.2
- Safetensors version: 0.4.3
- Accelerate version: 0.29.3
- Accelerate config:    not found
- PyTorch version (GPU?): 2.0.0+cu118 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

xtuner chat /root/autodl-tmp/add --prompt-template default


Traceback (most recent call last):
  File "/root/ChatGLM3/xtuner/xtuner/tools/chat.py", line 491, in <module>
    main()
  File "/root/ChatGLM3/xtuner/xtuner/tools/chat.py", line 237, in main
    tokenizer = AutoTokenizer.from_pretrained(
  File "/root/miniconda3/lib/python3.8/site-packages/transformers/models/auto/tokenization_auto.py", line 774, in from_pretrained
    return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
  File "/root/miniconda3/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 2028, in from_pretrained
    return cls._from_pretrained(
  File "/root/miniconda3/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 2260, in _from_pretrained
    tokenizer = cls(*init_inputs, **init_kwargs)
  File "/root/.cache/huggingface/modules/transformers_modules/add/tokenization_chatglm.py", line 108, in __init__
    super().__init__(padding_side=padding_side, clean_up_tokenization_spaces=clean_up_tokenization_spaces,
  File "/root/miniconda3/lib/python3.8/site-packages/transformers/tokenization_utils.py", line 363, in __init__
    super().__init__(**kwargs)
  File "/root/miniconda3/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 1602, in __init__
    super().__init__(**kwargs)
  File "/root/miniconda3/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 861, in __init__
    setattr(self, key, value)
AttributeError: can't set attribute

### Expected behavior

模型合并已经成功了，测试时罚下一直出现错误，求大佬解答。

Comments:
Hi @padsasdasd, thanks for opening an issue. 

Could you provide a reproducer, an explanation of what you're trying to do, and the expected behaviour? Without these there isn't really anything we can do to help. 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Ability to split special tokens with fast tokenizer
Author: scruel
Pull request: no
Labels: name: Feature request

### Feature request

The following workaround will work for same goal:
```
tokenizer._tokenizer.encode_special_tokens=False
```

but the following won't work:
```
tokenizer = AutoTokenizer.from_pretrained(
"meta-llama/Llama-2-7b-hf", local_files_only=True, split_special_tokens=True)
```

### Motivation

`split_special_tokens` param now only supports the slow tokenizer

### Your contribution

/

Comments:
Already fixed with [deba765](https://github.com/huggingface/transformers/commit/deba7655e6e54fb885e79204dec9f767393dd2df)
What if past_key_values is in model_kwargs but is None
Author: estelleafl
Pull request: no

https://github.com/huggingface/transformers/blob/4c6c45ba138202f42582b5cea98126af87195a95/src/transformers/generation/utils.py#L1317

This line fails for me when past_key_values is in model_kwargs but is None. Line 1321 raises an error 
Could you advice?

Thank you

Comments:
cc @gante @zucchini-nlp 

Same as #31017