noamaanMulla-03/transformers-issues huggingface dataset:
Update hyperparameter_search.py
Author: pacman100
Pull request: yes

# What does this PR do?
1. PR #24384 results in many tests failing related to HP Search. https://huggingface.slack.com/archives/C02CH2YP4EQ/p1687823783572709
2. Error being `tests/deepspeed/test_deepspeed.py::TrainerIntegrationDeepSpeed::test_hyperparameter_search
(line 122)  TypeError: is_available() missing 1 required positional argument: 'self'` when `default_hp_search_backend` is called.
3. This PR fixes it.

Tested this via:
```
export CUDA_VISIBLE_DEVICES="0"
export RUN_SLOW="yes"
export LOGLEVEL=INFO

cd transformers

pytest -sv tests/deepspeed/test_deepspeed.py::TrainerIntegrationDeepSpeed::test_hyperparameter_search
```

Comments:
_The documentation is not available anymore as the PR was closed or merged._
LLamaTokenizer padding_side='left'
Author: davidmrau
Pull request: no

Trying to Fine-tune the LLama model I encountered that the LLama tokenizer adds padding tokens on the left side by default. I expect the padding to be added to the right side. Is this a bug or supposed to be like this?



https://github.com/huggingface/transformers/blob/6fe8d198e3e30b709a7d233240f273804b886dcc/src/transformers/models/llama/tokenization_llama_fast.py#L85C5-L85C5

Comments:
Please ask questions like this on the [forums](https://discuss.huggingface.co/) as we keep issues for bugs and feature requests only.

All decoder models use padding on the left side since they can't properly generate the next token after a sentence if it ends with pad tokens. Llama is a decoder model, so follows the same rule.
Update `huggingface_hub` commit sha
Author: ydshieh
Pull request: yes

# What does this PR do?

To use `timeout=10.0` in [this commit](https://github.com/huggingface/huggingface_hub/commit/e4a419bf6bbaa95d14704cc781d3e81a49cef413).

The goal is to make sure the fix from infra team really works. But once verified, we can keep it as `10.0`.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_24527). All of your documentation changes will be reflected on that endpoint.
[Bug]Non-robust directory splitting and detection at get_cached_module_file
Author: Tpinion
Pull request: no

### System Info

- `transformers` version: 4.30.2
- Platform: Windows-10-10.0.22621-SP0
- Python version: 3.10.12
- Huggingface_hub version: 0.15.1
- Safetensors version: 0.3.1
- PyTorch version (GPU?): 2.0.1+cu118 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

1. I specified the correct model directory path under windows, but the library doesn't seem to find it correctly.
Just like this:
```python
# Directory THUDM/chatglm2-6b is existed at same directory with code.
tokenizer = AutoTokenizer.from_pretrained("THUDM/chatglm2-6b", trust_remote_code=True)
```
2. Then I found the reason was that the `submodule` was not splitted correctly.
**Source Code At**: `transformers/dynamic_module_utils:get_cached_module_file`
```python
# line: 235 This code returns true on both linux and windows.
is_local = os.path.isdir(pretrained_model_name_or_path)
if is_local:
    # line: 237 But this code cann't split path correctly, because os.path.sep is "\\" rather than "/".
    submodule = pretrained_model_name_or_path.split(os.path.sep)[-1]
```
3. So I solved the problem by modifying my path.
```python
tokenizer = AutoTokenizer.from_pretrained("THUDM\\chatglm2-6b", trust_remote_code=True)
```

### Expected behavior

Even on windows code, it is common to specify paths by forward slashes. Although it may not be a standard paradigm.

So I think there are two possible approaches:
1. Use a more robust way to split `submodule`, such as the pathlib.
2. Explicitly throws a warning asking windows users to pass in a standard windows path.



Comments:
Are you really certain `os.path.isdir(pretrained_model_name_or_path)` returns `True` with a path containing `/` on Windows? This seems really weird to me.

> Are you really certain `os.path.isdir(pretrained_model_name_or_path)` returns `True` with a path containing `/` on Windows? This seems really weird to me.

yes~
This is the screenshot of the debugging I just did on vscode.
![image](https://github.com/huggingface/transformers/assets/27395365/b623865f-a0cc-45db-9ed0-4f3f3a9c8517)


Annoying (that it's super inconsistent like this). Will look at this later today and try to come up with a fix!

Could you try if the PR mentioned above fixes your issue?

> Could you try if the PR mentioned above fixes your issue?

No problem~

> Could you try if the PR mentioned above fixes your issue?

Problem has been solved by this PR! Should I close this issue now?

It will be closed auto-magically by GitHub when the PR is merged :-)
Find module name in an OS-agnostic fashion
Author: sgugger
Pull request: yes

# What does this PR do?

Splitting on `os.path.sep` to get the filename is not fully-compatible on Windows as the OS recognizes a local path written as `folder/file.ext`, but `os.path.sep` is `\\`. Hopefully this way is better.

Fixes #24517

Comments:
_The documentation is not available anymore as the PR was closed or merged._

https://github.com/huggingface/transformers/blob/8a968e572c335d82c87dc3708fe0a9da79664d64/src/transformers/dynamic_module_utils.py#L274
This line should be change also~

Good point, it's added.
[Mask2Former] Remove SwinConfig
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR removes what was probably a leftover from the Mask2Former PR, the model works without requiring those lines of code.

Fixes #24244

Comments:
_The documentation is not available anymore as the PR was closed or merged._

gently ping @amyeroberts to see if we could merge 🤗
T5 zero-shot classification pipeline
Author: peregilk
Pull request: no

### Feature request

The current zero-shot classification pipeline support models like BERT and BART, but there does not seem to be any support for T5. 

I notice that the new [mT5-mnli from Alan Turing Institute](https://huggingface.co/alan-turing-institute/mt5-large-finetuned-mnli-xtreme-xnli) has code for extracting output probabilites and mapping this to the mnli entailment/contradiction, but they are not able to integrate it into the pipeline.

I am not sure what is exactly missing for this to be in place. What needs to be done in Transformers and how should the output from the model be adapted?

@patrickvonplaten 
@anton-l 

### Motivation

The medium/large T5 models has very impressive zero-shot abilities, and the combination of finetuning on NLI-tasks ([Yin et.al](https://arxiv.org/abs/1909.00161)) and the classification pipeline is a great way to use NLP for easy classification tasks. 

### Your contribution

My contribution will depend on what needs to be done. I am finetuning several models on MNLI, and can in any case contribute active in testing.

Comments:
Hi @peregilk 

T5 is not supported in zero-shot classification pipeline because it does not have a sequence classification head. With T5 the seq classification problem is formulated as text-to-text generation problem which is not possible to support in this zero-shot pipeline. 

@patil-suraj 
Thanks for the answer!

Just because I am trying to get a better understanding of this: Is there a fundamental difference between the output probabilities from a seq classification head and the output probabilites generated by the code at the bottom of this page: https://huggingface.co/alan-turing-institute/mt5-large-finetuned-mnli-xtreme-xnli ? Or is this simply related to the way the pipelines are made?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

> Hi @peregilk
> 
> T5 is not supported in zero-shot classification pipeline because it does not have a sequence classification head. With T5 the seq classification problem is formulated as text-to-text generation problem which is not possible to support in this zero-shot pipeline.

Hi, thank you for the answer. I am interested in using t5 for classification. Are there any examples on fine-tuning? How should we interpret class tokens in the decoder output? If there's documentation on token IDs for each classification task it would be very helpful.
Document QA Pipeline Suddenly Down?
Author: FFFiend
Pull request: no

### System Info

Transformers 4.29.2, MacOS, Python 3.9.6 64-bit,
I just have this piece of code:
```
    query_pipeline = pipeline(
        "document-question-answering", 
        model="impira/layoutlm-document-qa"
    )
```
and on this Python version it gives me: 
```
RuntimeError: Failed to import transformers.models.layoutlm.modeling_tf_layoutlm because of the 
following error (look up to see its traceback):
No module named 'keras.engine'
```
when it was working perfectly yesterday.


On Version: 4.30.2, MacOS, Python 3.10.12 64-bit, I get a different error somehow
```
  File "/opt/homebrew/lib/python3.10/site-packages/transformers/pipelines/__init__.py", line 988, in pipeline
    return pipeline_class(model=model, framework=framework, task=task, **kwargs)
  File "/opt/homebrew/lib/python3.10/site-packages/transformers/pipelines/document_question_answering.py", line 145, in __init__
    self.check_model_type(MODEL_FOR_DOCUMENT_QUESTION_ANSWERING_MAPPING)
NameError: name 'MODEL_FOR_DOCUMENT_QUESTION_ANSWERING_MAPPING' is not defined
```



### Who can help?

_No response_

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Steps to reproduce:
Paste the code snippet and run the Python script.

### Expected behavior

I expect the pipeline to work properly as intended. 

Comments:
Please follow the issue template and paste the result of `transformers-cli env`. It looks like you don't have PyTorch installed in your environment but it's hard to be sure without that. The document QA pipeline requires torch, it doesn't have an implementation in TensorFlow.

Resolved!! Thank you 😄
Make classifier backbone dynamic maskformer, mask2former
Author: tanzzilaalam
Pull request: no
Labels: name: Feature request

### Feature request

Make classifier backbone dynamic maskformer, mask2former. Currently it only supports swin transformer. 

link example:  https://github.com/huggingface/transformers/blob/main/src/transformers/models/mask2former/modeling_mask2former.py#L1393


### Motivation

If this feature request successful, it will help to benchmark for new clasifier easily.

Comments:
Thank you for solving this issue. But there is still problem.
The supported backbones variable hinders to allow any backbones other than
swin.
Please check this line. I had to write child class and had to modify this
line to solve this problem.

https://github.com/huggingface/transformers/blob/main/src/transformers/models/mask2former/configuration_mask2former.py#L124


On Tue, 27 Jun 2023, 11:34 pm Sylvain Gugger, ***@***.***>
wrote:

> Closed #24244 <https://github.com/huggingface/transformers/issues/24244>
> as completed via #24259
> <https://github.com/huggingface/transformers/pull/24259>.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/24244#event-9655252543>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AQSQGCPK433TDSJZKGBL7VDXNMKRDANCNFSM6AAAAAAZEYRWOA>
> .
> You are receiving this because you authored the thread.Message ID:
> ***@***.***>
>


cc @amyeroberts 

Hi @tanzzilaalam, you're completely right. I've opened a PR - #24532 -  which should resolve this and allow you to pass in any `backbone_config`.
Fix Typo
Author: tony9402
Pull request: yes

# What does this PR do?

Fixed wrong annotation 
- `(seq_len, batch, embed_dim)` -> `(batch, seq_len, embed_dim)`

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Finishing tidying keys to ignore on load
Author: sgugger
Pull request: yes

# What does this PR do?

A couple of tests are failing on main due to #24505 (and PRs merged between the start of the work on this branch and its merge). This should fix all of them.

Note: will merge as soon as CI is green to make main green again.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_24535). All of your documentation changes will be reflected on that endpoint.
`AutoModelForCausalLM.from_config` doesn't handle `revision="79ec93c"` correctly
Author: ibeltagy
Pull request: no

### System Info

To reproduce:

```
from transformers import AutoModelForCausalLM, AutoConfig
config = AutoConfig.from_pretrained("mosaicml/mpt-7b", trust_remote_code=True, revision="79ec93c")
config.n_layers = 0
config.d_model = 16
config.n_heads = 4
model = AutoModelForCausalLM.from_config(config, trust_remote_code=True, revision = "79ec93c")
```

and the error is:

```
In [21]: model = AutoModelForCausalLM.from_config(config, trust_remote_code=True, revision = "79ec93c")
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[21], line 1                                                                                                                                        
----> 1 model = AutoModelForCausalLM.from_config(config, trust_remote_code=True, revision = "79ec93c")

File /opt/conda/envs/torch2.0.1/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:422, in _BaseAutoModelClass.from_config(cls, config, *
*kwargs)
    420     model_class = get_class_from_dynamic_module(class_ref, repo_id, **kwargs)
    421     _ = kwargs.pop("code_revision", None)
--> 422     return model_class._from_config(config, **kwargs)
    423 elif type(config) in cls._model_mapping.keys():
    424     model_class = _get_model_class(config, cls._model_mapping)

File /opt/conda/envs/torch2.0.1/lib/python3.10/site-packages/transformers/modeling_utils.py:1143, in PreTrainedModel._from_config(cls, config, **kwargs)
   1141         model = cls(config, **kwargs)
   1142 else:
-> 1143     model = cls(config, **kwargs)
   1145 # restore default dtype if it was modified
   1146 if dtype_orig is not None:

TypeError: MPTForCausalLM.__init__() got an unexpected keyword argument 'revision'
```

### Who can help?

Probably @sgugger 

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Provided in the issue message

### Expected behavior

Loading the model not crashing

Comments:
You should pass `code_revision` and not `revision` in your second call: you are loading the model code for that revision, not the weights.

Works. Thank you.
Add bitsandbytes support for gpt2 models
Author: DarioSucic
Pull request: yes

# What does this PR do?

The current bitsandbytes integration only supports models using [nn.Linear](https://pytorch.org/docs/stable/generated/torch.nn.Linear.html#torch.nn.Linear), which excludes gpt2 and other models that instead use [Conv1D](https://github.com/huggingface/transformers/blob/68c92981ff2b804979d2e6107eeefe298d1e5183/src/transformers/pytorch_utils.py#L85). This PR enables loading/serialization of these models, as well as gpt2-xl tests for int8 and 4bit.

This is achieved by transposing the weight matrices of Conv1D layers before quantization.

Note: Following the suggestion in the bnb tests to use models with >1b params only leaves [gpt2-xl](https://huggingface.co/gpt2-xl), which is unfortunately a 6.4GB download due to being stored in float32.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

@younesbelkada, @TimDettmers

Comments:
_The documentation is not available anymore as the PR was closed or merged._

> FYI, on my side I get these failing tests, I believe there might be a small difference between our envs. We can always update the expected sentence later in case they fail on the daily CI (which probably will be the case). Happy also to add the missing test in a follow up PR.
> <img alt="Screenshot 2023-06-27 at 16 08 55" width="1314" src="https://user-images.githubusercontent.com/49240599/249179643-418abf8c-3408-49b6-8212-5e4e75e5f284.png">

Aha, it's been stable for me so far, but I can see that happening. If it's any help I'm running this on an RTX 4090 and `torch==2.1.0.dev20230603+cu121`.

> Also one test is failing for 4bit:
> 
> ```shell
> FAILED tests/bnb/test_4bit.py::Bnb4BitGPT2Test::test_memory_footprint - AttributeError: 'GPT2MLP' object has no attribute 'dense_4h_to_h'
> ```
> 
> Could quickly address a fix? 🙏 After that we should be ready to merge

Nice catch! I have a fix in mind that should also remove most of the int8 test code I added, so I'll get that in asap.
⚠️ Time to say goodbye to py37
Author: ydshieh
Pull request: yes

# What does this PR do?

[Sorry for the spams, GitHub had some issues]

Same as #24075, but that PR got freezed after I force pushed (after rebase), and my changes to address the comments were not able to appear)

(amy have already approved #24075)

----

Byebye!

EOL of python 3.7 is `2023/06/27`.

https://endoflife.date/python

Comments:
_The documentation is not available anymore as the PR was closed or merged._
feat: add support for protobuf 4
Author: jose-turintech
Pull request: yes



# What does this PR do?

add support for protobuf 4



Comments:
_The documentation is not available anymore as the PR was closed or merged._

As you can see from all the red crosses above, this sadly requires more work than just unpinning protobuf.

@sgugger yes, thanks for taking the time to pointing that out. Currently trying to identify the scope of needed changes and viability. If I can't work on providing right/complete support I'll close the PR.
Allow for warn_only selection in enable_full_determinism
Author: Frank995
Pull request: yes

Enable full determinism crashes if the model has layer that do not support it (like layoutlmv2). This fixes it.

Comments:
Or don't use the option with a model that does not support it? This is not something that is enabled by default.

_The documentation is not available anymore as the PR was closed or merged._

> Or don't use the option with a model that does not support it? This is not something that is enabled by default.

Well, not everyone can afford it, and it's not good practice to keep something that it's crash prone. e.g. In my company we are using a model to prod and would like to use full determinism to run full end-to-end evaluations to carry out fine tuning experiments. It doesn't really cost anything to add an option there (I pushed another commit), and would save people the burden of "hacking" a Docker image just to avoid crashes when deploying an image

Thank you for educating me on good practices.

To be honest, I started with a very simple request and you answered assuming to know what was our current situation: "don't use the option with a model that does not support it?" is not really a solution in many real world scenarios. I tried to explain to you our situation and you answered sarcastically.
My point was simply that you currently have an implementation of a function which is not very useable in the case I described to you above, but from your answer I assume that you probably don't care, even if is a two liner change.

Finally, even if it didn't make any sense you could have just thanked for the contribution instead of being standoffish

Hi there.

PyTorch has a built-in mechanism in this function to fail when it can't do its job properly, so that the user is not surprised to have non-reproducible result. You are, of course, free to change it in your experiments (and not get full reproducibility). You are wrong to assume that every person using Transformers would like to ignore the error. I apologize if my first answer was maybe too short and did not convey this point. There was nothing aggressive in it so you did not have to answer with a patronizing tone.

The function can be duplicated (it's five lines of code) and called with the arguments you require at the beginning of the script instead of calling via Transformers, which should solve your issue without changing the experience for other users.

You're right and you make a fair point. Indeed my first commit was a bit naive in assuming that everyone would like to have warn_only=True. That's why I changed it in the second commit (maybe you missed it). Do you think the current status is worth be applied?