noamaanMulla-03/transformers-issues huggingface dataset:
Fix minor bug to handle dynamic sequence length 
Author: duyvuleo
Pull request: yes
Labels: name: WIP

This PR is to fix a minor bug for handling dynamic sequence length, e.g., during building the serving model(s).  

Comments:
Hi! Could you run `make quality` at the root of your clone so that it passes the code quality test?

@LysandreJik : I did run `make quality` but got the following error:

`(pyvenv3-transformers-forked) ➜  transformers git:(master-minor-fix-t5) make quality
black --check examples tests src utils
All done! ✨ 🍰 ✨
621 files would be left unchanged.
isort --check-only examples tests src utils
flake8 examples tests src utils
python utils/style_doc.py src/transformers docs/source --max_len 119 --check_only
/Library/Developer/CommandLineTools/usr/bin/make extra_quality_checks
/Users/PZ9DU5/vuh/tools/pyvenv3-transformers-forked/lib/python3.7/site-packages/setuptools/dist.py:454: UserWarning: Normalizing '4.0.0-rc-1' to '4.0.0rc1'
  warnings.warn(tmpl.format(**locals()))
running deps_table_update
updating src/transformers/dependency_versions_table.py
python utils/check_copies.py
python utils/check_dummies.py
python utils/check_repo.py
Checking all models are properly tested.
Checking all models are properly documented.
Checking all models are in at least one auto class.
Traceback (most recent call last):
  File "utils/check_repo.py", line 400, in <module>
    check_repo_quality()
  File "utils/check_repo.py", line 396, in check_repo_quality
    check_all_models_are_auto_configured()
  File "utils/check_repo.py", line 342, in check_all_models_are_auto_configured
    all_auto_models = get_all_auto_configured_models()
  File "utils/check_repo.py", line 316, in get_all_auto_configured_models
    for attr_name in dir(transformers.models.auto.modeling_auto):
AttributeError: module 'transformers.models.auto' has no attribute 'modeling_auto'
make[1]: *** [extra_quality_checks] Error 1
make: *** [quality] Error 2`

anoy nody working on htis?

Closing as this PR is super old
Add DistilBertForCausalLM
Author: KMFODA
Pull request: yes
Labels: name: WIP

# What does this PR do?

Similar to the `BertLMHeadModel` this PR aims to add a `DistilBertForCausalLM` model in `modeling_distilbert.py`.

Fixes #7397 
Replaces #8387

## Who can review?

@patil-suraj 

Comments:
Hey @patil-suraj, I've worked on your very helpful comments in #8387 and implemented them in this PR. Apart from a few formatting changes which I still need to work on the main test that is still failing is `python3 -m pytest tests/test_modeling_distilbert.py::DistilBertModelTest::test_for_causal_lm_decoder`. 

I'm still debugging this but it seems that the `cross_attention_outputs` aren't being produced when the model is run as a decoder because the `encoder_hidden_states` are empty. I'll try and find out why that's the case.

Hey @patil-suraj. Managed to fix the few tests that were failing and now `make tests` passes for all tests. I'm still seeing a few failed tests in the CI but the error messages don't seem to be related to the DistilBert model.

Hi @KMFODA , yes the errors don't look related to this PR. Could you please rebase your branch and push again ? That should fix the CI issue. Let me know if it doesn't.

Done! Thanks for the suggestion @patil-suraj. PR now passes all checks. :)

Thanks for the review @patil-suraj. Made all the changes as per your comments.

No problem at all @patil-suraj. Once again I really appreciate the comments and help. Hopefully with this latest push your comments should be addressed :).

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

Hey @patil-suraj. Is there anything else I should change in the latest commit?

@KMFODA @patil-suraj Wondering if there's any reason this PR is not merged yet? It's been a few months. What is the usual turnaround time one should expect for PRs?

Hey @ghlai9665. I'm not sure what the usual turnaround time is but I'm aware the team are quite busy with multiple other projects. I'll make sure to look at anything @patil-suraj comes back with soon as.

Hey @KMFODA ! Sorry for being super slow here. Let me know if you still have time to work on this, I'll help with the merge conflicts and any other changes. Thanks :) 

Hey @patil-suraj, no problems at all. Yes happy to work on it still. Do you see any conflicts that would not be resolved by a rebase?

@KMFODA , Thanks a lot!
It would be awesome if you could rebase and resolve the conflicts. The PR is almost finished, will take another look once the conflicts are resolved. Let me know if you need any help with it!

Hi @patil-suraj , apologies for the delay. Was stuck in loops trying to ensure all tests passed and then rebasing. Should be good for review now.

Closing as this PR is super old and not needed anymore
[WIP][examples/flax] add gradient accumulation
Author: patil-suraj
Pull request: yes
Labels: name: WIP

# What does this PR do?

Adds gradient accumulation in flax language modeling scripts.

Comments:
Thanks a lot for adding this! That's super useful! It seems to require some bigger changes to core functionality to the script so I think we should be careful here. Also I'm starting to wonder whether the examples become to complicated to read with more and more functionality being added and whether we should maybe instead creating a new training script instead?

Also wouldn't it be better to use gradient accumulation functionality from `optax` such as https://optax.readthedocs.io/en/latest/api.html?highlight=ApplyEvery#optax.apply_every ? Think a lot of the code that is written here already exists in optax classes/functions no?

@sgugger - I'd love to have your feedback on the PR as well

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

Unstale

Hi, I've just checked the flax examples on master branch and it seems that gradient accumulation is still missing, so I'm coming back to this PR :)

@patrickvonplaten mentioned to use `optax`, and I've found this (working?) implementation of gradient acc. for T5 MLM pre-training from @gsarti. This may could help here :hugs: 





Closing as this PR is super old
[WIP] flax gradient checkpointing
Author: patil-suraj
Pull request: yes
Labels: name: WIP

# What does this PR do?

Adds gradient checkpointing to flax models

Comments:
Would be good to get some results on how much memory can be saved and if this is correctly implemented 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

unstale

Closing as this PR is super old
GLM Model implementation [WIP]
Author: spatil6
Pull request: yes
Labels: name: WIP

#11377 

Started implementation of GLM model.

@patil-suraj

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?





Comments:
I can review this! Ping me when you are ready.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

Gently pinging @spatil6 :) 

> Gently pinging @spatil6 :)

Yeah, i'm on it. will have update by next week,

Closing as this PR is super old
[WIP] SMITH
Author: gchhablani
Pull request: yes
Labels: name: WIP

# What does this PR do?

This PR adds SMITH encoder by Google Research and potentially closes #9526.

Potential reviewers:
@LysandreJik @patil-suraj

Comments:
Hi @gchhablani 

What is the state if this PR ?

Have you tried loading [the official SMITH checkpoint](https://github.com/google-research/google-research/tree/master/smith#pre-trained-model-checkpoint) ?

Amine

Hi @amineabdaoui

I had tried it a while back and it had worked. My focus switched to other things. I'll get back onto this PR this week.

Closing as this PR is super old
[WIP] Add ViLBERT
Author: gchhablani
Pull request: yes
Labels: name: WIP

# What does this PR do?

This PR adds ViLBERT.

Papers: [Multitask ViLBERT](https://openaccess.thecvf.com/content_CVPR_2020/html/Lu_12-in-1_Multi-Task_Vision_and_Language_Representation_Learning_CVPR_2020_paper.html) , [VilBERT](https://arxiv.org/abs/1908.02265).
GitHub: https://github.com/facebookresearch/vilbert-multi-task


Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

Unstale.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

Unstale

Closing as this PR is super old
new call for model addition 
Author: patil-suraj
Pull request: yes
Labels: name: WIP

# What does this PR do?

This PR add a "call for model addition" to add [GLM](https://github.com/THUDM/GLM) model into the lib.

Comments:
Think we still need to fill out a couple of things before publishing it :-) 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

unstale

@patil-suraj @patrickvonplaten 

I have started implementation of GLM model in HF. Need some input on implementation part.

1. Original GLM model is implemented using torch parallel distribution , In HF implementation, are we going to keep torch parallel distributed or convert to normal architecture(without parallelism )

2. There are six version of models (like GLM -base, GLM- large…etc), in which two diffrent tokenization is used  Wordpiece and BPE. Like GLM-base and large is having wordpiece and  'GLE-Roberta'and 'GLE-large' is having BPE.



Closing as this PR is super old
[WIP] Add PiT
Author: patil-suraj
Pull request: yes
Labels: name: WIP

# What does this PR do?

Adds `PoolingTransformer` for image classification. https://github.com/naver-ai/pit

Todos:
- [ ] Fix tests
- [ ] Add doc
- [ ] port and push all `PiT` checkpoints

Comments:
Closing as this PR is super old
Fix faulty rstrip in module loading
Author: Rocketknight1
Pull request: yes

Our module loading code uses `.rstrip(".py")` to remove the python file suffix if present, but this is incorrect, because it actually removes any number of characters in the list `[".", "p", "y"]` from the end of the filename, resulting in an error if the filename ends in `p` or `y`:

```python
>>> "lamp.py".rstrip(".py")
'lam'
```

Python has a proper method for doing what we want here, which is `.removesuffix()`, but it was only added in 3.9, and we still need to support 3.8. Instead, I just do it by hand.

Fixes #31061

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_31108). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Thanks @Rocketknight1 Sorry about the previous merge.

@ydshieh No problems, I don't think I would have spotted it either until I saw the error in the logs!
Python suffix stripping in `transformers.dynamic_model_utils.get_class_in_module` strips additional characters
Author: gmastrapas
Pull request: no
Labels: name: Should Fix

name: bug

### System Info

- `transformers` version: 4.41.1
- Platform: macOS-14.4.1-x86_64-i386-64bit
- Python version: 3.10.10
- Huggingface_hub version: 0.23.2
- Safetensors version: 0.4.2
- Accelerate version: 0.28.0
- Accelerate config:    not found
- PyTorch version (GPU?): 2.1.2 (False)
- Tensorflow version (GPU?): 2.15.0 (False)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No


### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

Try loading a model from the hub that has custom code files ending in `p` or `y`

For example
```
import torch
from transformers import AutoModel

model = AutoModel.from_pretrained(
    'BAAI/EVA-CLIP-18B', 
    torch_dtype=torch.float16,
    trust_remote_code=True,
)
```

### Expected behavior

The model should be loading fine. But what happens is the following error
```
ValueError: The model class you are passing has a `config_class` attribute that is not consistent with the config class you passed (model has <class 'transformers_modules.BAAI.EVA-CLIP-18B.ecb774f7cbba81ad031555b951e126e78f978186.configuration_evaclip.EvaCLIPConfig'> and you passed <class 'transformers_modules.BAAI.EVA-CLIP-18B.ecb774f7cbba81ad031555b951e126e78f978186.configuration_evacli.EvaCLIPConfig'>. Fix one of those so they match!
```

Notice how the filename `configuration_evaclip` is cropped to `configuration_evacli`

The buggy code is here https://github.com/huggingface/transformers/blob/0a064dc0fcba31092868f911772df087901d90fb/src/transformers/dynamic_module_utils.py#L201

`rstrip` removes all trailing characters that match these provided and not the `.py` suffix

Comments:
cc @Rocketknight1 

Also cc @leot13 who is impacted by this
Fix env.py in cases where torch is not present
Author: Rocketknight1
Pull request: yes

Recent changes to `env.py` in #31003 cause issues when torch isn't installed. The cause is that because pt_cuda_available defaults to "NA", which is a truthy value, the `if pt_cuda_available:` block is executed even if torch is not present, which causes our TF and Flax tests to fail. This PR should hopefully fix them!

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_31113). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Convert_mistral_weights_to_hf fails loading consolidated.safetensors
Author: HRKpython
Pull request: no

### System Info

Need the latest convert_mistral_weights_to_hf to load the Mistral v0.3 instruct.  The current code in the repo needs modifications:
No sliding_window in params.json
Torch.load cannot load the consolidated.safetensors file

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1- try running
Python convert_mistral_weights_to_hf with Mistral v0.3 model weights

### Expected behavior

Convert weights to hugging face format

Comments:
Hi @HRKpython 
https://github.com/huggingface/transformers/pull/30981 updates the conversion script to support v3 weights, can you try that new script while waiting to be merged?

Worked.  Thanks a lot

Thanks a lot @HRKpython !
Rm maintainer + migrate
Author: muellerzr
Pull request: yes

# What does this PR do?

Adjusts the relevant areas of the PR template and issue template now that Sourab has left. 

TL;DR:

* I will be taking over the FSDP and DeepSpeed issues, and eventually more maintainers will/are getting training that can assist as well.
* @SunMarc will be helping out with the core Trainer areas (Thanks Marc!) 🤗 

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@amyeroberts @LysandreJik 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_31089). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
FIX / Docs: Fix GPTQ expected number of bits
Author: younesbelkada
Pull request: yes

# What does this PR do?

Specifically addresses: https://x.com/younesbelkada/status/1794658913524260979

cc @SunMarc 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_31111). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Add per-parameter gradient logging (and before optimizer step callback)
Author: dhruvbpai
Pull request: no
Labels: name: Feature request

@RylanSchaeffer
### Feature request

I wish to log (in wandb) the norm of the gradient of each parameter in my transformer. Currently, supplying a max grad norm value will automatically log the gradient norm for the whole model, but there is no way to do this per named parameter.

Specifically, this is impossible even with a callback as on_step_begin is called before loss is computed and on_step_end is called after the optimizer zeroes the gradients. My requests are then as follows:

1. Add an option for per parameter gradient logging
2. Add a before optimizer step (akin to lightning) callback in the trainer

For context, I have shown below the current logic for the call of `on_step_end` and why it is insufficient.

[https://github.com/huggingface/transformers/blob/573565e35a5cc68f6cfb6337f5a93753ab16c65b/src/transformers/trainer.py#L2289](https://github.com/huggingface/transformers/blob/573565e35a5cc68f6cfb6337f5a93753ab16c65b/src/transformers/trainer.py#L2289)

### Motivation

I would like to monitor the properties of the gradients through callbacks, and I believe huggingface should add flexibility for this feature. As it stands, the current callback setup does not allow for this. 

### Your contribution

I can work on this, once I am given the green light.

Comments:
cc @muellerzr @younesbelkada 

Great feature @dhruvbpai - feel free to open a PoC PR and we'll take it from there!
Deprecate low use models
Author: amyeroberts
Pull request: yes

Deprecates models which are at least a year old and have had fewer than 5,000 downloads in the past 30 days
- graphormer
- xlm_prophetnet
- qdqbert
- nat
- ernie_m
- tvlt
- nezha
- mega
- jukebox
- vit_hybrid
- deta
- speech_to_text_2
- efficientformer
- realm
- gptsan_japanese

Left out of this (despite qualifying): 
* conditional detr - to give time to see effects of object detection script 
* time series transformer - to give time for checkpoints to be created
* x-clip was not added. It was picked up by the script run >month ago, but there are models currently with a large number of downloads in the past 30 days. 

Comments:
Amy, Conditional DETR is now in [object detection guide](https://huggingface.co/docs/transformers/main/en/tasks/object_detection). It would be great to see its usage after a month before we will deprecate it. Actually, from my short experience training these models, Conditional DETR could be a drop-in replacement for DETR, because it's as fast as DETR, but converges faster.

@qubvel OK, sounds good! I've removed Conditional DETR for now. We can see if the usage picks up over the next few months now the training script is available. 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_30781). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Add VLM generation default contributor
Author: gante
Pull request: yes

# What does this PR do?

Adds @zucchini-nlp as the default contributor for VLM + generate issues 🤗 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_31115). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.