noamaanMulla-03/transformers-issues huggingface dataset:
Import "transformers" could not be resolved
Author: givik
Pull request: no

### System Info

**I have tried different python versions 3.7 and 3.11**

![Screenshot 2023-03-16 at 13 18 32](https://user-images.githubusercontent.com/2458760/225571262-7deb343d-9d76-4d14-8a24-1c75da0276fa.png)


### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

from transformers import pipeline

classifier = pipeline("sentiment-analysis")

res = classifier ("I've been waiting for a Hugging Face course my whole life.")

print (res)

### Expected behavior

should work

Comments:
Hi @givik, thanks for raising this issue. 

This isn't related to transformers - it's to do with vscode and the environment. 

The error shown is coming from PyLance, and is indicating that the environment it's looking in doesn't have `transformers` installed. Please make sure transformers [is installed](https://huggingface.co/docs/transformers/installation) and PyLance is looking in the [correct place](https://code.visualstudio.com/docs/python/environments).

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

i get the same issue

tyu

cringe

In VSCode, If you have installed environment manager like conda or venv or something similar, you must point vs code to use exact same env that you are using to develop.

To achieve that behavior:
Step 1: ```Cmd (Ctrl) + Shift + P``` and search for ```Python: Select Interpreter```
Step 2: Select the right env (the env that you have installed the transformers)

Everything should work find after you set this
Add RobustSAM (CVPR 2024)
Author: leolu030066
Pull request: no
Labels: name: New model

name: Vision

### Model description

Robust Segment Anything Model (RobustSAM) is an enhanced version of the Segment Anything Model (SAM), designed to improve segmentation performance on low-quality images while preserving the original model's zero-shot segmentation capabilities and flexible prompting system.

### Open source status

- [x] The model implementation is available
- [x] The model weights are available

### Provide useful links for the implementation


- Paper: [https://openaccess.thecvf.com/content/CVPR2024/html/Chen_RobustSAM_Segment_Anything_Robustly_on_Degraded_Images_CVPR_2024_paper.html](https://openaccess.thecvf.com/content/CVPR2024/html/Chen_RobustSAM_Segment_Anything_Robustly_on_Degraded_Images_CVPR_2024_paper.html)
- arXiv: [https://arxiv.org/pdf/2406.09627](https://arxiv.org/pdf/2406.09627)
- Github repo: [https://github.com/robustsam/RobustSAM](https://github.com/robustsam/RobustSAM)

@robustsam 

Comments:
@leolu030066  - Hello!
RobustSAM is just an extension of SAM.
We have implemented the HF model cards here: 👀🤗👇:
https://github.com/robustsam/RobustSAM/issues/10#issuecomment-2294327808


Best.

> @leolu030066 - Hello! RobustSAM is just an extension of SAM. We have implemented the HF model cards here: 👀🤗👇: [robustsam/RobustSAM#10 (comment)](https://github.com/robustsam/RobustSAM/issues/10#issuecomment-2294327808)
> 
> Best.

Got it! Thanks for your contribution.
Sincerely,
Leo
Add RobustSAM model documentation
Author: leolu030066
Pull request: yes


# What does this PR do?

- Added RobustSAM model documentation under docs/source/en/model_doc/
- Modified toc file to include the new model documentation

<!-- Remove if not applicable -->

Fixes # (issue)

None.


## Motivation

The addition of the `RobustSAM` model documentation is intended to help users understand how to use the model within the Transformers library. This documentation provides detailed information on the model's capabilities, usage examples, and integration within the existing framework.

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)?[#32874](https://github.com/huggingface/transformers/issues/32874)
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?
Feedback on the current implementation and documentation structure is welcome. Tagging the following for review once ready:
- Author: @robustsam
- Documentation: @stevhliu
- Vision models: @amyeroberts



Comments:
More robust tests required for gradient checkpointing
Author: anferico
Pull request: no

As flagged in [this comment](https://github.com/huggingface/transformers/issues/31945#issuecomment-2233766500) from #31945, the [current test suite](https://github.com/huggingface/transformers/blob/72fb02c47dbbe1999ae105319f24631cad6e2e00/tests/test_modeling_common.py#L777) for gradient checkpointing may fail to spot subtle bugs. For example, the bug flagged in #31028 was not detected by any test, because the success criterion is simply that gradients are `not None` for each parameter that has `requires_grad=True`.

Proposal: every function whose behavior is affected by whether gradient checkpointing is enabled or not, such as a model's `forward()` method, should be tested twice: once with GC enabled and once with GC disabled, making sure that the outputs are exactly the same given certain inputs and configuration parameters.

Comments:
cc @ydshieh 

Would you like to contribute @anferico ?

@ydshieh sure, though this will likely take some time and require support

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Clean up Unpack imports
Author: molbap
Pull request: yes

# What does this PR do?

:broom: :broom: small cleanup PR to uniformize imports. 

cc @yonigozlan if I've forgotten a few models :) 

Comments:
I don't see any processors missing for now, but it reminds me I have to do it on my other PRs too :)  

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33631). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Misleading `ImportError` error
Author: davidgilbertson
Pull request: no
Labels: name: Usage

name: bug

### System Info

- `transformers` version: 4.42.3
- Platform: Linux-5.15.153.1-microsoft-standard-WSL2-x86_64-with-glibc2.31
- Python version: 3.10.12
- Huggingface_hub version: 0.23.4
- Safetensors version: 0.4.3
- Accelerate version: 0.32.1
- Accelerate config:    not found
- PyTorch version (GPU?): 2.4.1+cu121 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: No
- Using GPU in script?: No
- GPU type: NVIDIA GeForce RTX 3090


### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

This GitHub issue is not about resolving the errors below, it's about a logical error in the code that produces the errors.

I get the following error running some code:
```
ImportError: This modeling file requires the following packages that were not found in your environment: flash_attn.
```
But I have `flash_attn` installed. The underlying problem is that when importing `flash_attn`, some other code throws an `ImportError`, but that error is swallowed by `transformers` code.

To be specific, the logical mistake is in [the check_imports function](https://github.com/huggingface/transformers/blob/main/src/transformers/dynamic_module_utils.py#L187). It assumes that if you get an `ImportError` when trying to import a package, it means that the package isn't installed.

That's probably a fine assumption most of the time, but when something's going wrong like in my case, it makes it hard to find out what because the real error is being swallowed and replaced. This is the real error, by the way:
```
ImportError: libcudart.so.11.0: cannot open shared object file: No such file or directory
```

To reiterate, this issue is not about resolving these errors, it's about fixing the code that masks useful errors (although it someone happens to know the fix, great).

### Expected behavior

You should show the original error and then if you want to, also show the hand-written "missing packages" message.

Comments:
Hi @davidgilbertson, It's true we are not handle `ImportError` - will propose a PR to improve that a bit.
Fix DPT /Dinov2 sdpa regression on main
Author: molbap
Pull request: yes

# What does this PR do?

Should fix https://github.com/huggingface/transformers/issues/33649 identified by @avishaiElmakies by falling back to eager when sdpa is called with output_attentions=True. 

@amyeroberts @ArthurZucker 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33660). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@molbap the attention mask creation needs to be adjusted too in that case.

Also needs the additional case to check if the head mask is not None as in llama.

Ah nvm, apparently not mb 

@vasqu Yes, it seems the forward for DPT/Dinov2 is less demanding :) good looking out though
handle dependency errors in check_imports
Author: molbap
Pull request: yes

# What does this PR do?

Quick fix/improvement of error messages in import check. Adds a `logger.error` but one that's actionable for the user. 
Fixes #33604 

@ArthurZucker when you have time :) 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33622). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Sure, good point - changing to `warning` and merging this!

Nice one, thanks!
WIP: Clone tensors to be able to mutate
Author: dvrogozh
Pull request: yes

Submitting this as draft since I am not sure whether that's correct fix. However, it does help to resolve issue described at #33178 where we get "RuntimeError: cannot mutate tensors with frozen storage". So posting this change for illustration for above issue.

CC: @gante


Comments:
@SunMarc @dvrogozh could you help review the fix in #33287?

Superseded by #33303.
confusing deprecation msg for `DynamicCache.seen_tokens` - no `cache_position` in this class
Author: poedator
Pull request: no

The message says `"The `seen_tokens` attribute is deprecated and will be removed in v4.41. Use the `cache_position` model input instead."` however there is no `cache_position` in this class (it is present in StaticCache). Perhaps there may be a clearer message.

code ref: https://github.com/huggingface/transformers/blame/7c1149120829eb58e42a1ba4dbd2a9fa98864707/src/transformers/cache_utils.py#L91

tagging @gante 

Comments:
@poedator thank you for opening this issue! 🤗 

We're aware of it, and we're working on a [pre-requisite task](https://github.com/huggingface/transformers/issues/32685) before tackling the current confusion around `seen_tokens`/`get_seq_length`. (In the long run, we don't want to use these functions, but rather use the explicit `cache_positions` all the time.)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Difference in embedding weight initialization for randomly initialized T5 model
Author: dhruvbird
Pull request: no
Labels: name: bug

### System Info

transformers 


### Who can help?

@ArthurZucker 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

The problem is technical, so I will describe it here. I believe the idea is to keep the weight initialization the same for pytorch or tf models initialized from scratch. However, this is different. 

In https://github.com/huggingface/transformers/blob/main/src/transformers/models/t5/modeling_t5.py#L821 the embedding weights are initialized with a variance of 1. However, in tf, this is done by initializing with a standard deviation of 0.05. https://github.com/tensorflow/mesh/blob/fa19d69eafc9a482aff0b59ddd96b025c0cb207d/mesh_tensorflow/layers.py#L1635 

https://www.tensorflow.org/api_docs/python/tf/random_normal_initializer 

According to the docs, it's default initialized with these arguments: 

```
tf.random_normal_initializer(
    mean=0.0, stddev=0.05, seed=None
)
``` 

PyTorch initialization: 

```
            # Mesh TensorFlow embeddings initialization
            # See https://github.com/tensorflow/mesh/blob/fa19d69eafc9a482aff0b59ddd96b025c0cb207d/mesh_tensorflow/layers.py#L1624
            module.shared.weight.data.normal_(mean=0.0, std=factor * 1.0)
``` 

TF initialization: 

```
def embedding_weights(mesh,
                      vocab_dim,
                      output_dim,
                      variable_dtype,
                      name="embedding",
                      ensemble_dim=None,
                      initializer=None):
  """Embedding weights."""
  shape = mtf.Shape(
      [ensemble_dim] if ensemble_dim else []) + [vocab_dim, output_dim]
  if initializer is None:
    initializer = tf.random_normal_initializer()
  ret = mtf.get_variable(
      mesh, name, shape, dtype=variable_dtype, initializer=initializer)
  return ret
``` 

This is already mentioned in #16749 but since #16749 mentions 2 issues this first one seems to have gone unnoticed, so I am opening a separate issue for this one. 


### Expected behavior

I expect the initialization to be the same across TF and PyTorch T5 models. 


Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Fix error string after refactoring into get_chat_template
Author: tibor-reiss
Pull request: yes

This PR fixes a typo or improves the docs.

@Rocketknight1 

Comments:
LGTM now! Before we can merge, can you pull the latest changes from main, and then rebase your branch on main, then force push everything? That should get the tests green.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33652). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@tibor-reiss perfect - thank you for the PR!
Modular `transformers`: modularity and inheritance for new model additions
Author: ArthurZucker
Pull request: yes

# Modular `transformers`: supporting inheritance, while keeping our philosophy

This PR introduces **Modularity** for `transformers`, which has always been prohibited when working with transformers (see [blog post](https://huggingface.co/blog/transformers-design-philosophy) for the accompanying design philosophy). 

The core idea behind this PR is to facilitate model addition by enabling Pythonic inheritance while keeping true to our single-file policy in which models/processors must be contained within a single file, enabling working around the object without going through 10 layers of abstractions.

## A modular file

This PR introduces a new concept: a "modular" file under the format `modular_xxxx.py`.

This file accepts importing objects and modules from different modules; these objects and modules can serve as superclasses for the newly defined objects. Down the road, this `modular_xxxx.py` file and the inherited components will be unraveled in the commonly known `modeling_xxxx.py`, `config_xxxx.py`, and other processing files. These files will hold the "single-file" component. Contributors will see their work dramatically eased-up as their contributions can now be limited to the modular file, rather than many files spread across the repository.

This supports configuration, tokenizers, processors, image processors and modeling (though only modeling, tokenizer and config were properly tested). 

## A linter

The "linter", which unravels the inheritance and creates all single-files from the modular file, will flatten the inheritance while trying to be invisible to Python users. For example:
- If a configuration class inherits from another and adds/deletes an argument, the generated file will either directly reference it (in case of addition) or completely remove it (in case of deletion).
- If a class inherits from another, for example: `class GemmaModel(LlamaModel):`, dependencies are **automatically** inferred. All submodules will be automatically inferred from the superclass.

## A CI

This PR further adds a new CI replacing the current `check_copies` script. The aim is for this to tell the exact differences in a git-friendly manner as such:

<img width="699" alt="image" src="https://github.com/user-attachments/assets/0594dd82-b13d-4e41-b8d8-c17d58e78860">

The aim is to gradually replace the current system based on `# Copied from` statement to move towards a very pythonic API leveraging inheritance, while keeping the single-file policy that makes playing with models/switching from one model to the other/fixing a single model simple.

--- 

cc @zucchini-nlp as you played around with it 😉 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33248). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix CIs post merging modular transformers
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Simple fix to the LlavaNextVideo

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33681). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
'CohereModel' object has no attribute '_prune_heads'
Author: mnauf
Pull request: no
Labels: name: Good First Issue

name: bug

### System Info

class `CoherePreTrainedModel` docstring says that "This model inherits from [`PreTrainedModel`]. Check the superclass documentation for the generic methods the library implements for all its model (such as downloading or saving, resizing the input embeddings, **pruning heads** etc.)". However, it doesn't have `prune_heads()` methods, even though it explicitly mentions that it has in the docstring.

![Screenshot 2024-08-31 at 20 54 26](https://github.com/user-attachments/assets/e6c4b029-231e-4a03-84ab-07b9bd41ebec)
@ArthurZucker 


### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```
from transformers import AutoTokenizer, AutoModel

model_id = "CohereForAI/aya-23-8B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModel.from_pretrained(model_id, device_map="auto", torch_dtype=torch.bfloat16)
model.prune_heads({1: [1]})
```

### Expected behavior

I expect the method to prune heads of the cohere model. 

Comments:
#29622 

@saurabhdash2512 Please have a look

Thanks for your issue @mnauf ! The documentation should be updated, as indeed prune heads is only supported for older models as newer models haven't received this exploration from a research perspective.

Hey, I’d like to fix this issue. @LysandreJik , you mentioned that the documentation needs updating—could you clarify which specific documentation so I can work on fixing it?

Can I support prune_heads feature in this model? @LysandreJik @ArthurZucker

This has been addressed in https://github.com/huggingface/transformers/pull/33253, so I'm closing the issue - thanks all for offering your help!
Generation tests: update imagegpt input name, remove unused functions
Author: gante
Pull request: yes

Related to #33212 -- automatically run generation tests on all model classes where `model.can_generate()` is `True`

On #33212, we can see that we are not running generation tests on some VLM classes with generative capabilities. To accommodate them, some changes are needed in our `generate` test suite, which is very `input_ids`-centric. I'm breaking down related changes in independent PRs whenever possible, to avoid creating a massive PR.

This PR:
- removes unused code from the test suite
- updates `pixel_values` to `input_ids` in `imagegpt`. The `pixel_values` -> `input_ids` transition was done in the model, but not in its tests. Because `pixel_ids` were actually `input_ids`, this model requires several test overwrites, which were making #33212-related changes painful

______________________________

up next: rework `_get_input_ids_and_config` in `GenerationTesterMixin` to be VLM-friendly (FYI @zucchini-nlp )

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33663). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Improve Error Messaging for Flash Attention 2 on CPU
Author: sizhky
Pull request: yes

## Before submitting
- [x] This PR adds a better better error message on CPU for flash attention
Sometimes torch.version.cuda is available but due to CUDA_VISIBLE_DEVICES being empty, the current error message raised is unhelpful

Previously the message used to be 
```
FlashAttention2 has been toggled on, but it cannot be used due to the following error: Flash Attention 2 is not available.
```

Now the error message will be 
```
FlashAttention2 has been toggled on, but it cannot be used due to the following error: Flash Attention 2 is not available on CPU. Please make sure torch can access a CUDA device."
```


## Who can review?

@stevhliu


Comments:
@vignesh1507 which 3 locations?

3 lines of code (loc)

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33655). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.