noamaanMulla-03/transformers-issues huggingface dataset:
[wip: testing doc-builder]
Author: mishig25
Pull request: yes

Testing https://github.com/huggingface/doc-builder/pull/301


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_19195). All of your documentation changes will be reflected on that endpoint.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Fix flash attention bugs with Mistral and Falcon
Author: fxmarty
Pull request: yes

This PR fixes some important bugs in the Mistral and Falcon integration.

https://github.com/huggingface/transformers/pull/26933 broke flash attention for Falcon due to the modification of the layout

The following tests were not passing:
```
FAILED tests/models/mistral/test_modeling_mistral.py::MistralModelTest::test_flash_attn_2_generate_padding_right - AssertionError: ValueError not raised
FAILED tests/models/mistral/test_modeling_mistral.py::MistralModelTest::test_flash_attn_2_inference_padding_right - AssertionError: ValueError not raised

FAILED tests/models/falcon/test_modeling_falcon.py::FalconModelTest::test_flash_attn_2_generate_left_padding - RuntimeError: CUDA error: device-side assert triggered
FAILED tests/models/falcon/test_modeling_falcon.py::FalconModelTest::test_flash_attn_2_generate_padding_right - RuntimeError: CUDA error: device-side assert triggered
FAILED tests/models/falcon/test_modeling_falcon.py::FalconModelTest::test_flash_attn_2_generate_use_cache - RuntimeError: CUDA error: device-side assert triggered
FAILED tests/models/falcon/test_modeling_falcon.py::FalconModelTest::test_flash_attn_2_inference - RuntimeError: CUDA error: device-side assert triggered
FAILED tests/models/falcon/test_modeling_falcon.py::FalconModelTest::test_flash_attn_2_inference_padding_right - RuntimeError: CUDA error: device-side assert triggered
```

and Falcon with FA2 is not really usable on `main` due to an error in the shape (currently `[batch_size, num_head, seqlen, head_dim]` instead of the required `[batch_size, seqlen, num_head, head_dim]`.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Hi @amyeroberts thank you for the review! I added by mistake `test_flash_attn_2_generate_use_cache` in `test_modeling_llama.py` while it was meant to be added in `test_modeling_mistral.py`, hence the confusion, apology!

@fxmarty Thanks for clarifying. Tbh, I'm still a bit confused with the tests - it's not clear to me how this explicitly tests for the cache as `use_cache` isn't set anywhere 😅 

@amyeroberts good catch indeed... I just checked, we are going here that sets `use_cache=True`: https://github.com/huggingface/transformers/blob/f93c1e9eceafde40b1d33fbb03834de97556706c/src/transformers/generation/utils.py#L1602

and uses https://github.com/huggingface/transformers/blob/f93c1e9eceafde40b1d33fbb03834de97556706c/src/transformers/generation/configuration_utils.py#L266

@fxmarty OK - thanks for explaining! As a follow up, could you add `use_cache=True` explicitly into the tests? This way it's clearer for anyone who sees the code and isn't subject to silently not being tested anymore if the configs or config handling changes 

For sure @amyeroberts I will ping you there. Sorry I should have waited before merging..

@fxmarty No worries! It doesn't affect the functionality of this PR so it's fine to be done separately :)
remove the deprecated method `init_git_repo`
Author: ji-huazhong
Pull request: yes

# What does this PR do?

As docstring says `init_git_repo is deprecated and will be removed in v4.34.0 of Transformers.`


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @muellerzr and @pacman100

Integrations:

- deepspeed: HF Trainer/Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc and @younesbelkada

Documentation: @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->

cc @muellerzr


Comments:
bug in run_finetune
Author: dorost1234
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.3.3
- Platform: linux
- Python version: 3.7
- PyTorch version (GPU?): 1.8 
- Tensorflow version (GPU?): - 
- Using GPU in script?: -
- Using distributed or parallel set-up in script?: - 

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @jplu

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

@patrickvonplaten, @patil-suraj

## Information
I am running  run_seq2seq.py getting 

ImportError: cannot import name 'is_offline_mode' from 'transformers.file_utils'

## To reproduce

Steps to reproduce the behavior:

python run_seq2seq.py


thnaks

Comments:
sorry my mistake to use last version of examples 

hi how you solved this. i also face this error. i can't find "is_offline_mode"  and "get_full_repo_name" under transformers.utils. i use https://github.com/huggingface/transformers/tree/main/examples/pytorch/summarization

This should work if you are using `transformers` master/main 

> https://github.com/huggingface/transformers/tree/main/examples/pytorch/summarization

I miss the same issue. How do you solve it?
`resize_token_embeddings` always sets `requires_grad` (of parameters in the resized embedding) to `True`
Author: czy-orange
Pull request: no

### System Info

- `transformers` version: 4.26.1
- Platform: Linux-3.10.0-1160.90.1.el7.x86_64-x86_64-with-glibc2.31
- Python version: 3.11.3
- Huggingface_hub version: 0.17.2
- PyTorch version (GPU?): 2.0.1 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

``` Python console
In [1]: from transformers import AutoModel

In [2]: model = AutoModel.from_pretrained('bert-base-uncased')

In [3]: model.embeddings.word_embeddings.weight.requires_grad
Out[3]: True

In [4]: model.embeddings.word_embeddings.weight.requires_grad = False

In [5]: model.embeddings.word_embeddings.weight.requires_grad
Out[5]: False

In [6]: model.resize_token_embeddings(30526)
Out[6]: Embedding(30526, 768)

In [7]: model.embeddings.word_embeddings.weight.requires_grad
Out[7]: True
```



### Expected behavior

The `resize_token_embeddings` method is not supposed to change the `requires_grad` attribute of parameters in the embedding layer. However, the current version of `resize_token_embeddings` always sets `requires_grad` to `True`, which is not a desirable behavior if the `requires_grad` in the old embedding is `False`. As far as I'm concerned, the `resize_token_embeddings` should only perform resizing and keep all the other attributes unchanged.

After I read the source code carefully, I found that `resize_token_embedding` internally creates a new embedding, which does not copy attributes from the old embedding except for `dtype` and `device`.

Comments:
This issue was raised in forum but was not solved. This is the [original post](https://discuss.huggingface.co/t/how-to-freez-a-model/12819/2).

Hey! Seems like this has been the case for quite a while, but I think it makes sense to keep this info as well, preventing surprises. Would you like to open a PR for a fix? 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Allow multilabel classification mode for widgets in the models repo
Author: avidale
Pull request: no

# 🚀 Feature request

1. Enable multilabel classification mode and regression mode for the widgets in the model repo.
2. Create the corresponding tags that can be read from the model card.

## Motivation

Models for sequence classification by default support three modes: binary/multiclass classification, multilabel classification, and regression. However, the widgets in the model repository support only multiclass mode (where probabilities of classes sum to 1). This can be misleading for the users who taste the models using the widgets. For example, my model https://huggingface.co/cointegrated/rubert-tiny-toxicity, which is intended for multilabel classification, but the widget normalizes the predicted probabilities to sum to 1, which leads to confisuion of the potential users of the model.

## Your contribution

If you show me where to start, I could start working on implementing this feature. However, currently I don't know what part of the Huggingface repository is responsible for widgets and underlying computations.


Comments:
Hi @avidale, I'm closing this issue as I think it is an accidental duplicate of #12871.
Also, I've transferred #12871 to [huggingface_hub/#222](huggingface/huggingface.js#300) since thats where the widgets src is
How to get the number of trainable parameters for a hf model
Author: mathmax12
Pull request: no

### Feature request
'
peft_parameters = LoraConfig(
    lora_alpha=16,
    lora_dropout=0.1,
    r=8,
    bias="none",
    task_type="CAUSAL_LM"
)
train_params = TrainingArguments(
    output_dir="./results_modified",
    num_train_epochs=1,
    per_device_train_batch_size=4,
    gradient_accumulation_steps=1,
    optim="paged_adamw_32bit",
    save_steps=25,
    logging_steps=25,
    learning_rate=2e-4,
    weight_decay=0.001,
    fp16=False,
    bf16=False,
    max_grad_norm=0.3,
    max_steps=-1,
    warmup_ratio=0.03,
    group_by_length=True,
    lr_scheduler_type="constant",
    report_to="tensorboard"
)
fine_tuning = SFTTrainer(
    model=base_model,
    train_dataset=training_data,
    peft_config=peft_parameters,
    dataset_text_field="text",
    tokenizer=llama_tokenizer,
    args=train_params
)

fine_tuning.train()

I am using the above code for model training with Lora. I wonder after applying to Lora. How could I check the number of trainable parameters of the model before and after?

### Motivation

Understand the training process well

### Your contribution

I'd love to 

Comments:
Hey, I would use something like this:
```python 
def count_trainable_parameters(model):
    model_parameters = filter(lambda p: p.requires_grad, model.parameters())
    params = sum([np.prod(p.size()) for p in model_parameters])
    return params
```

note that  we try to keep the github issues for bugs/feature requests.
This kind of question should be asked on the [forum](https://discuss.huggingface.co/) instead
Thanks!
How can check hugging face pretrained model has metadata attribute?
Author: JinSeoung-Oh
Pull request: no

Hi, at first, I want to say thanks for you guys work and contribution.
My question is how can I check hugging face model has metadata attribute.

Theses day, I'm building GraphRAG with Llama-index and NubularDB for korean
Baseline for this work has already been completed.
But korean LLM performance is not good, so I tried the other model listed in OpenKo-LLM LeaderBoard
But all models I tried failed. 
Error message is 'Attribute Error: (Llamamodel / LlmaForcusalLM / Mistralmodel / MistralForcusalLM) has no attribute 'metadata''
It just my thought all xxxxForCusalML has no attribute 'metadata'.. Maybe

So, I just want to know how to check this.
I already asked it to LlamaIndex and NebularGraph, but I can not solve this problem.
According to LlamaIndex bot(dosu-bot), there is no reason why llama-index cannot be compatible with Hugging Face models
It said I have to check metadata attribution is properly defined for the model you plan to use.

How I can check it?

Comments:
Hey! 🤗 thanks for the kind words!
Could you share the full traceback and how you called `transformers`. 
I am not sure about the integration with Llama-index and I am this unfamiliar with the Metadata attribute

It's likely this is something outside `transformers`, and need to open an issue in those libraries. But yes, we can only tell with a full log.

@ArthurZucker @ydshieh 
Thanks!
And @ydshieh you are right.
I just misunderstood Llama-index bot' reply
This is korean(9:45 AM), and I just solved this problem 5 min ago.
I have to load hugging face model using HuggingFaceLM class in Llama-index not transformers.
Sorry for bothering..
Thanks!
Control first downsample stride in ResNet
Author: jiqing-feng
Pull request: yes

Hi @ArthurZucker 

Relate to [25856](https://github.com/huggingface/transformers/pull/25856). I added a new parameter in config to control the stride for the first bottleneck layer in stages. Would you please help me review it? Thx!

Comments:
Hi @ArthurZucker @rafaelpadilla .

Thanks for your advice. The resnet backbone which downsamples in bottleneck layer can be found in [resnet](https://huggingface.co/Jiqing/resnet-backbone-downsample_in_bottleneck).

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26374). All of your documentation changes will be reflected on that endpoint.

> LGTM appart for the arg that is not used. Let's have explicit and useful args, doing a if else is okay to chose the layer and more understandable

Sure!
docs: add addition library install
Author: pphuc25
Pull request: yes

Hi,
Following the tutorial instructions, I discovered that the `requirements.txt` file is missing some libraries, such as `transformers`, and contains an incorrect version for `tokenizer`. Consequently, I added the missing library to the list.
I would like to cc @sanchit-gandhi to review my PR.

Comments:
Hey @pphuc25 - our convention with the `examples` files is that we assume that the user has the latest version of Transformers through an [editable install](https://huggingface.co/docs/transformers/installation#editable-install). The `requirements.txt` files then specify the **additional** requirements needed to run the examples (e.g. `evaluate` or `jiwer`). So in this case, there's no need to include `transformers` or `tokenizers`, as they are installed when the user runs the editable installation! Hope that makes sense!

Thanks, this more make sense to me
Getting error "TypeError: 'NoneType' object is not callable" while using pretrained model checkpoints in TFAutoModelForQuestionAnswering
Author: pksX01
Pull request: no

### System Info

- `transformers` version: 4.34.0
- Platform: macOS-14.0-arm64-arm-64bit
- Python version: 3.10.12
- Huggingface_hub version: 0.17.3
- Safetensors version: 0.4.0
- Accelerate version: not installed
- Accelerate config: not found
- PyTorch version (GPU?): 2.1.0 (False)
- Tensorflow version (GPU?): 2.15.0-dev20231007 (False)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help?

@ArthurZucker @younesbelkada 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Run following code in Python 3 terminal.

```
from transformers import AutoTokenizer
from transformers import TFAutoModelForQuestionAnswering

model_checkpoint = "distilbert-base-cased"

tokenizer = AutoTokenizer.from_pretrained(model_checkpoint)
model = TFAutoModelForQuestionAnswering.from_pretrained(model_checkpoint)
```

### Expected behavior

Model safetensors shall be downloaded. But getting below error:
```
>>> model = TFAutoModelForQuestionAnswering.from_pretrained(model_checkpoint)
Using TensorFlow backend
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/rudra/mambaforge/envs/keras_tf/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 565, in from_pretrained
    return model_class.from_pretrained(
  File "/Users/rudra/mambaforge/envs/keras_tf/lib/python3.10/site-packages/transformers/modeling_tf_utils.py", line 2912, in from_pretrained
    model.build()  # build the network with dummy inputs
  File "/Users/rudra/mambaforge/envs/keras_tf/lib/python3.10/site-packages/keras/src/layers/layer.py", line 222, in build_wrapper
    original_build_method(*args, **kwargs)
  File "/Users/rudra/mambaforge/envs/keras_tf/lib/python3.10/site-packages/transformers/modeling_tf_utils.py", line 1131, in build
    if self.built or call_context().in_call:
TypeError: 'NoneType' object is not callable
```

Note: Same code is working in Google colab but failing in local on Mac M1.

Comments:
Hey! As you mention that this works on colab, I would recommend you to make sure your mac install of tensorflow is compatible. Running this on my M1 works as expected. I used `pip install tensforflow-macos tensorflow-metal` 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
[DPT] Add MiDaS 3.1 series
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR improves the DPT model by leveraging the `AutoBackbone` API.

DPT is a [depth estimation model](https://huggingface.co/Intel/dpt-large). Recently, the MiDaS team released a new [3.1 version](https://github.com/isl-org/MiDaS/tree/master#towards-robust-monocular-depth-estimation-mixing-datasets-for-zero-shot-cross-dataset-transfer) with various backbones: BEiT, Swinv2, etc. hence it's an ideal use case for the `AutoBackbone` class.

This PR:

- adds the `BeitBackbone` class
- adds the `Swinv2Backbone` class
- extends modeling_dpt.py to leverage the AutoBackbone API
- fixes the `keep_aspect_ratio` and `ensure_multiple_of` flags of `DPTImageProcessor`, which does not work on main due to them not being passed to the `resize` method.

To do:

- [ ] make sure `out_indices` are backwards compatible for BEiT

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_25799). All of your documentation changes will be reflected on that endpoint.

I've split up the PR in smaller pieces, see above for the first one

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.