noamaanMulla-03/transformers-issues huggingface dataset:
Zero-shot finetuning examples
Author: SangbumChoi
Pull request: no
Labels: name: Feature request

name: Vision

### Feature request

@amyeroberts @qubvel @NielsRogge 

Does HF team intersted in adding zero-shot finetuning example like https://github.com/SangbumChoi/transformers/tree/grounding_examples/examples/pytorch/zero-shot?

If the team merge these two following open PR then
https://github.com/huggingface/transformers/pull/31828
https://github.com/huggingface/transformers/pull/31964

we can finetune the groundingdino and write something like this.
https://blog.roboflow.com/how-to-fine-tune-paligemma/

I will left some model that you can run 
Model : https://huggingface.co/danelcsb/grounding-dino-tiny-finetuned-cppe-5-10k-steps-no-trainer/tree/main

### Motivation

People can use zero-shot model to finetune there own projects.

### Your contribution

I have made the branch already and ready to generate PR. Need some more modification.
Before
<img width="1002" alt="Screenshot 2024-08-06 at 6 08 57 PM" src="https://github.com/user-attachments/assets/92bbe009-91df-4be5-b786-eb998fe4c508">
After
<img width="793" alt="Screenshot 2024-08-06 at 6 09 05 PM" src="https://github.com/user-attachments/assets/dec37ec9-f0fb-4ef4-93a9-a5575196e472">



Comments:
Hi @SangbumChoi, the finetuned model example looks great! We are definitely interested in such an example for fine-tuning and blogpost! Let me know if you need any assistance!

Hi @SangbumChoi. First of all, It's great that now we can fine-tune zero-shot models like GDINO directly from transformers. But in the example picture above, It seems that the class names in the finetuned example are very incoherent in comparison to the first image. Why do you that that is? Do you think overfitting the model on the fine-tuning dataset fix this issue?

@zappy586  Thanks for the interest. That is actually nice catch. I also have realized that problem and fixed in the above PR (It was not overfitting, it was about text_encoder part). Please see the updated examples in the PR!
Docs: alert for the possibility of manipulating logits
Author: gante
Pull request: yes

# What does this PR do?

As @osanseviero told me on slack, popular logit manipulation strategies like `min_p` are a bit obscure in our docs.

A proper fix will come with the [`generate` refactor](https://github.com/huggingface/transformers/issues/30810), where a diagram of what `generate` does will be added. In that diagram, there will be a dedicated section for logits processors.

Meanwhile, a quick patch: this PR adds a tip in the decoding strategies section alerting for the existence and importance of logit manipulation.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32467). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

doc builder screenshot: 
![Screenshot 2024-08-06 at 14 50 06](https://github.com/user-attachments/assets/0187e8b3-0a96-4f77-95a9-ad6c0539d2b4)


Nice, thanks!
Using `numpy==2.0.0`
Author: ydshieh
Pull request: no
Labels: name: WIP

`transformers`  now remove the pin of `numpy<2.0.0`. However, some issues (with 3rd libraries) are known, and the following gives some guide:

- In an environment with `TensorFlow` and/or `Flax` --> downgrade to `numpy<2.0.0`
- issue with soxr --> `pip install soxr==0.4.0b1`
- issue with `faiss`  --> downgrade to `numpy<2.0.0`
- If error out with a message `using np.array(obj, copy=False) replace it with `np.asarray(obj) to allow a copy`: 
   - downgrade to `numpy<2.0.0`
   - or, install `datasets` dev. version

Comments:
Adding a wip tag so this can stay open without the bot for an extended period of time 

Relates to #31740 

See https://github.com/huggingface/transformers/issues/31740#issuecomment-2278144785
model.generate() using a user specified keyword argument
Author: jhwanflow
Pull request: no

I made a simple Sea2Seq model based on BartForConditionalGeneration.

My model just takes a additional keyward argument in its forward() method.

![image](https://user-images.githubusercontent.com/63226383/154217584-101cc596-bb96-4a52-b02c-8ad216af694a.png)

Everything is just same as the original forward() method(which is from BartForConditionalGeneration) but "responder_emb" is added.

I'm trying to generate a sequence using model.generate()

like this.

![image](https://user-images.githubusercontent.com/63226383/154215933-d6de7a3b-1d39-4084-b701-e99235c8714b.png)

![image](https://user-images.githubusercontent.com/63226383/154216463-7f8010f3-6766-4201-b26c-e982ef2cd218.png)

but I run into this error (TypeError: forward() got an unexpected keyword argument).

I need to pass "responder_emb" to the model's forward() method but generate() method can't deal with the argument.

How can I solve this error?




Comments:
cc @patrickvonplaten

Hey @JH-lee95,

Please note that this question is not related to the original Transformers code and not a bug so in the future it would be amazing if you could use the forum for such questions instead: https://discuss.huggingface.co/ . 

To answer your question, to make your use case work, you'll have to adapt the following function: https://github.com/huggingface/transformers/blob/86119c115496ca773b82f8a1c8cbf2d4e44149fa/src/transformers/models/bart/modeling_bart.py#L1369 in BartForConditionalGeneration as well as making sure that your param is passed correctly through generate to the model.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

Hi, I also have a similar problem. Have you addressed it?
🌐 [i18n-KO] Translated `agent.md` to Korean
Author: Jwaminju
Pull request: yes

# What does this PR do?

Translated the `agent.md` file of the documentation to Korean.
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [ ] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에만 N조 팀원들에게 리뷰 요청하는 아래 주석을 노출해주세요! 차례대로 1 ~ 5조 -->
<!-- May you please review this PR? -->
<!-- @jun048098, @yijun-lee, @mreraser, @shinhyunji36, @heuristicwave -->
<!-- @win2dvp21, @enchantee00, @fabxoe, @1kmmk1, @junejae -->
@SeungAhSon, @Jwaminju, @thsamajiki, @4N3MONE, @jungnerd
<!-- @mjk0618, @cjfghk5697, @ahnjj, @JinukHong, @nuatmochoi -->
<!-- @010kim, @chhaewxn, @boyunJang, @jeongiin, @harheem -->

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. N조 팀원들과 리뷰가 끝난 후에 아래 주석을 노출해주세요! -->
@stevhliu May you please review this PR?

Comments:
안녕하세요, 번역 해주신 내용 검토 결과 크게 수정할 부분 없는 것 같습니다! 고생하셨습니다!

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32351). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Translate docs into Arabic issue-number #32435
Author: AhmedAlmaghz
Pull request: yes

# What does this PR do?
Translate docs into Arabic issue-number #32435
@stevhliu and @MKhalusova

Comments:
Hi! 

It is extremely difficult to review 400+ file changes. For translations, you should work on translating one file at a time, get it reviewed by a native Arabic speaker, and then move on to the next document. Translations should also be done manually to ensure nothing gets lost in translation, which can sometimes happen when using machine translation programs/apps.

I noticed you opened similar PRs in the Diffusers/Datasets repositories. The same comment here applies to those PRs as well.

- https://github.com/huggingface/diffusers/pull/9121
- https://github.com/huggingface/datasets/pull/7094

I'd suggest closing these PRs and then opening just one PR focused on translating one document. Thanks! 🤗 

I have been working on the translation for more than two months and I manually translate each file and now that I have finished I have uploaded the translation to you.  
[HFDocs](https://github.com/DSYemen/HFDocs/blob/master/pages/index.mdx)

@stevhliu 

Hi @AhmedAlmaghz, thank you for dedicating so much time in translating the docs! 

It's simply just not feasible for someone to review a PR of this size. For context, the general rule for PRs is that there shouldn't be more than 400 lines of diff for the reviewer. Here there's 61k. You should be able to separate out the file commits into their own PRs, which can then be reviewed. 

@AhmedAlmaghz 
Thanks for the great work.

I will be happy to review small PRs, you did what I procrastinating for a long time. 
You can cherry-pick commit/s into separate branches, open a PR, and mention me for reviewing.

Thank you again for the hard work and great addition to the Arabic community (جزاء الله خيرأ)
[i18n-KO] Translated `aqlm.md` to Korean
Author: thsamajiki
Pull request: yes

# What does this PR do?

Translated the `<your_file>.md` file of the documentation to Korean.
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [ ] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에만 3조 팀원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
<!-- May you please review this PR? -->
@SeungAhSon, @Jwaminju, @thsamajiki, @4N3MONE, @jungnerd

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. N조 팀원들과 리뷰가 끝난 후에 아래 주석을 노출해주세요! -->
<!-- @stevhliu May you please review this PR? -->

Comments:
안녕하세요 @thsamajiki 님!
수정된 내용을 확인해보니 toctree.yml 파일이 영문 toctree을 가져오신 것 같네요.
한글 toctree로 수정바랍니다!

@jungnerd 안녕하세요. 한글 toctree로 수정했습니다!

안녕하세요 :) 본문 내용 읽어봤는데 자연스럽고 좋아 보이네요.
toctree title에서 (번역중)만 제거해 주시면 될 것 같아요.

@SeungAhSon 안녕하세요! 수정했습니다. 감사합니다 👍

@junejae 네 요청사항 반영했습니다 :)

LGTM! 수정이 필요한 부분들 다른 분들이 잘 리뷰해주셨네요 👍
Failed to import 
Author: ravi160822
Pull request: no

### System Info

python 3.10.7
Target: arm64-apple-darwin21.6.0
Thread model: posix
transformers==4.24.0

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

`from transformers import AlbertTokenizer, AlbertModel`
->
```
RuntimeError: Failed to import transformers.models.albert.modeling_albert because of the following error (look up to see its traceback):
libcublas.so.11: cannot open shared object file: No such file or directory
```

### Expected behavior

no error

Comments:
Looks like a problem in your CUDA installation.

it works on my local, but it fails when I use it with docker, do I need to give any additional command in docker apart from installing requirements

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

@ravi160822 
I'm currently getting 

`libcublas.so.*[0-9] not found in the system path ['/app/src', '/usr/local/lib/python311.zip', '/usr/local/lib/python3.11', '/usr/local/lib/python3.11/lib-dynload', '/usr/local/lib/python3.11/site-packages', '/app/src']`

Did you find a workaround?


I'm facing the same issue as @jagumpert, but only when running Github actions:

```libcublas.so.*[0-9] not found in the system path [(...)]```

This seems to be a PyTorch issue. I was using `torch` version 2.0.1, and downgrading to 2.0.0 fixed the issue.

I also have the same issue as @jagumpert when trying to build a docker image with --platform=linux/amd64 python:3.11 as the base image. 

@saattrupdan solution did not work for me ( downgrading torch to 2.0.0 did not fix the issue )

Same here  @jose-arguelles when trying to build a docker image. Did you find any workaround?

@saattrupdan 's solution helped me, thank you.
Downgrading torch to 2.0.0 version also helped me (`poetry add torch=2.0.0`) on Ubuntu without gpu on it!

> @saattrupdan 's solution helped me, thank you. Downgrading torch to 2.0.0 version also helped me (`poetry add torch=2.0.0`) on Ubuntu without gpu on it!

Could you please explain what changed between the versions? And why this was a solution. Thanks :)

> > @saattrupdan 's solution helped me, thank you. Downgrading torch to 2.0.0 version also helped me (`poetry add torch=2.0.0`) on Ubuntu without gpu on it!
> 
> Could you please explain what changed between the versions? And why this was a solution. Thanks :)


@codingbutstillalive this is due to poetry.
When installing torch2.0.0 it also download nvidia packages unlike torch2.1

still happens, installing torch 2.0.0 did not solve it for me

Did you guys find any workaround? @AlexCapchase @jose-arguelles
RobertaForClassification throws an error because of dimension mismatch
Author: ErikVogelLUH
Pull request: no

### System Info

transformers version 4.41.2, Windows 10



### Who can help?

@ArthurZucker

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

I wanted to train a Roberta model for classification. However, during the computation of the loss for multi-label classification, the dimensions are mismatched. I found the problem in transformers/models/roberta/modeling_roberta.py:1229:

`loss = loss_fct(logits.view(-1, self.num_labels), labels.view(-1))`

The labels are flattened but the logits are formed to have dimensions (batch_size, num_labels). Slightly changing the line fixes the problem

`loss = loss_fct(logits.view(-1, self.num_labels), labels.view(-1, self.num_labels))`

### Expected behavior

Compute the loss without the dimension mismatch

Comments:
Hey @ErikVogelLUH! It's a bit hard to help you without knowing the code that failed. We know this model works well with our scripts (such as this one: https://github.com/huggingface/transformers/tree/main/examples/pytorch/text-classification)

Did you start from one of our scripts or from something else?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Fix vision encoder decoder attention implementation pick
Author: fxmarty
Pull request: yes

Noticed this bug while making the CI pass for https://github.com/huggingface/transformers/pull/30138.

Essentially, `VisionEncoderDecoderModel.from_pretrained(xxx)` does not correctly load with SDPA (e.g. with bert, deit) while it should be. The reason is that `VisionEncoderDecoderModel` does not have a class attribute `_supports_sdpa`, and we can not generally set `_supports_sdpa = True` (as multiple archs are supported). Thus, in `from_pretrained` logic to pick the attention implementation, SDPA is never picked.

We need to rely on the encoder/decoder classes to rightfully pick the attention implementation.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_31203). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

This highlights a wider issue with how attention is selected for models through configs. Having things autoset breaks a lot of the "don't do things by magic" convention that we have in transformers and this has already caused quite a few issues / surpising behaviour within the library. 

Instead, I think we should completely re-think how attention is passed through and selected. 

agreed

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Invalid attention causal mask within the Llama model ?
Author: BoyeGuillaume
Pull request: no

Hello,

I was reading the implementation of `modelling_llama.py` and I've stumble upon the code handling the [causal_mask](https://github.com/huggingface/transformers/blob/48101cf8d127bbf22d751c7df118a6ce357e2e27/src/transformers/models/llama/modeling_llama.py#L428)) within the base `Attention` implementation.

I was wondering why there is an `+` instead of a multiplication to apply the causal mask here. From my understanding the value of the `attn_weights` will be $0+x$ where it should be masked and $1+x$ where it shouldn't. The softmax afterward will then take care of "*damping*" the value of the masked attention weights. However it is still non-zero. Am I missing something here ?

I am very new to this world and as such it is very probable that I am missing something in which case I am very sorry for bothering you

Comments:
[i18n-ar] Translated file `i18n/README_zh-hant.md` into Arabic 
Author: AhmedAlmaghz
Pull request: yes

 # What does this PR do? 
 Translate docs into Arabic issue-number #32435 
 @stevhliu , @MKhalusova , @abodacs 
 This pull request contains changes [**for file : i18n/README_zh-hant.md**](https://github.com/AhmedAlmaghz/transformers/blob/Branch_i18n_README_zh-hant_md/i18n/README_zh-hant.md) 

Comments:
[i18n-ar] Translated file `i18n/README_zh-hans.md` into Arabic 
Author: AhmedAlmaghz
Pull request: yes

 # What does this PR do? 
 Translate docs into Arabic issue-number #32435 
 @stevhliu , @MKhalusova , @abodacs 
 This pull request contains changes [**for file : i18n/README_zh-hans.md**](https://github.com/AhmedAlmaghz/transformers/blob/Branch_i18n_README_zh-hans_md/i18n/README_zh-hans.md) 

Comments:
[i18n-ar] Translated file `i18n/README_vi.md` into Arabic 
Author: AhmedAlmaghz
Pull request: yes

 # What does this PR do? 
 Translate docs into Arabic issue-number #32435 
 @stevhliu , @MKhalusova , @abodacs 
 This pull request contains changes [**for file : i18n/README_vi.md**](https://github.com/AhmedAlmaghz/transformers/blob/Branch_i18n_README_vi_md/i18n/README_vi.md) 

Comments:
[i18n-ar] Translated file `i18n/README_te.md` into Arabic 
Author: AhmedAlmaghz
Pull request: yes

 # What does this PR do? 
 Translate docs into Arabic issue-number #32435 
 @stevhliu , @MKhalusova , @abodacs 
 This pull request contains changes [**for file : i18n/README_te.md**](https://github.com/AhmedAlmaghz/transformers/blob/Branch_i18n_README_te_md/i18n/README_te.md) 

Comments:
[i18n-ar] Translated file `i18n/README_ru.md` into Arabic 
Author: AhmedAlmaghz
Pull request: yes

 # What does this PR do? 
 Translate docs into Arabic issue-number #32435 
 @stevhliu , @MKhalusova , @abodacs 
 This pull request contains changes [**for file : i18n/README_ru.md**](https://github.com/AhmedAlmaghz/transformers/blob/Branch_i18n_README_ru_md/i18n/README_ru.md) 

Comments:
[i18n-ar] Translated file `i18n/README_fr.md` into Arabic 
Author: AhmedAlmaghz
Pull request: yes

 # What does this PR do? 
 Translate docs into Arabic issue-number #32435 
 @stevhliu , @MKhalusova , @abodacs 
 This pull request contains changes [**for file : i18n/README_fr.md**](https://github.com/AhmedAlmaghz/transformers/blob/Branch_i18n_README_fr_md/i18n/README_fr.md) 

Comments:
[i18n-ar] Translated file `i18n/README_de.md` into Arabic 
Author: AhmedAlmaghz
Pull request: yes

 # What does this PR do? 
 Translate docs into Arabic issue-number #32435 
 @stevhliu , @MKhalusova , @abodacs 
 This pull request contains changes [**for file : i18n/README_de.md**](https://github.com/AhmedAlmaghz/transformers/blob/Branch_i18n_README_de_md/i18n/README_de.md) 

Comments:
[i18n-ar] Translated file `i18n/README_pt-br.md` into Arabic 
Author: AhmedAlmaghz
Pull request: yes

 # What does this PR do? 
 Translate docs into Arabic issue-number #32435 
 @stevhliu , @MKhalusova , @abodacs 
 This pull request contains changes [**for file : i18n/README_pt-br.md**](https://github.com/AhmedAlmaghz/transformers/blob/Branch_i18n_README_pt-br_md/i18n/README_pt-br.md) 

Comments:
Add 3D/4D mask support for sdpa.
Author: Leoyzen
Pull request: yes

Fixes # (issue)
* add 3D/4D support for sdpa.
* see also https://github.com/huggingface/transformers/issues/31036#issuecomment-2149673751

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@ArthurZucker @younesbelkada 

Comments:
Is this Issue still open? if yes can it be assigned to me?

hey @amanjam we don't assign issue, feel free to open a PR that references the issue and this PR! 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.