noamaanMulla-03/transformers-issues huggingface dataset:
H
Author: lil-fahad
Pull request: no

import AutoTokenizer, AutoModelForSequenceClassification

tokenizer = AutoTokenizer.from_pretrained("icyGS/StockPredictor")

model = AutoModelForSequenceClassification.from_pretrained("icyGS/StockPredictor")

Comments:
Hi @lil-fahad, thanks for raising an issue. 

So that we can best help you, could you fill in the issue template including information such as the environment (run `transformers-cli env` in the terminal), the issue being encountered, the expected behaviour and a full traceback please? 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
clip loss
Author: hljjjmssyh
Pull request: no

https://github.com/mlfoundations/open_clip/blob/37b729bc69068daa7e860fb7dbcf1ef1d03a4185/src/open_clip/loss.py#L49


In the implementation of open_clip, logits distributed across multiple gpus are gathered for calculating loss. However, I cannot find the code related to this feature in this repository. I think more negative samples are very important for contrastive learning. @younesbelkada @ydshieh 

Comments:
Hi @hljjjmssyh The loss computation management is in the `Trainer` class, see

https://github.com/huggingface/transformers/blob/da005253b82395b6097623bcee44b819bfe72b87/src/transformers/trainer.py#L2649-L2650

That's only for models wrapped in DataParallel @ydshieh 

@hljjjmssyh We don't include code requiring torch.distributed as it then fails when the script is used on one GPU. However we could use the Accelerate library to have something that works in both situation. If you want to explore this and open a PR, I'll be happy to review!

I think I’m missing something, it looks like this could be done for CLIP today with accelerate’s implementation in `examples/pytorch/image-classification/run_image_classification_no_trainer.py` running it with the appropriate args? Or maybe accelerate would nonetheless be a welcome addition somewhere else for the above mentioned purpose? 

It also looks [here](https://github.com/huggingface/transformers/blob/5990743fddb4780b15b8af2ed7ab55145ab40455/src/transformers/trainer.py#L1386-L1388) like the model would in fact be wrapped in DataParallel when training on multiple gpus.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Is there or will there be support for xformers?
Author: ethansmith2000
Pull request: no

### Feature request

xformers, (i couldn't find anything online or in the docs, but i suspect its very likely I'm just missing something)

### Motivation

speed and memory improvement

### Your contribution

I am unsure, but willing to help.

Comments:
cc @younesbelkada 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
added GPTNeoXForTokenClassification
Author: peter-sk
Pull request: yes

# What does this PR do?

It adds the class GPTNeoXForTokenClassification, which allows using GPT NeoX models for token classification tasks. The implementation follows the one for other models (such as GPT2 and GPT Neo) closely and simply adds a linear layer after the hidden states.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [X] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [X] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@ArthurZucker
@younesbelkada

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Ready for review, @ArthurZucker  and @younesbelkada  👍
🌐 [i18n-KO] Translated `multilingual.mdx` to Korean
Author: HanNayeoniee
Pull request: yes

<!-- PR의 제목은 "🌐 [i18n-KO] Translated `<your_file>.mdx` to Korean" 으로 부탁드립니다 -->
# What does this PR do?

Translated the `multilingual.mdx` file of the documentation to Korean.
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179
<!-- 메인 이슈에 기록이 남아요! 가짜연구소 리포를 사용해 연습하실때는 제거해주시면 감사하겠습니다! :smile: -->

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [x] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에만 가짜연구소 팀원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
Team PseudoLab, may you please review this PR? @0525hhgus, @KIHOON71, @sim-so, @gabrielwithappy, @HanNayeoniee, @wonhyeongseo, @jungnerd

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. 가짜연구소 팀원들과 리뷰가 끝난 후에만 허깅페이스 직원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
@sgugger, @ArthurZucker, @eunseojo May you please review this PR?

Comments:
_The documentation is not available anymore as the PR was closed or merged._
added GPTNeoForTokenClassification
Author: peter-sk
Pull request: yes

# What does this PR do?

It adds the class GPTNeoForTokenClassification, which allows using GPT Neo models for token classification tasks. The implementation follows the one for other models (such as GPT2) closely and simply adds a linear layer after the hidden states.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@ArthurZucker
@younesbelkada



Comments:
_The documentation is not available anymore as the PR was closed or merged._

Hey! Could you make sure the CI tests are green? Can review then! 

@ArthurZucker
Sure. I'm getting the hang of it. Now, the only failing tests are connected to flax and seem unrelated to this pull request.

If the flax errors are not due to the PR, this is ready to be reviewed, @ArthurZucker and @younesbelkada :-)

I just checked the logs for the remaining errors one more time. The errors are related to the import of the optax library, where jax.Array is used in a type. Apparently there is no name "Array" in the top-level namespace of the jax module.

I cannot see how this could be related to my PR.

The jax version used in the examples_flax test is 0.3.6:
Collecting jax!=0.3.2,<=0.3.6,>=0.2.8 (from transformers==4.28.0.dev0)
    Using cached jax-0.3.6-py3-none-any.whl
This version clearly has no Array class.
I am unsure why such an old version should be used?

Figured out that optax <= 0.1.4 is needed. And found out that upstream/main has that change already 👍  Now everything should be cleared for review.

Definitely ready for review,  @ArthurZucker and @younesbelkada :-)


Cool! Reviewing now

All done and ready to be merged, @ArthurZucker and @younesbelkada 👍 

I implemented the same change as for GPTNeoXForTokenClassification, i.e., I removed the hasattr etc. and just use config.classifier_dropout directly.

@sgugger Ready to merge when the checks complete. Thanks for the fast action 👍

... and more to come in the next weeks!
Using a Model without any pretrained data
Author: mnitin73
Pull request: no

## ❓ Questions & Help

<!-- Sorry for a very basic question. Can I use your library without any pertained data? For example, I want to use a BERT transformer model, but using only my corpus of data. In the docs, I only see examples using pretrained models. Thanks. -->


Comments:
Just don't use the [from_pretrained](https://huggingface.co/transformers/main_classes/model.html#transformers.PreTrainedModel.from_pretrained) method and initialize the class with a config.
```
from transformers import BertModel, BertConfig

#model with pretrained weights
model_with_Pretrained = BertModel.from_pretrained('bert-base-uncased')

#model without pretrained weights
config = BertConfig()
model_without_Pretrained = BertModel(config)
```

@cronoik Thanks

Hi, I also encounter the same  question, is the solution still valid in the latest v4.28?
thanks!
Add model parallelism to CodeGen
Author: LostBenjamin
Pull request: yes

This PR adds model parallelisim to the CodeGen model. I have been using this since August.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20016). All of your documentation changes will be reflected on that endpoint.

ok, I did not know that. Thanks for the info!

Does `device_map="auto"` also work for CodeGen?

Yes, it's supported (at least on the main branch)!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

> Thanks for your PR but this code for model parallelism is deprecated and will be removed from other code files. To use parallelism, please load the model with `device_map="auto"`.

I'm a bit confused about model parallelism in Huggingface. I'm trying to fine-tune a CodeGen model using Huggingface Trainer. Is loading the model with `device_map="auto"` the right way to enable model parallelism?

From what I read [here](https://huggingface.co/docs/transformers/v4.28.1/perf_train_gpu_many#naive-model-parallelism-vertical-and-pipeline-parallelism), model parallelism is only supported by GPT2 and T5.
Fix CLAP link across all READMEs
Author: ehsanmok
Pull request: yes

# What does this PR do?

Fixes CLAP link across all READMEs

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).

Comments:
_The documentation is not available anymore as the PR was closed or merged._

I'm getting

```
Traceback (most recent call last):
  File "/Users/ehsan/workspace/transformers/utils/check_task_guides.py", line 58, in <module>
    "asr.mdx": transformers_module.models.auto.modeling_auto.MODEL_FOR_CTC_MAPPING_NAMES,
  File "/Users/ehsan/workspace/transformers/src/transformers/utils/import_utils.py", line 1150, in __getattr__
    raise AttributeError(f"module {self.__name__} has no attribute {name}")
AttributeError: module transformers.models.auto has no attribute modeling_auto. Did you mean: 'processing_auto'?
```

not sure what's going on so I added it manually to all the `index.mdx`. It seems not all models research paper links have been synced across docs.

Thanks again!
Feature Request: Support Longformer 3D attention mask ?
Author: Maybewuss
Pull request: no
Labels: name: wontfix

I try to use 3d attntion mask, but failed.

Comments:
Hey @Maybewuss,
could you post a code snippet example, where you show the use case of using a 3D attention mask? :-) 
What would you use it for?

> Hey @Maybewuss,
> could you post a code snippet example, where you show the use case of using a 3D attention mask? :-)
> What would you use it for?

@patrickvonplaten 
I want each word to foucs different words in an sentence, for example, in the generate task, the attntion mask is a lower triangular 3d matrix. Besides, other models like Bert and Roberta support 3d mask.

```
model = Lonformer.from_pretrained('model_path')
input_ids = torch.randint(20, (3, 5), dtype=int)
attn_masks = torch.randint(2, (3, 5, 5), dtype=int)
model(input_ids, attn_masks)
```

@Maybewuss , I think this feature should actually already be implemented for Longformer.

Can you provide me with a code snippet with the error message?
The code snippet should include a public model and functional code so that I can copy-paste into my console to see the error. Otherwise, it will be difficult for me to help you. Thanks! 

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


I used 3d attention mask in the LongFormer, but also failed. I find that 
the code 
attention_mask = nn.functional.pad(
                attention_mask, (0, padding_len), value=0
            )  # no attention on the padding tokens 
in line 1626 in modeling_longformer may be not support the 3D attention mask. Please correct me if I am wrong
When download model. Error: DefaultCPUAllocator: can't allocate memory: you tried to allocate 
Author: af913337456
Pull request: no

### System Info

I want to download the GPT4all-J model. 

according to this weblink : https://huggingface.co/nomic-ai/gpt4all-j

download code:

```python
from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("nomic-ai/gpt4all-j", revision="v1.2-jazzy")

```

it got thie error in the end:

```
RuntimeError: [enforce fail at alloc_cpu.cpp:75] err == 0. DefaultCPUAllocator: can't allocate memory: you
tried to allocate 268435456 bytes. Error code 12 (Cannot allocate memory)
```

----

and the memory of my machine is enough for 268435456 bytes.

```
[root@VM-0-5-centos ~]# free
              total        used        free      shared  buff/cache   available
Mem:       19750992      386456    18700296        2024      664240    19070948
Swap:             0           0           0
```

> 19070948 KB > 268435456 bytes

---

@vanpelt @pvl @arfon @xeb 

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1. create the download python file;

```python

from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("nomic-ai/gpt4all-j", revision="v1.2-jazzy")
```

2. run step 1;
3. wait for it;
4. got the error.

### Expected behavior

download success.

Comments:
what's wrong with the code above.

This means you do not have enough RAM to load the model, not disk memory. You can try adding `device_map="auto"` to load directly the model on the GPUs if you have any, or `torch_dtype=torch.float16` to save 2x the memory (inference only).
Make `_test_xla_generate` less flaky
Author: ydshieh
Pull request: yes

# What does this PR do?

Make `_test_xla_generate` less flaky by relaxing the condition:
- if number of examples < 10: be strict, no difference is allowed
- otherwise, only fail the test if there are more than 10% of examples give different outputs between XLA and non-XLA versions.

Since this test is slow (generation), better not to decorate with `is_flaky`.

For `TFPegasusModelTest::test_xla_generate_slow`: there were more than 10 failures in 70 runs. With this PR, 0 failure shows.


Comments:
_The documentation is not available anymore as the PR was closed or merged._
[docs] Doc TOC updates
Author: MKhalusova
Pull request: yes

This PR restructures TOC for the documentation. All the previous links remain working (except the two pages that have been removed: migration guide and converting from TF). 
Here's the scope of the restructure: 
a) TOC is sorted from “beginner” topics to more advanced making it easier to know where an answer to a question might be
b) Some topics have been renamed to be concise with the rest in the same section and (in some cases) more descriptive
c) Task Guides are collapsed by default and now are on on the same level (currently NLP task guides are hidden, and not aligned with other modalities)
d) “General usage” has been  renamed to “Developer Guides”
e) Benchmarks, notebooks, and community resources have been moved under Developer Guides 
f) “Converting from TensorFlow checkpoints” and "Migrating from previous packages" pages removed

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Cuda rng_state_all is used when saving in distributed mode so same should also be used when loading
Author: ShivamShrirao
Pull request: yes

When saving in distributed mode this snippet uses `torch.cuda.random.get_rng_state_all()`.

https://github.com/huggingface/transformers/blob/a0e733283930bdb9ae2b1afdc53ec5f2daefb033/src/transformers/trainer.py#L2417-L2421

But while loading `torch.cuda.random.set_rng_state_all()` is not being used for distributed causing issues when resuming training.

https://github.com/huggingface/transformers/blob/a0e733283930bdb9ae2b1afdc53ec5f2daefb033/src/transformers/trainer.py#L2323-L2328



Comments:
_The documentation is not available anymore as the PR was closed or merged._
Add memory-efficient attention and optional features to Llama
Author: s-JoL
Pull request: yes

This PR adds memory-efficient attention to Llama, resulting in a 30% improvement in training efficiency. We also removed some transposes to adapt to the shapes allowed by the *memory_efficient_attention* operation. Additionally, we have added hidden dropout and attention dropout to the model, which helps with better generalization during training.

Furthermore, two optional features have been added: stable embedding, used in Bloom, and shared input-output vectors, used in PALM. These features have been tested and found to improve training stability and performance.
The main changes are as follows:
```python
if xops is not None and self.training:
   attn_weights = None
   attn_output = xops.memory_efficient_attention(query_states, key_states, value_states, 
      attn_bias=self.causal_mask, p=self.dropout_prob)
```
As we use operators from the xformers library, we need to add a dependency on xformers.

We implemented pre-training of the Llama model based on transformers + accelerate, incorporating the modifications described above.
https://github.com/Bayes-Song/Open-Llama/blob/main/README_en.md

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_22386). All of your documentation changes will be reflected on that endpoint.

> Thanks for your PR. Transformers is not meant to be a modular toolbox, so we don't add every feature to every model. Llama was trained without stable embedding or shared input-output vectors, so we won't add them to the modeling code of Llama. Likewise for the dropouts.
> 
> Since you are training new models using this code, as soon as you have checkpoints available, I would advise to make a PR with a new model (mostly copied from Llama) like we have all the variants of GPT-2 for instance.

Thank you for your response. The memory_efficient_attention in xformers is actually mentioned in the original Llama paper. So, it is possible to integrate this component into the Llama training code.

@Bayes-Song 
Thanks for the PR
Can we use this when Torch2.0 is supported?
Like in
https://github.com/huggingface/diffusers/pull/2303/files

cc: @sgugger 

If it's non-breaking and actually faster on **all** setups, we can add it yes. The PR makes other modifications for the time being, which we cannot accept as mentioned in my comment above.

Currently I have trained a new model based on the above changes, and I am adding a new model to the transformers library based on @sgugger 's suggestion. I will re-open a PR after I finish all the code.
[XLN] Fix XLN
Author: ArthurZucker
Pull request: yes

# What does this PR do?
[This commit](https://github.com/huggingface/transformers/commit/87e6e4fe5c7e65cb69e70306f22de6daf16b6e14) broke the XLNet docstyle explaining what the output of the `create_mask` function should be. 


Comments:
Still a draft, will be fixing  #21626 via a deprecation cycle

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_21631). All of your documentation changes will be reflected on that endpoint.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.