noamaanMulla-03/transformers-issues huggingface dataset:
Custom tokenizer no longer compatible with transformers 4.34
Author: dakinggg
Pull request: no

### System Info

- `transformers` version: 4.34.0
- Platform: macOS-13.5-arm64-arm-64bit
- Python version: 3.10.12
- Huggingface_hub version: 0.17.3
- Safetensors version: 0.4.0
- Accelerate version: 0.20.3
- Accelerate config: 	not found
- PyTorch version (GPU?): 2.1.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed

### Who can help?

@ArthurZucker

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

In [1]: import transformers

In [2]: replitt = transformers.AutoTokenizer.from_pretrained('replit/replit-code-v1-3b', trust_remote_code=True)

...

File ~/.cache/huggingface/modules/transformers_modules/replit/replit-code-v1-3b/cc0a4f17a8d72b71d62ea53cb0e23e4dac352067/replit_lm_tokenizer.py:76, in ReplitLMTokenizer.get_vocab(self)
     75 def get_vocab(self):
---> 76     vocab = {self.convert_ids_to_tokens(i): i for i in range(self.vocab_size)}
     77     vocab.update(self.added_tokens_encoder)
     78     return vocab

File ~/.cache/huggingface/modules/transformers_modules/replit/replit-code-v1-3b/cc0a4f17a8d72b71d62ea53cb0e23e4dac352067/replit_lm_tokenizer.py:73, in ReplitLMTokenizer.vocab_size(self)
     71 @property
     72 def vocab_size(self):
---> 73     return self.sp_model.get_piece_size()

AttributeError: 'ReplitLMTokenizer' object has no attribute 'sp_model'

### Expected behavior

I don't actually know if it is expected to work or not, but maybe you can advise on the fix.

Comments:
There's pretty much nothing we can do here, this custom model needs to be updated. The break is expected, mostly put the call to super.__init__() at the end of the __init__

Yeah understood this is a custom tokenizer. Thanks for the fix suggestion! Passing along to replit.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
`QuantAct` in `IntSoftmax` receives integer value
Author: mbekmyrz
Pull request: no

**Problem/Question:**

`IntSoftmax` class has `QuantAct`:

https://github.com/huggingface/transformers/blob/fc63914399b6f60512c720959f9182b02ae4a45c/src/transformers/models/ibert/quant_modules.py#L379-L380

and it is called in `IntSoftmax.forward()` by passing `exp_int` which is integer:
https://github.com/huggingface/transformers/blob/fc63914399b6f60512c720959f9182b02ae4a45c/src/transformers/models/ibert/quant_modules.py#L418

and then within `QuantAct.forward` it is passed to `FixedPointMul` where the integer value is rescaled again to find 'integer' value:
https://github.com/huggingface/transformers/blob/fc63914399b6f60512c720959f9182b02ae4a45c/src/transformers/models/ibert/quant_modules.py#L206-L213

https://github.com/huggingface/transformers/blob/fc63914399b6f60512c720959f9182b02ae4a45c/src/transformers/models/ibert/quant_modules.py#L787

Whereas it was an integer already. All the other modules in IBert that has `QuantAct` passes real valued `x`, and it's okay for `FixedPointMul` to divide by scaling factor to find  `z_integer`. But within `IntSoftmax` input `x` was already an integer.

Same error in the original source code: https://github.com/kssteven418/I-BERT/blob/1b09c759d6aeb71312df9c6ef74fa268a87c934e/fairseq/quantization/utils/quant_modules.py#L652C9-L652C72

**Solution:**
```
exp, exp_scaling_factor = self.act(exp_int * exp_scaling_factor, exp_scaling_factor)
```

tagging @kssteven418 as the designer of the model:)

Comments:
Hey can I contribute to this issue can you tell me what all techstack I need and how should I start

Sure! Feel free to open a PR for a fix! 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
ClearML task closes when done training - unnecessary and causes issues
Author: johnml1135
Pull request: no

### System Info

When a python script ends, it automatically closes the task.  After training, some people (me included) may want to do more for the task, including inferencing on the model.  Therefore, always automatically closing the task on training completion, while sounding nice, is not needed and causes issues.



### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

See code here:
https://github.com/huggingface/transformers/blob/0ebee8b93358b6ef0182398b8fcbd7afd64c0f97/src/transformers/integrations/integration_utils.py#L1488-L1493

### Expected behavior

The ClearML task does not close when training is complete. 

Comments:
Hi @johnml1135 ! I think that the task should indeed close if it was created by the callback class. If you don't want it to close, just initialize the task externally, i.e. call `Task.init` before creating the callback instance. In this case, the task will not close on when the training ends.

I do initialize the task well before I create the model to train:
* https://github.com/sillsdev/machine.py/blob/main/machine/jobs/build_nmt_engine.py
* https://github.com/sillsdev/machine.py/blob/main/machine/jobs/nmt_engine_build_job.py

I don't know what's could more be done...

@johnml1135 Note that our fix has not been officially released yet. Have you installed `transformers` by cloning this repository and doing something like:
```
cd transformers
python3 -m pip install -e .
```
?

I think I see - you are making a fix to resolve the issue as you specified above.  Can you link to the commit/PR that implements the fix here?

@johnml1135 You can find the PR here: https://github.com/huggingface/transformers/pull/26614

Great - that should resolve our issue!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
How to manually set the seed of randomsampler generator when training using transformers trainer
Author: young-chao
Pull request: no

### System Info

I used a [script](https://github.com/huggingface/transformers/blob/v4.33.0/examples/pytorch/language-modeling/run_clm.py) to continue pre-training the llama2 model. In the second epoch, the loss began to explode, so I chose to reload the checkpoint to continue training, but the loss changes were completely consistent with before, which made me doubt the iteration of the dataset is always consistent. So I tried modifying the [seed.](https://github.com/huggingface/transformers/blob/v4.33.0/examples/pytorch/language-modeling/run_clm.py#L309C33-L309C33) But in the end, my training loss is always consistent, and the state I print randomsampler is always the same. 
I hope someone can tell me how to solve this problem, including where the seed of this generator is specified.

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

transformers==4.33.0
pytorch==1.13.1
accelerate==0.21.0
deepspeed==0.10.0

### Expected behavior

I hope that the sampling of training data set should be different every time.

Comments:
![image](https://github.com/huggingface/transformers/assets/34190033/af693746-4e27-4c6b-bf5b-0b58ef173a38)
I found that the default initialized state of the generator is always this tensor, regardless of the cuda version, pytorch version, etc.

Two things you can try: 
- pass a generator 
- use `set_seed(X)` which is a utility trainer. 


> Two things you can try:
> 
> * pass a generator
> * use `set_seed(X)` which is a utility trainer.

Thank you for your reply, I will try your solution. 

@ArthurZucker  I am going to try your solution, but I found that the trainer does not receive dataloader or sampler as a parameter, so where should I pass the generator to the sampler. Moreover, the trainer does not have this set_seed method. Do you mean the set_seed method of transformers? The script I use uses this method, however it only affects the default generator.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

你好，你的邮件已收到，谢谢！

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Fix idx2sym not loaded from pretrained vocab file in Transformer XL
Author: jtang98
Pull request: yes

# What does this PR do?
Fixes [27584](https://github.com/huggingface/transformers/issues/27584).

When loading vocab file from a pretrained tokenizer for Transformer XL, although the pickled vocabulary file contains a idx2sym key, it isn't loaded, because it is discarded as the empty list already exists as an attribute.

Solution is to explicitly take it into account, just like for sym2idx.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?

## Who can review?

@ArthurZucker @younesbelkada

Comments:
How to always use initial prompt in Whisper?
Author: GanymedeNil
Pull request: no

I checked this PR (#22496 ) but still can't figure out how to always use the initial prompt. is it possible to provide a use case?

Comments:
Hey 🤗  thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discuss.huggingface.co/) instead? I'm sure the community will be of help!

Thanks!

> Hey 🤗 thanks for opening an issue! We try to keep the github issues for bugs/feature requests. Could you ask your question on the [forum](https://discuss.huggingface.co/) instead? I'm sure the community will be of help!
> 
> Thanks!

I will go to the forum to ask questions.
translate Trainer.md to chinese
Author: jiaqiw09
Pull request: yes

# What does this PR do?

Part of  #26803 

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests? _not necessary_


## Who can review?
@stevhliu

Comments:
@stevhliu,

Here is another PR of trainer.md, I will fix merge conflict later.

And @statelesshz, I find the @stevhliu has assigned a pr review work for you of deepspeed translation work, thansk for your help. And You can check this one too.

Best
translate deepspeed.md to chinese
Author: jiaqiw09
Pull request: yes

# What does this PR do?

Part of  #26803 

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests? _not necessary_


## Who can review?
@stevhliu

Comments:
@stevhliu

Hi, 

I just translate deepspeed doc. As this part is such vital and hard to be fully correctly translated, would you mind having a double check?  I think many people will use this function, so it' s best to provide a great translation work without ambiguity.

Best

> LGTM! My Chinese isn't that great (its pretty basic 😅 ) so maybe @statelesshz, would you be interested in helping double check the translation?

My Pleasure!

@statelesshz thanks for your reviw.

and @stevhliu would you mind having a check and finish the merge. I think I will update this file oneday I learn more about deepspeed.

Best

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_27495). All of your documentation changes will be reflected on that endpoint.

Great job and thanks again for tackling such a big and dense doc! 👏
[Torchscript] Fix docs
Author: patrickvonplaten
Pull request: yes

<!-- This line specifies which issue to close after the pull request is merged. -->
Fixes #6714


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/6740?src=pr&el=h1) Report
> Merging [#6740](https://codecov.io/gh/huggingface/transformers/pull/6740?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/64c7c2bc158364ff5c53dce2f19698078b2f9d78?el=desc) will **decrease** coverage by `0.29%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/6740/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/6740?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #6740      +/-   ##
==========================================
- Coverage   80.00%   79.70%   -0.30%     
==========================================
  Files         156      156              
  Lines       28426    28426              
==========================================
- Hits        22741    22656      -85     
- Misses       5685     5770      +85     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/6740?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/tokenization\_marian.py](https://codecov.io/gh/huggingface/transformers/pull/6740/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fbWFyaWFuLnB5) | `66.66% <0.00%> (-32.50%)` | :arrow_down: |
| [src/transformers/tokenization\_xlnet.py](https://codecov.io/gh/huggingface/transformers/pull/6740/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25feGxuZXQucHk=) | `66.66% <0.00%> (-23.43%)` | :arrow_down: |
| [src/transformers/tokenization\_reformer.py](https://codecov.io/gh/huggingface/transformers/pull/6740/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fcmVmb3JtZXIucHk=) | `81.66% <0.00%> (-13.34%)` | :arrow_down: |
| [src/transformers/tokenization\_openai.py](https://codecov.io/gh/huggingface/transformers/pull/6740/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fb3BlbmFpLnB5) | `71.21% <0.00%> (-12.88%)` | :arrow_down: |
| [src/transformers/tokenization\_dpr.py](https://codecov.io/gh/huggingface/transformers/pull/6740/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fZHByLnB5) | `57.65% <0.00%> (+4.50%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/6740?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/6740?src=pr&el=footer). Last update [64c7c2b...02d9292](https://codecov.io/gh/huggingface/transformers/pull/6740?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
use_flash_attention_2 not supported for AutoConfig and/or from_config
Author: dakinggg
Pull request: no

### Feature request

Let me know if I've missed something, but I think `use_flash_attention_2` is only supported via the `from_pretrained` API. So I think I have to do something like `config._flash_attn_2_enabled = use_flash_attention_2` outside of the normal transformers API in order to initialize a model with flash attention 2 from a config. It would be great if the config based APIs supported this argument too.

### Motivation

Parity across the ways of initializing a transformers model.

### Your contribution

N/A

Comments:
cc @younesbelkada and @LysandreJik I remember we had a debate on whether or not this should be saved 😃 

Rather than being saved, I think @dakinggg is asking to be able to pass it when initializing the model with a config, so probably something like this:
```
model = BertModel(BertConfig(), use_flash_attention_2=True)
# or
model = BertModel(BertConfig(use_flash_attention_2=True))
```

This seems reasonable to me, is that what you had in mind @dakinggg ?

Yeah, but then we would need to pop it to make sure it's not saved as we wanted to prevent this use case

Yes, that is correct @LysandreJik! With one minor addition to also support it via `AutoModel.from_config(cfg)` (maybe this ends up with the same code paths you mentioned, not sure of the exact internal implementation)
[`FA-2`] Add fa2 support for `from_config`
Author: younesbelkada
Pull request: yes

# What does this PR do?

Fixes: https://github.com/huggingface/transformers/issues/26878

As per title and as asked by the community. This PR extends the FA2 API so that users can load FA2 models through `from_config` interface

Added also nice tests!

```python 
import torch
from transformers import AutoConfig, AutoModelForCausalLM

model_id = "trl-internal-testing/tiny-random-LlamaForCausalLM"

config = AutoConfig.from_pretrained(model_id)
model = AutoModelForCausalLM.from_config(config, use_flash_attention_2=True, torch_dtype=torch.float16)

print(model)
```

cc @LysandreJik @ArthurZucker 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26914). All of your documentation changes will be reflected on that endpoint.
Co-Pilot Store in Huggingface
Author: vikaskesamreddy
Pull request: no
Labels: name: Feature request

### Feature request

I think its high time we have one open source Co-pilot store to compete with the GPT Store, where we can setup, train, and add our custom documents, tools to any open source custom model and have a option to share it with the world or keep it private with the users.

### Motivation

The motivation is to provide a alternative to GPT store, where by default you can create just the GPT based models and kill the competition.

here we don't even have option to control the cost.

### Your contribution

I can help in creating the vision and Roadmap for the feature, and maybe spend sometime in development activities aswell.

Comments:
Would love to see what the community can do! 🤗
Report to none for reduce_lr_on_plateu
Author: muellerzr
Pull request: yes

# What does this PR do?

Modifies test to explicitly use `report_to="none"` rather than the default "all", as if trackers are installed it can lead to issues (https://github.com/huggingface/transformers/pull/27352#issuecomment-1819131456) and is a current failing test in Accelerate.


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@ArthurZucker 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Add OpenAI Whisper Large-v3 weights
Author: gau-nernst
Pull request: no

### Feature request

OpenAI released Whisper Large-v3.

https://github.com/openai/whisper/discussions/1762

I haven't looked into it closely but it seems the only difference is using 128-bin Mel-spec instead of 80-bin, thus weight conversion should be the same.

### Motivation

NA

### Your contribution

NA

Comments:
cc @sanchit-gandhi 

I saw that the weights are already ported here: https://huggingface.co/openai/whisper-large-v3, and the announcements made by the team. Awesome work!
Are there any reasons why this issue is not closed?

 @gau-nernst As it's your issue, you can close it if you think it's resolved. #27336 hasn't been merged yet - so full v3 compatibility hasn't been shipped - depends what your "solved" criteria are :)