noamaanMulla-03/transformers-issues huggingface dataset:
Add DistilBERTForCausalLM
Author: leonjovanovic
Pull request: yes

# What does this PR do?

Similar to the BertLMHeadModel this PR aims to add a DistilBertForCausalLM model in modeling_distilbert.py.

Fixes https://github.com/huggingface/transformers/issues/7397

Replaces https://github.com/huggingface/transformers/pull/8387, https://github.com/huggingface/transformers/pull/11085

## Who can review?

@patrickvonplaten
@ArthurZucker
@younesbelkada 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_22614). All of your documentation changes will be reflected on that endpoint.

There is no checkpoint available for this task and DistilBERT, which is an encoder model. What is your usecase for adding this?

> There is no checkpoint available for this task and DistilBERT, which is an encoder model. What is your usecase for adding this?

@sgugger To fine-tune DistilBERT models for text generation with EncoderDecoder class.

How is it different than using BERT?

@sgugger It is not possible to create Transformer model with EncoderDecoderModel using DistilBERT checkpoint (e.g. BertConfig is supported, but DistilBertConfig is not).
If I try to create Encoder Decoder model with distilbert checkpoints like this:
`
model_name = 'distilbert-base-multilingual-cased'
model = EncoderDecoderModel.from_encoder_decoder_pretrained(model_name, model_name)
`

Error is raised: 
> ValueError: Unrecognized configuration class  for this kind of AutoModel: AutoModelForCausalLM.
> Model type should be one of BartConfig, BertConfig, BertGenerationConfig, BigBirdConfig, BigBirdPegasusConfig...

@sgugger 
@patrickvonplaten
@ArthurZucker
@younesbelkada

Anyone looking at this?

Those are not changes we want everyone to have in the distilBERT: it makes the model code too unreadable just so that you can use it in the EncoderDecoder framework. We can leave the fork open if you want to share it with others, and you can also push it in any repo on the Hub using the dynamic code feature.

I followed same code structure as in BERT. Its not only for EncoderDecoder, your current version doesnt allow usage of DistilBERT in Text generation, which can be useful.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Removing one of the twice defined position_embeddings in LongFormer
Author: GregorySenay
Pull request: yes

# What does this PR do?

The self.position_embeddings in LongFormerEmbeddings is defined twice. Removing the first without padding_idx
l. 451/452

- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline]

## Who can review?
- text models: @ArthurZucker and @younesbelkada


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Still cannot import cached_path
Author: Fshrink
Pull request: no

### System Info

- `transformers` version: 4.29.1
- Platform: Linux-5.19.0-41-generic-x86_64-with-glibc2.36
- Python version: 3.11.3
- Huggingface_hub version: 0.14.1
- Safetensors version: 0.3.1
- PyTorch version (GPU?): 2.0.1 (False)
- Tensorflow version (GPU?): 2.12.0 (False)
- Flax version (CPU?/GPU?/TPU?): 0.6.10 (cpu)
- Jax version: 0.4.10
- JaxLib version: 0.4.10
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: ?



### Who can help?

@sanchit-gandhi   I guess...

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

from transformers import cached_path

be it as a single instruction in ipython or when using the ConvAIModel from simpletransformers

Tried all versions from before 4.22.0

### Expected behavior

Import the method

Comments:
I don't think that that exists. Did you mean `default_cache_path` or maybe `TRANSFORMERS_CACHE`? Those are in file_utils (and actually utils/hub)

```python
from transformers.file_utils import default_cache_path, TRANSFORMERS_CACHE
```

https://github.com/huggingface/transformers/blob/cf11493dce0a1d22446efe0d6c4ade02fd928e50/src/transformers/utils/hub.py#L80-L102

Bram,
Thanks so much for answering!
This is what I get from running the model:

![Screenshot from 2023-05-14 09-08-29](https://github.com/huggingface/transformers/assets/132883741/bb9a0b2f-8e9d-4981-b239-c1b4f157c82e)

Which refers to this part of the >>>miniconda3/envs/ST2/lib/python3.11/site-packages/simpletransformers/conv_ai/conv_ai_utils.py:

![Screenshot from 2023-05-14 09-16-11](https://github.com/huggingface/transformers/assets/132883741/4871b7ed-5698-4375-8bdb-d5cb5d0d76b3)

And that's where all stops since this function is used across the script and creates objects that are used by the model.py.

Of course this is a simpletransformers code , but it requests an import from transformers that apparently does not exist...

Thanks!



Unfortunately that is not a problem of `transformers` but of `simpletransformers`. In their requirements, they specify `"transformers>=4.6.0"` so it is possible that they do not test against/support the most recent versions.

https://github.com/ThilinaRajapakse/simpletransformers/blob/365b27feb27e8337a7f4f0244eff8683c5763ef8/setup.py#L29

I suggest that you try 4.6.0 and if that does not work, you should ask them on their repository because there is not much that can be done on this end.


Will do. Appreciate it.

Thanks @BramVanroy! And best of luck @Fshrink with getting your `simpletransformers` script working @Fshrink!
Added type hints for `Graphormer` pytorch version
Author: dewasahu2003
Pull request: yes

@Rocketknight1  👋
- I added type hint for `graphormer` pytorch 
- checked formatting with black and ruff

if some checks on ci/cd do not, please do comment and correct

Comments:
_The documentation is not available anymore as the PR was closed or merged._

This looks pretty good! Is there a reason to use `Union[torch.Tensor, torch.LongTensor]` instead of just `torch.LongTensor`?

@Rocketknight1 Hi 👋
- `Union[torch.Tensor, torch.LongTensor]` is used because the file had a lot of `nn.embedding` instances which expects either IntTensor or LongTensor
- so to avoid any confusion 😕 i used that
- [nn.embedding docs](https://pytorch.org/docs/stable/generated/torch.nn.Embedding.html#torch.nn.Embedding)

![image](https://user-images.githubusercontent.com/95997298/235670386-6d4fd5e1-84d7-4023-ac7f-58aad7a25255.png)

*if still changes are required i would be happy to make it*🙂


Hi @dewasahu2003, I think in most cases we just annotate those types as `LongTensor`! Your version is probably more correct, but for simplicity just `LongTensor` is fine, since that's what people usually use.

@Rocketknight1 Hi 👋 
- if LongTensor is  preferred then i would make changes along
- that would help the code to be 🤒 bloat free


Yep, I think replacing with LongTensor is slightly better, and does make the code a bit cleaner too.

Sure 

Done. Thanks for the PR, we really appreciate it!
Fix test typos - audio feature extractors
Author: LWprogramming
Pull request: yes

# What does this PR do?

Fixes typos I discovered when I was writing [#23309](https://github.com/huggingface/transformers/pull/23309)


## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sanchit-gandhi 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Add Unlimiformer to 🤗 transformers
Author: tanaymeh
Pull request: no
Labels: name: New model

### Model description

I want to add the recently released Unlimiformer (Long-Range Transformers with Unlimited Length Input) model to 🤗 transformers.

Will this be a good addition?

### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

Paper: https://arxiv.org/abs/2305.01625
Code: https://github.com/abertsch72/unlimiformer
Model weights: https://github.com/abertsch72/unlimiformer#trained-models

Authors: @abertsch72 
cc @LysandreJik 

Comments:
Replace NumPy Operations with JAX NumPy Equivalents for JIT Compilation Compatibility
Author: gojiteji
Pull request: yes

# What does this PR do?
This PR modifies the Transformers library to replace NumPy operations with their JAX NumPy equivalents. The main change is the use of JAX's immutable update methods as substitutes for in-place assignments.

Using NumPy methods instead of JAX NumPy results in an error during jit compilation (jit).

```
transformers/models/mbart/modeling_flax_mbart.py", line 226, in shift_tokens_right
    prev_output_tokens = np.array(input_ids).copy()
jax._src.errors.TracerArrayConversionError: The numpy.ndarray conversion method __array__() was called on the JAX Tracer object Traced<ShapedArray(int32[4,260])>with<DynamicJaxprTrace(level=0/1)>
```

Here's a brief summary of the changes:

previous:
```python
    prev_output_tokens = np.array(input_ids).copy()

     if pad_token_id is None:
         raise ValueError("self.model.config.pad_token_id has to be defined.")

     # replace possible -100 values in labels by `pad_token_id`
     prev_output_tokens = np.where(prev_output_tokens == -100, pad_token_id, input_ids)
     index_of_eos = (np.where(prev_output_tokens != pad_token_id, 1, 0).sum(axis=-1) - 1).reshape(-1, 1)
     decoder_start_tokens = np.array(
         [prev_output_tokens[i, eos_idx] for i, eos_idx in enumerate(index_of_eos)], dtype=np.int32
     ).squeeze()

     prev_output_tokens[:, 1:] = prev_output_tokens[:, :-1].copy()
     prev_output_tokens[:, 0] = decoder_start_tokens


```

modefied:
```python
     prev_output_tokens = jnp.array(input_ids).copy()

     if pad_token_id is None:
         raise ValueError("self.model.config.pad_token_id has to be defined.")

     # replace possible -100 values in labels by `pad_token_id`
     prev_output_tokens = jnp.where(prev_output_tokens == -100, pad_token_id, input_ids)
     index_of_eos = (jnp.where(prev_output_tokens != pad_token_id, 1, 0).sum(axis=-1) - 1).reshape(-1, 1)
     decoder_start_tokens = jnp.array(
         [prev_output_tokens[i, eos_idx] for i, eos_idx in enumerate(index_of_eos)], dtype=jnp.int32
     ).squeeze()

     prev_output_tokens = prev_output_tokens.at[:, 1:].set(prev_output_tokens[:, :-1])
     prev_output_tokens = prev_output_tokens.at[:, 0].set(decoder_start_tokens)
```

- @sanchit-gandhi






Comments:
_The documentation is not available anymore as the PR was closed or merged._

done✅ https://github.com/huggingface/transformers/pull/23356/commits/175ab5dd0b8276f70b69ab21ddb4356aa353d611

@gojiteji To resolve the failing quality tests, you'll need to run `make fix-copies` and `make style` and push the changes. It seems the MT5 also doesn't have `jnp` defined in the modeling file.
Update `test_batched_inference_image_captioning_conditioned`
Author: ydshieh
Pull request: yes

# What does this PR do?

The test `tests/models/pix2struct/test_modeling_pix2struct.py::Pix2StructIntegrationTest::test_batched_inference_image_captioning_conditioned` starts to fail on CI run of `April 27` which includes the merged PR #23023.

@younesbelkada Could you double check if the changes in this PR are reasonable? Thank you.



Comments:
_The documentation is not available anymore as the PR was closed or merged._

@younesbelkada If you can take over this PR to avoid `generate weird output right after`, it would be really nice :-)

The test should be now fixed, the generated text produces different output than before, probably due to https://github.com/huggingface/transformers/pull/23051 that now made the model using a causal attention mask on the text decoder (which was not the case before)

Thanks a lot!

> I think that we should educate users that for text-conditioned generation we should never add special tokens to the tokenizer - as introduced in https://github.com/huggingface/transformers/pull/23004

@younesbelkada The best place to do this I think is in the example docstring for the model, as this is what a lot of users will reference, and it currently doesn't do that. Could you open a PR to update this? 

> @younesbelkada The best place to do this I think is in the example docstring for the model, as this is what a lot of users will reference, and it currently doesn't do that. Could you open a PR to update this?

Sure yes will do! 

> Changes look fine - my only concern is that the generations appear to have become worse. @younesbelkada @ydshieh do we have any other generation samples to make sure the model is behaving as expected?

Yes! I was relieved since we do have the tests `test_batched_inference_image_captioning` & `test_inference_image_captioning` that still pass --> meaning that the un-conditional text generation seem to be unaffected!

> > @younesbelkada The best place to do this I think is in the example docstring for the model, as this is what a lot of users will reference, and it currently doesn't do that. Could you open a PR to update this?
> 
> Sure yes will do!

I am going to merge this PR and leave @amyeroberts 's suggestion for @younesbelkada in a separate PR. Thank you for the review and the refine of this PR.
OPT/BioGPT: Improved attention mask shape exception
Author: gante
Pull request: yes

# What does this PR do?

Related exception: #23197

Currently, in OPT/BioGPT, if we don't pass an attention mask or if we pass an attention mask with a wrong shape, an exception will be printed in the attention layer: `Attention mask should be of size {(bsz, 1, tgt_len, src_len)}, but is {attention_mask.size()}`. This exception has the following problems:
1. It checks the expanded attention mask, not the attention mask as set by the user (or the default). Even as a maintainer, I can't immediately decode the error message, as I would need to know how the mask is expanded for the model in question.
2. If there is a bug computing `bsz`, `tgt_len`, or `src_len`, the exception will be misleading.

In #23197 we found that when the length of `past_key_values` is equal to the length of the `attention_mask`, `tgt_len` and `src_len` will be wrong (in at least these 2 models), triggering the exception with an incorrect message. This PR solves both issues: it prevents the incorrect computation of `tgt_len` and `src_len` by checking the shape of `attention_mask` in the main model class, printing a user-friendly message.

If this PR gets approved, I will add a similar check to the other models.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
[Llama Tokenizer] Fast llama template
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Adresses #22794 and #22877

Comments:
_The documentation is not available anymore as the PR was closed or merged._

The test `tests/models/llama/test_tokenization_llama.py::LlamaIntegrationTest::test_conversion` is failing since April 27th, which is likely due to this PR. See issue page #23400
Minor fixes in transformers-tools
Author: Wauplin
Pull request: yes

Really just a few things as I dig a bit into the implementation of transformers-tools:
- `upload_folder` instead of `os.listdir` + `create_commit` (more robust against recursion)
- some typing
- use `metadata_update` with correct `repo_id` when pushing to Hub
- use `build_hf_headers` instead of `HfFolder` for token retrieval
- use `super().__init__()` and `super().setup()` in `PipelineTool` (otherwise the pipeline is setup again at each run)

Comments:
_The documentation is not available anymore as the PR was closed or merged._
[`Pix2Struct`] Add conditional generation on docstring example
Author: younesbelkada
Pull request: yes

# What does this PR do?

As discussed in https://github.com/huggingface/transformers/pull/23391#issuecomment-1549555750 - this PR adds an example for users to run conditional generation using pix2struct. In fact, users shouldn't add special tokens when pre-pending the text - therefore it should be explicitly mentioned in the docs (done on the aformentioned PR) but also on the example snippets.

cc @amyeroberts 


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Generate: faster `can_generate` check on TF and Flax
Author: gante
Pull request: yes

# What does this PR do?

Same as https://github.com/huggingface/transformers/pull/22643, but on TF and Flax.

[This comment](https://github.com/huggingface/transformers/pull/22643#issuecomment-1501033074) shows that it reduces the exec time of this line from 1-500 ms to <0.01 ms

Comments:
_The documentation is not available anymore as the PR was closed or merged._
is it possible to add `system prompt` to  Blenderbot ?
Author: SKbarbon
Pull request: no

This is a simple `BlenderBot` app:

```python
from transformers import BlenderbotSmallTokenizer, BlenderbotSmallForConditionalGeneration
import os

class BlenderBot:
    def __init__(
        self,
        model_name: str ='facebook/blenderbot_small-90M',
    ):
        if not os.path.exists('./models/blenderbot'):
            BlenderbotSmallForConditionalGeneration.from_pretrained(model_name).save_pretrained('./models/blenderbot')
            BlenderbotSmallTokenizer.from_pretrained(model_name).save_pretrained('./models/blenderbot')

        self.model = BlenderbotSmallForConditionalGeneration.from_pretrained('./models/blenderbot')
        self.tokenizer = BlenderbotSmallTokenizer.from_pretrained('./models/blenderbot')

    def __call__(self, inputs: str) -> str:
        inputs_tokenized = self.tokenizer(inputs, return_tensors='pt')
        reply_ids = self.model.generate(**inputs_tokenized)
        reply = self.tokenizer.batch_decode(reply_ids, skip_special_tokens=True)[0]

        return reply

    def run(self):
        while True:
            user_input = input("User: ")
            print("Bot:", self(user_input))
```

The problem is i dont know how to add any system prompts to manage the out puts of the chatbot. Any help with that ?

Comments:
Hi @SKbarbon, thanks for raising an issue! 

This is a question best placed in our [forums](https://discuss.huggingface.co/). We try to reserve the github issues for feature requests and bug reports.

Ah I am sorry @amyeroberts !
Fix translation no_trainer
Author: muellerzr
Pull request: yes
Labels: name: Examples

# What does this PR do?

This PR fixes the reason translation has been failing, by adding in the same `num_beams` that were found to be used in the test.

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sgugger, cc @ydshieh 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@sgugger it hasn't and has been failing for some time, due to the fact `num_beams` was `None` (the default in the CLI), and we need to pass it in. When checking the diff/blame there was not anything explicitly that changed in this file to have this happen, however this is the fix that is needed for the test to pass. 

Ah ok, got confused because it's slow.
Model isn't loaded with the right type with AutoModel with torch_dtype="auto" 
Author: eladsegal
Pull request: no

### System Info

- `transformers` version: 4.30.0.dev0

### Who can help?

@stas00 
(as the relevant change was made in https://github.com/huggingface/transformers/pull/21524)

### Reproduction

```python
from transformers import AutoModelForSeq2SeqLM, T5ForConditionalGeneration

auto_model = AutoModelForSeq2SeqLM.from_pretrained("ybelkada/flan-t5-xl-sharded-bf16", torch_dtype="auto")
print(auto_model.dtype)  # torch.float32

t5_model = T5ForConditionalGeneration.from_pretrained("ybelkada/flan-t5-xl-sharded-bf16", torch_dtype="auto")
print(t5_model.dtype)  # torch.bfloat16
```


### Expected behavior

`AutoModelForSeq2SeqLM` should also load the model in `torch.bfloat16`. 

Comments:
cc @younesbelkada 

Hi @eladsegal 👋
![image](https://github.com/huggingface/transformers/assets/95997298/ee869958-221f-4605-a741-ef8058d32cb0)
- The model was saved in `bfloat16` with `"T5ForConditionalGeneration"` architecture so the model was loaded in `bfloat16`
-  But in `AutoModel.from_pretrained ` method torch_dtype is set to `auto` and you can read the doc ( image i have uploaded) that dtype picked by `auto` is generally `float32`
- Hope it helps , If i misunderstood your question ,Please 🙏 give a feedback

**You can check this --> [doc](https://huggingface.co/docs/transformers/v4.29.1/en/main_classes/configuration#transformers.PretrainedConfig.torch_dtype)**

Thank you for the report, @eladsegal - that's indeed a bug that I introduced while trying to fix another issue. 

Please try this fix: https://github.com/huggingface/transformers/pull/23379

Thank you @stas00 for the quick fix! Works just as expected. 

Thank you for confirming that, @eladsegal!