noamaanMulla-03/transformers-issues huggingface dataset:
Enable prompts on the Hub
Author: sgugger
Pull request: yes

# What does this PR do?

This PR enables users to share prompt templates for the Agent on the Hub by supporting a repo_id instead of a string for prompts. In terms of API I'm still hesitating between

1. Let the user pass a the prompt template or a repo for both `run_prompt_template` and `chat_prompt_template` which has the cons of a bit of a brittle check (to determine if we have a repo ID or a real prompt) and the repetition of the repo twice for a repo that implements both a `run_prompt_template` and a `chat_prompt_template`

2. Add a new argument `prompts_repo_id` which has the cons of being a new argument in a public API and to add some checks if the repo only implements one of the prompts but not both.

I went with 1 in this PR but curious to have your advice.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
resume_from_checkpoint is not used in TrainingArguments
Author: SingL3
Pull request: no

### System Info

N/A

### Who can help?

@sgugger 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

Just pass a `resume_from_checkpoint` to `TrainingArguments`.

### Expected behavior

I have read the code and I see `resume_from_checkpoint` is used for `Trainer.train(resume_from_checkpoint=...)`, and this setting in `TrainingArguments` is not used. Any reasons?

Comments:
@SingL3 In the [docs for TrainingArguments](https://huggingface.co/docs/transformers/v4.29.1/en/main_classes/trainer#transformers.TrainingArguments.resume_from_checkpoint), it notes that `resume_from_checkpoint` is intended to be used in your own training/evaluation scripts. In the original PR, [this was discussed and decided upon](https://github.com/huggingface/transformers/pull/11492#discussion_r622339347) in order to remove ambiguity. 

I see but it is somehow confusing. Closing this issue.
Suppress warning when using `DataCollatorForSeq2Seq`
Author: pietrolesci
Pull request: no

### Feature request

When using a fast tokenizer in the `DataCollatorForSeq2Seq`, currently the following warning is printed 

```
You're using a T5TokenizerFast tokenizer. Please note that with a fast tokenizer, using the `__call__` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding.
```

As a temporary solution, what is the suggested way to suppress it?

Is there any plan to update `DataCollatorForSeq2Seq`?

Thanks a lot in advance for your help!

### Motivation

N/A

### Your contribution

N/A

Comments:
cc @sgugger 

Could you post a clear reproducer of the issue? Thanks a lot!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

> Could you post a clear reproducer of the issue? Thanks a lot!

When fine tuning a DialoGPT-medium model with a custom Dataset class as so:

```
class ConversationDataset(Dataset):
    def __init__(self, tokenizer, file_path, block_size):
        self.tokenizer = tokenizer
        self.block_size = block_size
        self.inputs = []
        self.responses = []

        with open(file_path, 'r', encoding="utf-8") as file:
            lines = file.readlines()
            for i in range(0, len(lines), 2):
                if i + 1 < len(lines):
                    self.inputs.append(lines[i].strip().replace("input: ", ""))
                    self.responses.append(lines[i + 1].strip().replace("response: ", ""))

        # Tokenize and pad inputs and responses in a single step
        self.input_tensors = tokenizer(self.inputs, return_tensors='pt', padding=True, truncation=True, max_length=self.block_size)
        self.response_tensors = tokenizer(self.responses, return_tensors='pt', padding=True, truncation=True, max_length=self.block_size)
```

This is being fixed by #23742
Add LlamaIndex to awesome-transformers.md
Author: ravi03071991
Pull request: yes

# What does this PR do?

This PR adds `LlamaIndex` to `awesome-transformers.md`. `LlamaIndex` is a project that provides a central interface to connect your LLM's with external data.

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Interestingly, you have `transformers` as an optional dependency when the Python version is above 3.9. It seems to be used quite a bit for tokenizing/streaming/image captioning etc. 

If you'd rather not have it as a main dependency, I'd love to hear why so that we may improve! We've been trying to keep the package on the lighter side of dependencies so that it wasn't too heavy to add.

Thank you!
Fix psuh_to_hub in Trainer when nothing needs pushing
Author: sgugger
Pull request: yes

# What does this PR do?

This PR fixes `push_to_hub` in the `Trainer`. Since `Repository.push_to_hub`can return `None` or a tuple, we have to do a small test before unpacking the output.

Fixes #23712

Comments:
_The documentation is not available anymore as the PR was closed or merged._
LongformerEmbeddings "position_embedding_type"  parameter are not used.
Author: doveppp
Pull request: no

### System Info

- `transformers` version: 4.28.1
- Platform: Linux-5.4.0-144-generic-x86_64-with-glibc2.28
- Python version: 3.9.7
- Huggingface_hub version: 0.13.4
- Safetensors version: not installed
- PyTorch version (GPU?): 2.0.0+cu117 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@sgugger 
@ArthurZucker 
@younesbelkada 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

In transformers.models.longformer.modeling_longformer.py file:
```python
class LongformerEmbeddings(nn.Module):
    """
    Same as BertEmbeddings with a tiny tweak for positional embeddings indexing.
    """

    def __init__(self, config):
        super().__init__()
        self.word_embeddings = nn.Embedding(config.vocab_size, config.hidden_size, padding_idx=config.pad_token_id)
        self.position_embeddings = nn.Embedding(config.max_position_embeddings, config.hidden_size)
        self.token_type_embeddings = nn.Embedding(config.type_vocab_size, config.hidden_size)

        # self.LayerNorm is not snake-cased to stick with TensorFlow model variable name and be able to load
        # any TensorFlow checkpoint file
        self.LayerNorm = nn.LayerNorm(config.hidden_size, eps=config.layer_norm_eps)
        self.dropout = nn.Dropout(config.hidden_dropout_prob)

        self.position_embedding_type = getattr(config, "position_embedding_type", "absolute")

        self.padding_idx = config.pad_token_id
        self.position_embeddings = nn.Embedding(
            config.max_position_embeddings, config.hidden_size, padding_idx=self.padding_idx
        )
```

"position_embedding_type" are not work. By the way, self.position_embeddings has redundant initialization

### Expected behavior

Add support for “position_embedding_type”

Comments:
Hey! Thanks for reporting this. I'm opening a PR to remove the unused parts, however I don't think it has to support the `position_embedding_type` as the model did not use it.
[LongFormer] code nits, removed unused parameters 
Author: ArthurZucker
Pull request: yes

# What does this PR do?
LongformerEmbeddings "position_embedding_type"  parameter are not used. 
Fixes #23730


Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @sgugger

Integrations:

- deepspeed: HF Trainer: @stas00, Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam

Documentation: @sgugger, @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: @sgugger
- TensorFlow: @Rocketknight1

 -->


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Okay, seems there was this #23343 but they are compatible
Can not import T5BiLDModel
Author: sufeidechabei
Pull request: no

### System Info

I try from transformers.models.t5.modeling_t5 import T5Model, but it doesn't work. I build the library from this repo @ArthurZucker @younesbelkada 

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

from transformers.models.t5.modeling_t5 import T5BiLDModel

### Expected behavior

It can import the model.

Comments:
Hi @sufeidechabei, thanks for raising an issue. 

So that we can best help you, could you follow the issue template and share information about the running environment (run `transformers-cli env` in your terminal and share what's printed out). 

Could you elaborate on what you mean by ` I build the library from this repo`? Is this running from a fork of the repo or from code on the hub? 

From the code for this hub @amyeroberts 


Here is the print information:ImportError: cannot import name 'T5BiLDModel' from 'transformers.models.t5.modeling_t5' (/nobackup/haozhang/venv/lib/python3.9/site-packages/transformers/models/t5/modeling_t5.py). I use python 3.9.13 and pytorch 2.0

@sufeidechabei  As requested, could you please share the information printed out when you run `transformers-cli env` in your terminal? 

Could you also point to the code on the hub which has the model implementation? 

- `transformers` version: 4.25.1
- Platform: Linux-5.15.0-69-generic-x86_64-with-glibc2.35
- Python version: 3.9.13
- Huggingface_hub version: 0.11.1
- PyTorch version (GPU?): 1.13.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

@amyeroberts 

@sufeidechabei  - it's not possible to directly import the model in the following way:

```
from transformers.models.t5.modeling_t5 import T5BiLDModel
```
as `T5BiLDModel` isn't a model in the `modeling_t5` module. 

It's possible to use checkpoints from models defined on the hub using the `AutoModel` API. See documentation [here](https://huggingface.co/docs/transformers/custom_models#using-a-model-with-custom-code). 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
[`Nllb-Moe`] Fix nllb moe accelerate issue
Author: younesbelkada
Pull request: yes

# What does this PR do?

Fixes: https://github.com/huggingface/transformers/issues/23385 

Before this PR, it seemed that `_no_split_modules` was not properly set. Due to the skip connections in `NllbMoeEncoderLayer` and `NllbMoeDecoderLayer` one needs to add these modules inside `_no_split_modules` instead of `NllbMoeAttention`.

All accelerate tests pass

cc @ArthurZucker @sgugger 


Comments:
_The documentation is not available anymore as the PR was closed or merged._
considering add some logic here, responsing with some text like "I don't know" when the model output is with some probs lower than threshold ,which means that it is not that confident.
Author: antibits
Pull request: no

https://github.com/huggingface/transformers/blob/f67dac97bdc63874f2288546b3fa87e69d2ea1c8/src/transformers/generation/utils.py#LL2650C69-L2650C69

considering add some logic here, responsing with some text like "I don't know" when the model output is with some probs lower than threshold ,which means that it is not that confident.

Comments:
stopping_criteria may works
Verify that a test in `LayoutLMv3` 's tokenizer is checking what we want
Author: SaulLu
Pull request: no

I'm taking the liberty of opening an issue to share a question I've been keeping in the corner of my head, but now that I'll have less time to devote to `transformers` I prefer to share it before it's forgotten.

In the PR where the `LayoutLMv3` model was added, I was not very sure about the target value used for one of the tests that had to be overridden (the value was 1 in one of the previous commits and then changed to 0). The comment I am referring to is this one: https://github.com/huggingface/transformers/pull/17060#discussion_r872265358 .  

Might be of interest to @ArthurZucker 

Comments:
Thanks! Will try to get familiar with this 😉
use_fast=False when loading OPT's tokenizer?
Author: jiangwangyi
Pull request: no

### System Info

platform==Ubuntu 18.04.01
python==3.10
transformers==4.29.1

### Who can help?

@sgugger @stevhliu @MK

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

It is shown in the OPT model [documentation](https://huggingface.co/docs/transformers/model_doc/opt) (in Tips) that it is required to pass `use_fast=False` when loading a OPT tokenizer, as OPT tokenizer will add `
` to the beginning of every prompt.

I made a trial:
```python
>>> import transformers
>>> tokenizer = transformers.AutoTokenizer.from_pretrained("facebook/opt-1.3b", use_fast=False)
>>> tokenizer_fast = transformers.AutoTokenizer.from_pretrained("facebook/opt-1.3b", use_fast=True)
>>> text = "I like you."
>>> tokenizer(text)
>>> {'input_ids': [2, 100, 101, 47, 4], 'attention_mask': [1, 1, 1, 1, 1]}
>>> tokenizer_fast(text)
>>> {'input_ids': [2, 100, 101, 47, 4], 'attention_mask': [1, 1, 1, 1, 1]}
```

`
` is correctly added and no difference is observed.

### Expected behavior

Is the tips wrong or in some other cases `use_fast=Fast` is actually required?

Comments:
cc @ArthurZucker 

Hey! Thanks for reporting. Pretty sure the doc is wrong, but `use_fast=True` use to not be supported for OPT, which could explain this.
[HF Trainer] [PyTorch FSDP] Add support for backward_prefetch, forward_prefetch, limit_all_gathers
Author: cavdard
Pull request: no

### **Feature request**
Can we add Trainer support for the following [FSDP](https://pytorch.org/docs/1.13/fsdp.html?highlight=fsdp#module-torch.distributed.fsdp) features? `backward_prefetch`, `forward_prefetch` and  `limit_all_gathers`


### **Motivation**

`backward_prefetch`, `forward_prefetch` and  `limit_all_gathers` are important to achieve best performance when training at scale with FSDP.

Comments:
cc @pacman100 

There are also 2 newly added[ sharding strategies](https://pytorch.org/docs/master/fsdp.html#torch.distributed.fsdp.ShardingStrategy) with PyTorch 2.0: `HYBRID_SHARD` and `_HYBRID_SHARD_ZERO2` that can impact performance. 

The ^^ above fix only addes backward_prefetch and forward_prefetch options in fsdp, limit_all_gathers and other sharding strategies is not available with current pytorch version used in repo.

Hi! These two fixes still doesn't allow the following two strategies
> There are also 2 newly added[ sharding strategies](https://pytorch.org/docs/master/fsdp.html#torch.distributed.fsdp.ShardingStrategy) with PyTorch 2.0: HYBRID_SHARD and _HYBRID_SHARD_ZERO2 that can impact performance.

We could potentially enable it by adding two conditions here? https://github.com/huggingface/transformers/blob/118e9810687dd713b6be07af79e80eeb1d916908/src/transformers/trainer.py#L453-L458

> Hi! These two fixes still doesn't allow the following two strategies
> 
> > There are also 2 newly added[ sharding strategies](https://pytorch.org/docs/master/fsdp.html#torch.distributed.fsdp.ShardingStrategy) with PyTorch 2.0: HYBRID_SHARD and _HYBRID_SHARD_ZERO2 that can impact performance.
> 
> We could potentially enable it by adding two conditions here?
> 
> https://github.com/huggingface/transformers/blob/118e9810687dd713b6be07af79e80eeb1d916908/src/transformers/trainer.py#L453-L458

Yes, I will open a PR shortly.
can we use ckpt model file generated after finetuning the pre-trained models on custom dataset
Author: SagarPalyal
Pull request: no
Labels: name: wontfix

Hi Team,

I have fine tuned pegasus huggingface wikihow model with my own custom dataset and end up with below files:

model.ckpt-1000.data-00000-of-00001
model.ckpt-1000.index
model.ckpt-1000.meta
events.out.tfevents.1608192436.ip-xxx-xx-x-xx
events.out.tfevents.1608192510.ip-xxx-xx-x-xx.v2

When I am running below code, it is taking pytorch_model.bin file automatically for generating summary. Where as I want to use ckpt model file for generating summary which I got after training.

**Code I am using is :**

model_path ='local-pegasus-wikihow'

torch_device = 'cuda' if torch.cuda.is_available() else 'cpu'
tokenizer = PegasusTokenizer.from_pretrained(model_path)

model = PegasusForConditionalGeneration.from_pretrained(model_path).to(torch_device)

batch = tokenizer.prepare_seq2seq_batch(src_text, truncation=True, padding='longest', return_tensors="pt").to(torch_device)
translated = model.generate(**batch)
tgt_text = tokenizer.batch_decode(translated, skip_special_tokens=True)
print(tgt_text[0])

Do I need to change code here so that it will take ckpt model file for generating summary instead of searching for pre-trained pytorch_model.bin or .h5 file?

Comments:
hi @SagarPalyal  not sure what you mean here. by ckpt file do you mean a saved checkpoint ?

Yes you are right. This is saved checkpoint file only.

One thing to note: `pytorch_model.bin` is the name of the weights file, and every torch HF model will have that file.

To load the checkpoint simply pass the path for the checkpoint instead  of `model_path`

Hey, @SagarPalyal did you solve the problem? I met the same problem. 'from_pretrained' function needs pytorch model file, not tensorflow model file. I don't know how to convert a costumed pegasus tensorflow model to pytorch model. Anyone knows??? 

I am not able to figure out how to convert .ckpt file to pytorch model file but in your case if you have tensorflow model file then you can use parameter from_tf=True

This issue has been automatically marked as stale and been closed because it has not had recent activity. Thank you for your contributions.

If you think this still needs to be addressed please comment on this thread.
Weird behavior for initial tokens in BERT Base Cased
Author: mawilson1234
Pull request: no

### System Info

transformers version: 4.27.4
python version: 3.8.8

### Who can help?

@ArthurZucker @younesbelkada 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

I'm running a simple MLM task using BERT Base Cased. I'm noticing weird behavior when decoding the first token (after the CLS token) in the output. Here's an example:

```python
from transformers import AutoModelForMaskedLM, AutoTokenizer
import torch

model = AutoModelForMaskedLM.from_pretrained('bert-base-cased')
tokenizer = AutoTokenizer.from_pretrained('bert-base-cased')

inputs = tokenizer(['The laws have done [MASK] harm.'], return_tensors='pt')

with torch.no_grad():
    outputs = model(**inputs)

tokenizer.batch_decode(torch.argmax(outputs.logits, dim=-1))
```
This produces the output: `.. laws have done no harm..`. I know the first and last dots correspond to predictions for the CLS and EOS tokens, so they should be ignored, but the second dot is where `The` should be. This happens with a variety of words in many sentences, but it doesn't always happen for the same words. It does seem to be paying attention to this initial word even when it is not produced, since the results differ depending on the initial word, even if it's not decoded from the output. But it looks weird. Is this normal behavior?

When I use the fill-mask pipeline, I get a different result, but I'm assuming that the pipeline just internally uses string replacement for the mask token rather than actually decoding the full output.
```python
from transformers import pipeline
pipe = pipeline('fill-mask', 'bert-base-cased')
pipe('The laws have done [MASK] harm.')[0]['sequence']
```
Produces `The laws have done no harm.`, as expected.


### Expected behavior

I'd expect that given tokens would be retained as is, for the most part. Sentence initial `The` and `I` seem to cause this problem a lot, which is odd, given I'd expect those to be well-attested in the training data.

Comments:
Hey! Thanks for reporting, I believe that this is somewhat expected, as the `mask-fill` pipeline does not exactly use just `argmax`. There is a bit more process involved in how to obtain the correct output. This is normal! 🤗 

I understand why the `fill-mask` output differs from the output when using `argmax` now, but is it still expected that it predicts `.` instead of `The` when using `argmax`?

No I believe that the most important is that it correctly predicts the masked word, which the loss will be computed on. Other tokens are ignored

Got it, thanks!