noamaanMulla-03/transformers-issues huggingface dataset:
Getting high cossine similarity for any given pair of random text from the xlmr model
Author: rishabhm12
Pull request: no

### System Info

Hey guys,
I am using xlm-roberta-base with transformers==4.16.0 torch==1.13.1

For any given pair of random text, I always get a high cosine similarity. 
  ````
model_id = "xlm-roberta-base"
  tokenizer = XLMRobertaTokenizerFast.from_pretrained(model_id)
  xlmr_model = XLMRobertaModel.from_pretrained(model_id)
  
  xlmr_model.eval()
  
  def get_model_output(text, max_length=30):
      text_input = tokenizer(
      text,
      padding="max_length",
      max_length=max_length,
      truncation=True,
      return_tensors="pt",
      add_special_tokens=True
      )
      print(text_input)
      text_embedding = xlmr_model(
      **text_input
      )

    embedding = text_embedding.pooler_output.flatten().tolist()
    return embedding

def cosine_similarity(v1, v2):
    return float(np.dot(v1, v2) / (np.linalg.norm(v1) * np.linalg.norm(v2)))

text1 = "white joota with blue lace"   # (white shoes with blue lace)
text2 = "vcvbhjook  jjjjj"

embedding1 = get_model_output(text1)
embedding2 = get_model_output(text2)

print(cosine_similarity(embedding1, embedding2)) # 0.99
````
I basically want to use xlmr for a multilingual ecommerce usecase but I am stuck at this point. Let me know if I am mising something or there's some problem with the artifacts


Thank You!

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

  ````
model_id = "xlm-roberta-base"
  tokenizer = XLMRobertaTokenizerFast.from_pretrained(model_id)
  xlmr_model = XLMRobertaModel.from_pretrained(model_id)
  
  xlmr_model.eval()
  
  def get_model_output(text, max_length=30):
      text_input = tokenizer(
      text,
      padding="max_length",
      max_length=max_length,
      truncation=True,
      return_tensors="pt",
      add_special_tokens=True
      )
      print(text_input)
      text_embedding = xlmr_model(
      **text_input
      )

    embedding = text_embedding.pooler_output.flatten().tolist()
    return embedding

def cosine_similarity(v1, v2):
    return float(np.dot(v1, v2) / (np.linalg.norm(v1) * np.linalg.norm(v2)))

text1 = "white joota with blue lace"  # (white shoes with blue lace)
text2 = "vcvbhjook  jjjjj"

embedding1 = get_model_output(text1)
embedding2 = get_model_output(text2)

print(cosine_similarity(embedding1, embedding2)) # 0.99
````

### Expected behavior

Low cosine similarity as these are two random texts

Comments:
Please use the [forums](https://discuss.huggingface.co/) for questions like this.

Sure, will raise it there. Closing the issue
Metas MMS speech recognition 
Author: flozi00
Pull request: no
Labels: name: New model

### Model description

In their Blogpost they are writing it's 1B sized wav2vec 2 models, so only a new converter script should be needed?

A nice alternative to compare to whisper

### Open source status

- [ ] The model implementation is available
- [ ] The model weights are available

### Provide useful links for the implementation

https://github.com/facebookresearch/fairseq/tree/main/examples/mms

@sanchit-gandhi 
@patrickvonplaten FYI

Comments:
Interested Model!! Thank @flozi00 to open request.

👀 https://huggingface.co/models?other=mms

> :eyes:  https://huggingface.co/models?other=mms

Nice! This is the pretrained model. 
I am looking to also convert the ASR models; any pointers? 

The scripts I am looking at (such as [this very useful boi](https://huggingface.co/HfSpeechUtils/convert_wav2vec2_to_hf/blob/main/run_convert.sh)) require dict and HF config but maybe it is the same configuration as `facebook/wav2vec2-large` :thinking: 

Leeez go - working on the conversion as we speak :-) 

I also made a request in _#23811_. Looking forward to it!

PR merged.

Also see:
- https://huggingface.co/docs/transformers/main/en/model_doc/mms
- https://github.com/huggingface/transformers/pull/23813
- https://huggingface.co/facebook/mms-1b-all
Trainer: fixed evaluate raising `KeyError` for ReduceLROnPlateau
Author: claudius-kienle
Pull request: yes

If the `metric_for_best_model` does not start with `eval_`, the trainer always appends the `eval_` string before querying the metric value. Except in one code snippet, that only gets triggered if one uses the `ReduceLROnPlateau` scheduler. 
I have added the check for `eval` like in the rest of the trainer.

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@sgugger 


Comments:
_The documentation is not available anymore as the PR was closed or merged._
[Whisper Tokenizer] Skip special tokens when decoding with timestamps
Author: sanchit-gandhi
Pull request: yes

# What does this PR do?

Decoding with timestamps always returns the special tokeniser tokens, ignoring the argument `skip_special_tokens`. This PR fixes this behaviour to respect `skip_special_tokens`, both with and without timestamp decoding.

Before:
```python
from transformers import WhisperTokenizer

tokenizer = WhisperTokenizer.from_pretrained("openai/whisper-tiny")

encoded_input = [
    50258, 50363, 50364, 634, 575, 12525, 22618, 1968, 6144,
    35617, 20084, 1756, 311, 589, 307, 534, 10281, 934,
    439, 293, 50676, 50676, 393, 4411, 294, 309, 457,
    707, 295, 33301, 286, 392, 6628, 13, 50836, 50257,
]

tokenizer.decode(encoded_input, decode_with_timestamps=True, skip_special_tokens=True)
```
**Output:**
```


<|0.00|> He has grave doubts whether Sir Frederick Layton's work is really Greek after all and<|6.24|><|6.24|> can discover in it but little of rocky Ithaca.<|9.44|>

```

Now:
```python
tokenizer.decode(encoded_input, decode_with_timestamps=True, skip_special_tokens=True)
```
**Output:**
```
<|0.00|> He has grave doubts whether Sir Frederick Layton's work is really Greek after all and<|6.24|><|6.24|> can discover in it but little of rocky Ithaca.<|9.44|>
```

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Add an option to reduce compile() console spam
Author: Rocketknight1
Pull request: yes

This is a very simple PR to add an option to reduce the console spam from our `compile()` method.

Briefly, Keras model expect you to pass a loss function to `compile()`. However, `transformers` models usually compute loss internally. The solution we used was that if the user didn't pass a `loss` argument to `compile()`, we would read the model's `loss` output and use that as the loss. This was non-standard Keras behaviour, though, so we added a warning to make sure users knew what was going on.

Now that we've been doing it for a while, though, that warning is probably just more console spam. This PR adds the option to specify `loss="auto"`, which has exactly the same behaviour but eliminates the warning. The warning also includes a line telling users that they can do that.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

I think we tried that, but then most users would miss it! I think we definitely want to make users aware of what's going on (because this is the one big place we diverge from the Keras training standard), but give them an option to disable the warning once they know.

I think proper documentation is a better solution than throwing a warning all the time. Users are pretty mad at us with those already.

This could be a good opportunity to document stuff, actually! Do you prefer a sidebar tutorial, or some text in the docstring that gets added to all of our TF models?

I think in the official examples where we actually use `torch.compile` is probably the best place.

This is TF compilation, not `torch.compile()`! All of our TF examples use it - you can't call `model.fit()` in Keras before calling `model.compile()`

Sorry I meant `model.compile`. And yes, all the examples sounds about right.

Done! I added a comment when we call `compile()` in the example scripts. I also fixed up one unnecessary use of `run_eagerly` in `run_mlm.py` - oops! 

I moved the warning message to `logging.info`, so it should be invisible to most users now, and they can still pass `loss="auto"` to disable it entirely if desired.
Publish instance types best suited to finetune/inference of a popular model
Author: i-am-neo
Pull request: no

### Feature request

It would be very helpful to see a chart of instance types on the large public clouds (AWS, GCP, Oracle) most suitable for popular public LLMs like Google Flan-T5 family.

This generalizes a page from @philschmid , whose notebooks indicate how he finetuned certain models using certain instances.  You could publish the instance types on the model card broken down by inference and finetuning.

Would this be possible?

### Motivation

To help folks like me to not spin our wheels on trying to locate the most suitable vCPU-GPU combinations. It's a jungle on AWS for sure.

### Your contribution

Happy to help how I can.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Many places are type-annotated as 1-tuple when should be arbitrary length tuple
Author: anentropic
Pull request: no

### System Info

n/a

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

I've seen this in a couple of disparate places so I guess the problem is endemic.

Two examples I can point to are:

- https://github.com/huggingface/transformers/blob/main/src/transformers/modeling_outputs.py#L46
  the docstring says:
  > _"Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, + one for the output of each layer)"_
  
  but it is annotated as: `hidden_states: Optional[Tuple[torch.FloatTensor]]`
- https://github.com/huggingface/transformers/blob/main/src/transformers/models/bert/modeling_bert.py#L283
  annotated as returning `-> Tuple[torch.Tensor]` but the tuple has a varying number of elements:
  ```python
        outputs = (context_layer, attention_probs) if output_attentions else (context_layer,)

        if self.is_decoder:
            outputs = outputs + (past_key_value,)
        return outputs
  ```

In both cases there are many examples throughout those files.

### Expected behavior

Unlike `list` and `dict` etc, typed tuples have a specific size.

https://mypy.readthedocs.io/en/stable/cheat_sheet_py3.html#useful-built-in-types

the annotation `Tuple[torch.Tensor]` means a tuple with a _single element_ of that type.

for a tuple of varying size it should be annotated `Tuple[torch.Tensor, ...]`

Comments:
The type annotations should only be read as a doc helper. They are not exact, and will never be checked by a type checker as Python is not a statically typed language anyway. When we have to decide between complexity of the type annotation and ease of the user/readability, we always pick the later.

I mentioned because it confused me, since they currently describe something different from what is returned

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Change the probability of generation of certain tokens
Author: Oxi84
Pull request: no

### Feature request

OpenAI has a feature, that allows you to provide list of token ids, together with how much you want to increase or decrease probability of generation of these tokens.

For example if we have id = 112 that has logit= 23.72, we make logit = 23.72 + custom_value. Custom value  can have positive or negative value.


This should be a very usefull feature to have, I read the generation documentation and I am pretty sure you still did not implement it.

### Motivation

It is much more flexible that for example, the option to just remove a list of words 100% from generation.  And it is useful for increasing the diversity, so input tokens for example are different from output token, even though this can be partially done with encoder_repetition_penalty if I am not wrong. But here you can choose exact words, and sometimes you need to lower or increase probability of generation for just certain words.

### Your contribution

It should not be too difficult to implement, (even though there are always some problems) just in the function for totally removing some words from generation, you should instead of assigning logit of -inf, you replace the current logit value the_curent_logit + the_custom_value, where the custom value is the value users specify.






Comments:
cc @gante

Hi, @Oxi84! I agree with you on the increased flexibility and the possible implementation path. I did something like that for the feature request [here](https://github.com/huggingface/transformers/issues/22168#issue-1624460056). Feel free to react to @gante's [comment](https://github.com/huggingface/transformers/issues/22168#issuecomment-1477998997) if you find it useful

Hi @Oxi84 👋 @iiglesias-asapp said it all, see the links shared :) 

(by the looks of it, it seems the feature will have enough supporters soon!)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Fine-tune T5 model for Casual Language Modeling(CLM)
Author: nanbeitk
Pull request: no

Dear all,
I am new to NLP and has some strange questions, I try to explain them clearly.

My goal is to using a specific corpus to fine-tune t5-base model with a casual language modeling, I find this [document](https://huggingface.co/docs/transformers/main/en/tasks/language_modeling#causal-language-modeling) and it use `AutoModelForCasualLM`, but this liabrary just not include series of t5 models. 

So my question is:
1. How should I do to finetune t5 model for CLM object? In my understanding, CLM is a process of predicting `token_2` from `token_1` , `token_3` from `token_1, token_2` until the end of input sequence, so i am confused how to finish this process myself.

2. I try to spilt one my train data into something like this (ti == token_i, 1 == eos_token):
            input_ids&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;labels

-    `[t1, 1, 1, 1, 1, 1, ...]`&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;`[t1, t2, 1, 1, 1, 1, ...]`

-    `[t1, t2, 1, 1, 1, 1, ...]`&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;`[t1, t2, t3, 1, 1, 1, ...]`

-    `[t1, t2, t3, 1, 1, 1, ...]`&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;`[t1, t2, t3, t4, 1, 1, ...]`

-    `[t1, t2, t3, t4, 1, 1, ...]`&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;`[t1, t2, t3, t4, t5, 1, ...]`
The first problem is obvious, the expanded dataset is too large and requires more time to fine-tune; The second problem is that this seems strange, and I don't know if this fulfills the CLM's mission requirements. This is the only idea that i can catch up to solve this problem, does it work?

Thanks!!




Comments:
Hi, @nanbeitk thanks for raising an issue! 

This is a question best placed in our [forums](https://discuss.huggingface.co/), as we try to reserve the github issues for feature requests and bug reports.


> Hi, @nanbeitk thanks for raising an issue!
> 
> This is a question best placed in our [forums](https://discuss.huggingface.co/), as we try to reserve the github issues for feature requests and bug reports.

Thanks for your remind and i will post it to forums soon.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
trainer gradient_accumulation_steps
Author: congchan
Pull request: no

Hi, correct me if I am wrong.
I think it would be better to remove this line, as it ignore the case that last step in epoch and step is **larger** than gradient_accumulation_steps, which is quite usual.

https://github.com/huggingface/transformers/blob/c016dbdbdaf79339ae6d275d4651dc9f380be055/src/transformers/trainer.py#L1336

Comments:
cc @sgugger 

No, this line is here for the edge case of very small datasets, where one full epoch does not produce enough gradient accumulation steps, to make sure that there is still a step in that case.

Ok this seems to be a very rare case. Further more, this line will result in “drop last” for large dataset.

------------------ Original ------------------
From: Sylvain Gugger ***@***.***&gt;
Date: Wed,Nov 10,2021 3:45 AM
To: huggingface/transformers ***@***.***&gt;
Cc: cOng ***@***.***&gt;, Author ***@***.***&gt;
Subject: Re: [huggingface/transformers] trainer gradient_accumulation_steps (Issue #14309)






No, this line is here for the edge case of very small datasets, where one full epoch does not produce enough gradient accumulation steps, to make sure that there is still a step in that case.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

Hi @sgugger 
As @congchan mentioned "drop last", it is very common for large dataset.

It will cause that , some samples may be not used for optimizer.
(When total_batched_samples=15 and gradient_accumulation_steps=4, samples with 13,14,15-th batch will not be used for optimizer) 

Is this okay? As I checked the latest trainer code, it does not updated.
(https://github.com/huggingface/transformers/blob/bacaab1629972b85664fe61ec3caa4da7b55b041/src/transformers/trainer.py#L1965)
DataCollator that allows strings in datasets and untouch the strings
Author: ShengdingHu
Pull request: no

### Feature request

A default data_collator that allows strings to pass through and untouch the strings.

### Motivation

When I use `remove_unused_columns=True`， I can't  get the raw input sequences in string format in my batch.
When I use `remove_unused_columns=False`, I can't use the default data collator, because it raises the following error.
```
ValueError: Unable to create tensor, you should probably activate truncation and/or padding with 'padding=True' 'truncation=True' to have batched tensors with the same length. Perhaps your features (`translation` in this case) have excessive nesting (inputs type `list` where type `int` is expected).
```
I constantly fail into this issue and I have to implement my own data collator each time. I wonder if there will be/has been an official implementation for this feature. 

### Your contribution

I can submit a PR, but I am not sure if the feature already exists somewhere that I did not know.

Comments:
Data collators inside Transformers exist for the Trainer, which in turn send all inputs to the model directly. The model won't be able to accept the raw strings, so the use case is not obvious.

Transformers is a library of models at its core, so while we provide some functionality to train/fine-tune them easily, our goal isn't to be comprehensive :-)

But it's very easy to pop those raw strings out of the batch before passing into the model. If the raw strings are not passed into the batch, it will be hard to align the raw string to tokenized data because the data is randomly shuffled.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Is there are reason for "labels_ids" to be constrained to numeric values? (torch.long and torch.float). I'm working on a use case where the labels are represented as alphanumeric and it would be nice to preserve the original ID as it traces back to the original source of data. You can always convert alphanumeric values to numeric, but then you have to keep a dictionary offline to map it back. The other workaround would be to build our custom data collator as suggested in the original post but why write another function for this small change. I agree that it doesn't make sense to allow strings for a model that won't accept them. Can we make an exception for the "labels_ids"?