noamaanMulla-03/transformers-issues huggingface dataset:
Fix typo
Author: ihoromi4
Pull request: yes

# What does this PR do?

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Comments:
Discrepancy in tokenization results using HF's AlbertTokenizer and sentencepiece library
Author: anjali-chadha
Pull request: no

Hi -

I recently noticed that tokenized results from AlbertTokenizer and sentencepiece library differ for some inputs. Check below:

```
!pip install sentencepiece

import sentencepiece as spm
sp = spm.SentencePieceProcessor()
sp.load('<SPM_MODEL>')
print(sp.encode_as_pieces('3.0,'))
print(sp.encode_as_ids('3.0,'))

Output:
['▁3.0,']
[72369]
```


```
! pip install transformers

from transformers import AlbertTokenizer
tokenizer = AlbertTokenizer.from_pretrained('<SPM_MODEL>')
print(tokenizer._tokenize("3.0,"))
print(tokenizer._convert_token_to_id(tokenizer._tokenize("3.0,")))

Output:
['▁3.0', ',']
[16047, 254713]
```

After looking at AlbertTokenizer codebase, I see that the if condition here is leading to the differences in the outputs above. https://github.com/huggingface/transformers/blob/master/src/transformers/models/albert/tokenization_albert.py#L218

Could you explain the intuition behind having this additional steps in AlbertTokenizer and what purpose do they serve here?


Thanks!

Comments:
Hello! We strive for perfect reproducibility between the original code and the HF code. This part originates from the original codebase: https://github.com/google-research/albert/blob/master/tokenization.py#L67

I would open an issue there instead. 

Thank you @LysandreJik for the pointer. I wasn't aware that HF implementation is based on google's albert repo. 

Opened a question there: https://github.com/google-research/albert/issues/249

Resolving this issue.
[FX] Fix passing None as concrete args when tracing
Author: thomasw21
Pull request: yes

# What does this PR do?

Makes sure that we never call `keys()` method on None.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

cc @michaelbenayoun @stas00 

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.


Comments:
TF Model train and eval step metrics for seq2seq models.
Author: pedro-r-marques
Pull request: yes

When using a model with a seq2seq output compute metrics against logits.

# What does this PR do?

This PR changes the TF train and test steps so that metrics can be correctly computed when using keras model.fit.

The keras Model train/test step functions are supposed to compare the labels (y_true) and the predictions (y_pred).
The previous code was passing as y_pred the ModelOutput dataclass (basically a dict) of values which results in
TF/keras attempting to compute metrics between the variable 'y' (y_true) and each of the elements in the ModelOutput dict.

## Who can review?

@sgugger
@Rocketknight1 


Comments:
This seems like a good fix, thank you! The changes to `train_step` are very recent and we were expecting some initial problems like this. I'm going to do some testing with your PR today/tomorrow and hopefully approve and merge after that. 

I did some quick testing - the impact of the change seems limited as `y_pred` is only used for computing metrics at that stage of the function, so I don't think this can have any major unwanted side-effects. Approving.

@Rocketknight1 I did some more testing. When one compiles the model multiple times the metric doesn't work correctly. If a model is compiled only once it does work. I don't know if this is a concern.

@Rocketknight1 Please take another look. The test makes sure that the metric now works as expected. I also removed a call to compute the loss on `test_step` which seems to me to be a duplicate.

@pedro-r-marques You're correct that some vestigial code made it into test_step - that's not great (at our end, you did a good job in spotting it!). Let me double-check that bit and tidy it up in your branch before we merge.

Done! The old code comes from a period when we were experimenting with a different way of handling the model's internal loss computations, and should have been removed. I fixed it now, and the rest of the code looks good. If you're happy with my changes, we can merge once the tests are good.

(That torch failure has nothing to do with this PR, don't worry)

@Rocketknight1 Checks are green now. Please merge the PR, if you are happy with it. Thanks a lot !

The rebase reverted the fix to the vestigial code, so I took it out again. Will merge once it's all green!

@Rocketknight1 apologies for squashing your changes unintentionally :-(. Cleaned up the git log; hopefully preserving your changes this time and took another roll at the CI dice.

@pedro-r-marques It's okay, I wrecked your changes too. Git is just hard, lol. Anyway, it looks good now and tests are green, so merging!
updated setup.py
Author: anukaal
Pull request: yes

checking if python is running with the older version in the system or not 


Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Adding GPT2 for translation/summarization
Author: dorost1234
Pull request: no

# 🚀 Feature request

Hi
In the code base GPT-2 is not listed as the model compatible for summarization and translation. Is there a reason for this?

## Motivation
GPT-2 is one of the most important seq2seq models and many papers build on top of it. It would be great to include this model. 

Thanks. 

Comments:
The `run_summarization.py` and `run_translation` scripts are designed for encoder-decoder (also called seq2seq) models like T5, BART, PEGASUS. 

However, GPT-2 is a Transformer decoder-only model. This model is also capable of doing summarization/translation, but it would be solved in a different way. GPT-2 is just a language model (i.e. it predicts the next token given the previous ones). So you could fine-tune GPT-2 using the [`run_clm.py`](https://github.com/huggingface/transformers/tree/master/examples/pytorch/language-modeling) script. OpenAI has done this for example in [this paper]().

What typically happens is one provides the text, appended by "TDLR: " to the model, and the model is then capable of generating a summary (just because it is trained on a large portion of the internet).



I was actually looking for this to do some Legal Text Summarization. Thanks Niels for explaining it well. Looks like I need to do it on a different model. Do you have any model to suggest that works well with legal text?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
How to use the parameters of a certain layer
Author: ghost
Pull request: no

print(bert.embeddings.LayerNorm.bias)  #Run successfully
print(bert.encoder.layer.0.attention.self.query.weight)     #invalid syntax appears in "layer.0."
so how to use the parameters of a certain layer?

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

@mengruwg 

The general way to access any parameter of a PyTorch model via its name is

`parameter = model.state_dict()[<parameter_name>].`

Since huggingface models override PyTorch nn.Module you can use the above method for any model. [Reference](https://discuss.pytorch.org/t/how-to-manipulate-layer-parameters-by-its-names/1282)


I see. Thank you very much for your prompt reply.
















At 2021-08-28 01:53:47, "Rahul Shiv Chand" ***@***.***> wrote:

@mengruwg

The general way to access any parameter of a PyTorch model via its name is

parameter = model.state_dict()[<parameter_name>].

Since huggingface models override PyTorch nn.Module you can use the above method for any model. Reference

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Settings for perfect Story writing based on the input text?
Author: krigeta
Pull request: no

Hello, thank you so much for this awesome project.

What settings do you suggest so by using those settings it seems like a story is continuing based on the previous story input as a text file?

Comments:
Hello, sorry for the long answer time! Unfortunately, we favor using the [forum](https://discuss.huggingface.co) for questions like this where you're much more likely to get an answer than on Github Issues. If that's not already the case, do you mind opening a thread there?

Thanks!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Can DeepSpeed ZeRO-3 be applied for training? 
Author: avionkmh
Pull request: no

# 🌟 New model addition

We have applied DeepSpeed v0.3.10(ZeRO-2) on T5 training. 
I heard DeepSpeed ZeRO-3 library has been released 10 days ago(8/MAR). 
I'd like to adopt ZeRO-3 for our training.
Can this library be applied, especially for T5 training? 
Do you have any experience applying this library?
If any, could share your experience?  


Comments:
Hi! You might find this reply https://github.com/huggingface/transformers/issues/10789#issuecomment-802100991 by @stas00  of interest.

@avionkmh, very soon it'll be supported, you may want to track: https://github.com/huggingface/transformers/pull/10753

@stas00, @LysandreJik, thank you for your kind information. I'm going to track the issues you recommended. 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Fix: replace assert statements with exceptions in file src/transformers/models/lxmert/modeling_lxmert.py
Author: murilo-goncalves
Pull request: yes

# What does this PR do?

Replaced assert statements with ValueError exceptions in file src/transformers/models/lxmert/modeling_lxmert.py,
as discussed in issue #12789

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sgugger


Comments:
Trainer._load_rng_state() misbehavior
Author: tlby
Pull request: no

Resuming from checkpoints have been giving me a warning I did not expect.
```
Didn't find an RNG file, if you are resuming a training that was launched in a distributed fashion, reproducibility is not guaranteed.
```
even though I do see a `rng_state.pth` file in the checkpoint directory.  A little digging turned up
https://github.com/huggingface/transformers/blob/3892d09f4f55607399ac6b9df14dbe9b3e92a3f7/src/transformers/trainer.py#L1513-L1514
which seems to prepend the checkpoint path to the filename twice for the `isfile()` check.  I think this was a typo.


Comments:
Oh indeed! Do you want to make a PR to fix it, since you found the problem?

If that PR doesn't look right, then feel free to post another and I can close mine out.  Thanks!
Replace assertions in src/transformers/data/datasets/language_modeling.py
Author: skpig
Pull request: yes

# What does this PR do?

<!--
-->
The PR replaces the assertions in src/transformers/data/datasets/language_modeling.py with ValueError exceptions. 

<!-- Remove if not applicable -->

Fixes # (#12789)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?
## Who can review?
@sgugger

Comments:
Add ASR colabs
Author: patrickvonplaten
Pull request: yes

# What does this PR do?

Adds ASR colabs to the READMEs.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Comments:
Wait until those PRs are merged:

- https://github.com/huggingface/notebooks/pull/98
- https://github.com/huggingface/notebooks/pull/97

cc @anton-l - feel free to go into the PR to add AudioClassification right away
[ASR] Make speech recognition example more general to load any tokenizer
Author: patrickvonplaten
Pull request: yes

# What does this PR do?

Many Wav2Vec2-like models don't implement their own tokenizer. For such models, we need to rely on `tokenizer_class` in the config to load the correct tokenizer.


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Comments: