noamaanMulla-03/transformers-issues huggingface dataset:
Fix hp search for non sigopt backends
Author: sgugger
Pull request: yes

# What does this PR do?

This PR fixes the bug introduced in #13572 for the hyperparameter search using Ray Tune or Optuna. More speficially the bug introduced is on [this line](https://github.com/huggingface/transformers/blob/36fc401621297d2f917a0b4df4a0718067966c12/src/transformers/trainer.py#L1241)
```py
self.state.trial_params = hp_params(trial.assignments) if trial is not None else None
```

which used to be
```py
self.state.trial_params = hp_params(trial) if trial is not None else None
```

For the HP backends different from SigOpt, the `trial` object has no `assignmnents`, hence the failure.

Fixes #13875 

Comments:
[Trainer] Fix nan-loss condition
Author: anton-l
Pull request: yes

Fixes a change in the nan loss handling logic introduced by #13896

With `is_torch_tpu_available()==False` and `(torch.isnan(tr_loss_step) or torch.isinf(tr_loss_step))==False` 
the `else:` condition was never reached, so the training loss was always `0.0`.

Comments:
`min_length` in `generate` method
Author: sooftware
Pull request: no

Hi! I have a question about `generate` method.  
Model's original eos_token is `
` but, I designated eos_token `<sep>` when I use generate method.  
If min_length is specified, I want to determine the minimum length until <sep> is generated.  
What can I do?

Comments:
Hi! Could you please post this question on the [forum](https://discuss.huggingface.co/)? We use issues for bug reports and feature requests, for such general questions use the forum. Thank you!

Ah, OK! Thank you!
Transformer Hosted API Broken
Author: farpace
Pull request: no

When you change the default example the following error shows up on Hosted inference API

![Screen Shot 2021-09-29 at 1 32 30 PM](https://user-images.githubusercontent.com/91635425/135344821-c4b7866f-0cf9-464e-8fe8-1b0b1d1506f1.png)


Comments:
cc @nreimers 

The issue is known and has to do with some cache file cleaning we use for the widgets: Files older than 30 days are deleted, but this causes some issues with the caching system of sentence-transformers.

 @osanseviero has posted a PR in sentence-transformers that will re-download files. The PR is merged in sentence-transformers and will soon pushed to pip. It will then solve this issue. 

[This](https://github.com/UKPLab/sentence-transformers/pull/1116) is the PR for reference. I'll update this issue once all changes are pushed to pip + deployed in the API.

The newest version of sentence-transformers is pushed to pip: version 2.1.0

It can now be deployed in the API.

This will be fixed with https://github.com/huggingface/huggingface_hub/pull/383 deployment

This is fixed now!
Inference of ONNX exported BART
Author: GRIGORR
Pull request: no

As I understand correctly, one should use generate function for Seq2Seq model inference. I have exported BART to ONNX, but ONNX InferenceSession().run() returns result of BART.forward(), which is different from BART.generate(). Is there any way to get the BART.generate() from ONNX? Thanks.

Comments:
The `generate` method in ONNX isn't available yet. It is on the roadmap, but it is a big item that will require quite a bit of time to solve.

You can follow the progress here: https://github.com/huggingface/transformers/pull/13578

Thanks for the response. Just a small clarification, in the pull request you mentioned is handling of beam search, if I want to use greedy decoding, I still cant have it for ONNX BART right?

Nope, for now you need to implement your own greedy search or beam search for generating.
#12789 Replace assert statements with exceptions
Author: djroxx2000
Pull request: yes

# What does this PR do?

Replaces the assertions in utils_qa.py file at examples/tensorflow/question-answering, examples/flax/question-answering and examples/pytorch/question-answering (bound by strict copy check) with ValueError and EnvironmentError exceptions as necessary. 
Also fixed a typo changing predicitions -> predictions in postprocess_qa_predictions_with_beam_search function.

Contributes towards fixing issue [#12789 ](https://github.com/huggingface/transformers/issues/12789)

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


Comments:
Can someone please take a look at my PR. The error log mentions a config.json file to be missing in a tmp directory and then goes on to throw 2 assertion errors. But the files where I have made my changes are not mentioned in the logs anywhere. I'm having trouble understanding why my changes may have caused a missing config file. Sorry about the force-push, it was required for a little cleanup.

Thanks again!
Add missing whitespace to multiline strings
Author: aphedges
Pull request: yes

# What does this PR do?

When running a text generation experiment, I got the following error message:

> Input length of input_ids is 104, but ``max_length`` is set to 100.This can lead to unexpected behavior. You should consider increasing ``config.max_length`` or ``max_length``.

The error is missing a space before `This can lead`. Checking the source, the issue turned out to be that the string was wrapped across multiple lines, but there was no whitespace within the string itself to separate the two sentences. I searched for `"\s+f?"` throughout all of `src/transformers`, and this PR contains a fix for all similar whitespace errors I could find. The good news is that most multiline strings already handled whitespace properly, but there were still a lot that didn't. My search probably missed a few, but I did not feel like being exhaustive here.

I also made a couple of other error message fixes related to whitespace and punctuation, but focused on this specific whitespace issue.

## Before submitting

- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

## Who can review?

"Documentation" is the closest item to this PR, so @sgugger

Comments:
Wrong weight initialization for TF t5 model
Author: danshirron
Pull request: no

## Environment info
      - `transformers` version: 4.9.2
      - Platform: Linux-4.15.0-142-generic-x86_64-with-glibc2.29
      - Python version: 3.8.10
      - PyTorch version (GPU?): not installed (NA)
      - Tensorflow version (GPU?): 2.5.0 (True)
      - Flax version (CPU?/GPU?/TPU?): 0.3.4 (gpu)
      - Jax version: 0.2.18
      - JaxLib version: 0.1.69
      - Using GPU in script?: Yes
      - Using distributed or parallel set-up in script?: yes

### Who can help
@patil-suraj 
@patrickvonplaten 
## Information

Model I am using: Pre-training T5-base

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [X] my own modified scripts: (give details below): Added to run_mlm.py the t5 data collator and keras adafactor optimizer

The tasks I am working on is:
* [X] my own task or dataset: Pre-training T5 base with oscar dataset (as in FLAX example)

## Expected behavior
Before updating init weights to normal distribution (as in transformers/src/transformers/models/t5/modeling_flax_t5.py) loss stuck at 4.5 (unlike FLAX behaviour). after update of init weights i get same behaviour as in FLAX and reach <2 loss.

Example:
In flax code: class: FlaxT5DenseReluDense: lines 95:,96 
    wi_init_std = self.config.initializer_factor * (self.config.d_model ** -0.5)
    wo_init_std = self.config.initializer_factor * (self.config.d_ff ** -0.5)

In TF code, the default initializer is used. My suggested fix:
        wi_initializer = tf.keras.initializers.RandomNormal(mean = 0, stddev = config.initializer_factor * (config.d_model ** -0.5))
        wo_initializer = tf.keras.initializers.RandomNormal(mean = 0, stddev = config.initializer_factor * (config.d_ff ** -0.5))
        self.wi = tf.keras.layers.Dense(config.d_ff, use_bias=False, name="wi",kernel_initializer=wi_initializer)
        self.wo = tf.keras.layers.Dense(config.d_model, use_bias=False, name="wo",kernel_initializer=wo_initializer)

This is relevant for all weights and embeddings initialization.



Comments:
I agree! Would you like to open a PR to fix it? :-)

Will try to do it on coming days

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Wav2Vec2ForCTC is not BaseModelOutput
Author: yc-li20
Pull request: no

On the website of huggingface: https://huggingface.co/transformers/model_doc/wav2vec2.html#wav2vec2forctc
it says Wav2VecForCTC is "BaseModelOutput".
But actually it is "CausalLMOutput", it has no attribute 'last_hidden_state' or the others of "BaseModelOutput".
Its returns should belong to "CausalLMOutput": https://huggingface.co/transformers/main_classes/output.html#causallmoutput

**The description of the returns of Wav2VecForCTC on the website:**
<img width="1031" alt="130951325-9fd86ab4-4b2a-4965-b4bf-88b2cc556b46" src="https://user-images.githubusercontent.com/15671418/131300701-11106f9c-ab2a-42b8-8c35-9a7418e37474.png">

**The error when call "the last hidden state" of Wav2VecForCTC:**
<img width="663" alt="130951343-eb4655a3-af57-4a2f-a387-0fa628f854dc" src="https://user-images.githubusercontent.com/15671418/131300840-107609a5-6d30-4d89-a0e1-ae003feb3934.png">

**The description of CasualLMOutput which the Wav2VecForCTC shoud be:**
<img width="1077" alt="130951334-093e1df0-6207-45f0-b803-76b276f17f7b" src="https://user-images.githubusercontent.com/15671418/131300953-a8eaf063-db8d-46f3-836e-4baf534e1554.png">

@patrickvonplaten

Comments:
Hi @patrickvonplaten , I read the source code and found the wav2vecctc only conducts word-level tokenization. Does it  support ctc fine-tuning on grapheme level or character level? Thanks.

Oh yeah that's a typo in the docs indeed, but it's already fixed on master I think :-) 
See: https://huggingface.co/transformers/master/model_doc/wav2vec2.html#wav2vec2forctc

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
How to extract the encoded data of feed & forward layer in tfbertmodel?
Author: 1148330040
Pull request: no

env：tf2.2

model：TFBertModel.from_pretrained('hfl/chinese-bert-wwm-ext')

i'm working on an information extraction project. First, I predict the “**subject**” through Bert CRF, then **tf. Gather ()** the coding layer of shared Bert and the location information corresponding to the “**subject**”, and then predict the “**object**“, but I can't extract the **feed & forward** layer of Bert now

I want to extract the output of the **feed & forward** layer of the Bert model as the shared coding layer, but I can't find the corresponding method. I want to obtain the output similar to the following:

# Tensor("Transformer-11-FeedForward-Add/add:0", shape=(None, None, 768), dtype=float32)

I tried through ”**model.trainable_weights[- 5]**”layer, but the extracted output is obviously not what I need, and I don't want to directly use "model (ids, masks, tokens) [0]", because Bert's last layer is processed by "layerNormal"


Comments:
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discuss.huggingface.co) instead?

Thanks!

cc @Rocketknight1 

I asked questions where you said, but no one replied to me. I hope you can tell me the answer to this question. Thank you！
https://discuss.huggingface.co/t/how-to-extract-the-encoded-data-of-feed-forward-layer-in-tfbertmodel/9320

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
[Wav2Vec2] Fix mask_feature_prob
Author: patrickvonplaten
Pull request: yes

# What does this PR do?

When using `mask_feature_prob > 0` it's important that the `attention_mask` is not passed as the attention_mask's dimension is `[batch_size, sequence_length]`. Since attention_mask = 1 vectors are padded anyways it doesn't matter whether some of those vectors are zero-ed out or not. Thus we should just never pass `attention_mask` for `mask_feature_prob > 0`.
We however need it for `mask_time_prob` since `mask_time_prob` is used for `Wav2Vec2ForPretraining` where the created outputs will be used as labels.

Comments:
@anton-l - feel free to merge if the change is OK for you :-)

Makes sense, thanks for adding the tests too! :+1:
Fix LED
Author: Rocketknight1
Pull request: yes

The hidden states and last states returned by the model were incorrect in both TF and PT, so this hopefully resolves everything.

In addition, this PR disables LayerDrop in the TF model, as its implementation would not work in any model compiled with Keras or `tf.function`

Resolves #13776 

Comments:
Hmm, it seems like some Torch tests are now failing because they expect the longer padded output length. Not sure whether this is a problem with my fix or with the tests

@Rocketknight1 - Thanks a lot for the fix! Some generation tests are failing...if you want I can dive into the PR and fix them - lemme know :-)
[trainer] memory metrics: add memory at the start report
Author: stas00
Pull request: yes

While debugging a user report https://github.com/huggingface/transformers/issues/13616, I have noticed when we use deepspeed w/ zero3 it allocates some memory on GPU before Trainer's init, so this memory doesn't get reported and then I got a mysterious negative delta:
```
  train_mem_gpu_alloc_delta  =     -153MB
  train_mem_gpu_peaked_delta =     2012MB
```
so it was very confusing. This PR adds another 2 metrics which report memory usage at the very start of the Trainer (init), so now the report becomes:
```
  before_init_mem_gpu        =      312MB
  train_mem_gpu_alloc_delta  =     -153MB
  train_mem_gpu_peaked_delta =     2012MB
```
and it becomes obvious that some memory was allocated before the Trainer started and now the minus can be added to a non-zero initial value.

For symmetry added the cpu memory usage as well.

Examples of the new report (deleted all, but memory metrics):
```
$ CUDA_VISIBLE_DEVICES=0 PYTHONPATH=src python examples/pytorch/language-modeling/run_clm.py \
--model_name_or_path gpt2 --do_train --train_file tests/fixtures/sample_text.txt --fp16 true \
--output_dir output --overwrite_output_dir 1 --per_device_train_batch_size 1 --report_to none \
--skip_memory_metrics 0

***** train metrics *****
  before_init_mem_cpu        =     1945MB
  before_init_mem_gpu        =        0MB
  init_mem_cpu_alloc_delta   =     3249MB
  init_mem_cpu_peaked_delta  =      443MB
  init_mem_gpu_alloc_delta   =      487MB
  init_mem_gpu_peaked_delta  =        0MB
  train_mem_cpu_alloc_delta  =      151MB
  train_mem_cpu_peaked_delta =        0MB
  train_mem_gpu_alloc_delta  =     1431MB
  train_mem_gpu_peaked_delta =      562MB

$ PYTHONPATH=src deepspeed --num_gpus 1 examples/pytorch/language-modeling/run_clm.py \
--deepspeed tests/deepspeed/ds_config_zero3.json --model_name_or_path gpt2 --do_train \
--train_file tests/fixtures/sample_text.txt --fp16 true --output_dir output \
--overwrite_output_dir 1 --per_device_train_batch_size 1 --report_to none \
--skip_memory_metrics 0

***** train metrics *****
  before_init_mem_cpu        =     5262MB
  before_init_mem_gpu        =      312MB
  init_mem_cpu_alloc_delta   =        0MB
  init_mem_cpu_peaked_delta  =        0MB
  init_mem_gpu_alloc_delta   =        0MB
  init_mem_gpu_peaked_delta  =        0MB
  train_mem_cpu_alloc_delta  =     3288MB
  train_mem_cpu_peaked_delta =      465MB
  train_mem_gpu_alloc_delta  =     -153MB
  train_mem_gpu_peaked_delta =     2012MB
```
For the non-DS case, the gpu memory usage is usually 0 and it's reflected in the report, and non-0 with DS.

Additionally I prepared several other possible reports if someone puzzles over the math or has a complex situation. Left those in the comments if someone needs them.

@sgugger 

Comments:
rgwwfq
Author: fatakesto
Pull request: no
Labels: name: New model

<html xmlns:v="urn:schemas-microsoft-com:vml"
xmlns:o="urn:schemas-microsoft-com:office:office"
xmlns:x="urn:schemas-microsoft-com:office:excel"
xmlns="http://www.w3.org/TR/REC-html40">

<head>

<meta name=ProgId content=Excel.Sheet>
<meta name=Generator content="Microsoft Excel 15">
<link id=Main-File rel=Main-File
href="file:///C:/Users/BUSHRA~1/AppData/Local/Temp/msohtmlclip1/01/clip.htm">
<link rel=File-List
href="file:///C:/Users/BUSHRA~1/AppData/Local/Temp/msohtmlclip1/01/clip_filelist.xml">
<style>
<!--table
	{mso-displayed-decimal-separator:"\.";
	mso-displayed-thousand-separator:"\,";}
@page
	{margin:.75in .7in .75in .7in;
	mso-header-margin:.3in;
	mso-footer-margin:.3in;}
.font5
	{color:#0F1419;
	font-size:15.0pt;
	font-weight:400;
	font-style:normal;
	text-decoration:none;
	font-family:"Segoe UI", sans-serif;
	mso-font-charset:0;}
tr
	{mso-height-source:auto;}
col
	{mso-width-source:auto;}
br
	{mso-data-placement:same-cell;}
td
	{padding-top:1px;
	padding-right:1px;
	padding-left:1px;
	mso-ignore:padding;
	color:black;
	font-size:11.0pt;
	font-weight:400;
	font-style:normal;
	text-decoration:none;
	font-family:Calibri, sans-serif;
	mso-font-charset:0;
	mso-number-format:General;
	text-align:general;
	vertical-align:bottom;
	border:none;
	mso-background-source:auto;
	mso-pattern:auto;
	mso-protection:locked visible;
	white-space:nowrap;
	mso-rotate:0;}
.xl65
	{border:.5pt solid windowtext;}
.xl66
	{text-align:left;
	vertical-align:middle;
	border:.5pt solid windowtext;}
.xl67
	{color:#1D9BF0;
	font-size:15.0pt;
	font-family:"Segoe UI", sans-serif;
	mso-font-charset:0;}
-->
</style>
</head>

<body link="#0563C1" vlink="#954F72">



https://twitter.com/i/events/1445809158612267009
--
https://twitter.com/i/events/1445810128742014983
https://twitter.com/i/events/1445810868453732354
https://twitter.com/i/events/1445811643233890307
https://twitter.com/i/events/1445813748271247363
https://twitter.com/i/events/1445814196017537027
https://twitter.com/i/events/1445815001994854418
https://twitter.com/i/events/1445816928778457091
https://twitter.com/i/events/1445851201002631176
https://twitter.com/i/events/1445851897542295552
https://twitter.com/i/events/1445852186282446852
https://twitter.com/i/events/1445852523601043462
https://twitter.com/i/events/1445852932113674240
https://twitter.com/i/events/1445853683195973633
https://twitter.com/i/events/1445854031482474498
https://twitter.com/i/events/1445854458487869443
https://twitter.com/i/events/1445854794799669248
https://twitter.com/i/events/1445855323672219650
https://twitter.com/i/events/1445855701239275525
https://twitter.com/i/events/1445856296096391168
https://twitter.com/i/events/1445856911652364288
https://twitter.com/i/events/1446068878837231621
https://twitter.com/i/events/1446069626182504449
https://twitter.com/i/events/1446071685527052294



</body>

</html>


Comments:
Fixed typo: herBERT -> HerBERT
Author: adamjankaczmarek
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes typo: `herBERT` -> `HerBERT`


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sgugger, @Qbiwan, @forest1988, @rmroczkowski 




Comments:
New transformers.onnx CLI does not support ONNX quantization
Author: minimaxir
Pull request: no

# 🚀 Feature request

New transformers.onnx CLI introduced in 4.9.0 does not support ONNX quantization, which is a notable missing feature from the `convert_graph_to_onnx.py` script.

Semi-related: the [source quantize() function](https://github.com/microsoft/onnxruntime/blob/79097ef5535cc5ac18fc8e9010c99de08df21340/onnxruntime/python/tools/quantization/quantize.py#L56) that script leverages is depreciated so it might be a good time to switch to `quantize_dynamic()` too.


Comments:
Hi @minimaxir, 

Thanks for reporting this.

With the new "configuration based" capabilities we are taking a very different approach from the initial `convert_graph_to_onnx.py` which was relying on heuristics to match dynamic axes and was exporting them in the wrong order quite often.

The new approach focus on a more reliable approach and on exporting only to "raw" ONNX graphs which can then be consumed by different "runtimes" not only onnxruntime. Thus we are not exposing anymore optimizations/quantizations features as part of transformers.

Still, we are currently working on another project will provide such features, leveraging the new configuration based export. It should be available in August and ONNX Runtime will be one of the first component we will provide optimizations for. 

Stay tuned 🤗 

SGTM. (it's not a dealbreaker as Microsoft's approach is to create a raw ONNX and quantize it too).

Excited to see future ONNX support!

@mfuntowicz Excidetd to see onnxruntime supported BART/MBART models

Hi @mfuntowicz, any updates on the new project? Thanks.