noamaanMulla-03/transformers-issues huggingface dataset:
Support custom StoppingCriteria in model.generate
Author: Linyxus
Pull request: no

# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->

It could be helpful to support custom StoppingCriteria in `model.generate`.

Currently, `model.generate` implemented in generation_utils.py ([permlink](https://github.com/huggingface/transformers/blob/68d925195ede14826dd0e83258b64c2222133988/src/transformers/generation_utils.py#L747)) does not support supplying a custom StoppingCriteria in function arguments.

We could add a `stopping_criteria` argument in the function, and use the supplied criteira to control the generation progress.

## Motivation

<!-- Please outline the motivation for the proposal. Is your feature request
     related to a problem? e.g., I'm always frustrated when [...]. If this is related
     to another GitHub issue, please link here too. -->

I am a NLP researcher and recently I have been using GPT-J provided by transformers to follow the Codex research conducted by OpenAI (Codex [paper](https://arxiv.org/abs/2107.03374)), where I found it useful to supply a custom StoppingCriteria when generating source code (stop when encountering a new function/class definition).

## Your contribution

<!-- Is there any way that you could help, e.g. by submitting a PR?
     Make sure to read the CONTRIBUTING.MD readme:
     https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md -->

This may be as simple as appending the user-supplied criteria [here](https://github.com/huggingface/transformers/blob/68d925195ede14826dd0e83258b64c2222133988/src/transformers/generation_utils.py#L1109).


Comments:
Hi @Linyxus , passing custom criteria is now supported, see #14779

Thank you! :D
doc-builder -> doc-build
Author: LysandreJik
Pull request: yes

Changes the repository on which the documentation builds will be pushed.

Comments:
Just want to make sure [this feature](https://huggingface.slack.com/archives/C02GLJ5S0E9/p1639171159299300) is unrelated to this PR

> Just want to make sure this feature is unrelated to this PR

I have not touched the notebook part of the PR, the notebooks will still get automatically updated AFAICT
[Fix doc example] - ProphetNetDecoder
Author: ydshieh
Pull request: yes

# What does this PR do?

The doc example in `ProphetNetDecoder` will fail

https://github.com/huggingface/transformers/blob/68cc4ccde25b1ed8a9b77fdf6f78c833bdff0e9c/src/transformers/models/prophetnet/modeling_prophetnet.py#L1473-L1474


ProphetNetDecoder never set `is_decoder = True`. It is set only at the level at `ProphetNetModel` and `ProphetNetForCausalLM`.

This PR removes that assertion.





Comments:
[examples/flax/language-modeling] set loglevel
Author: stas00
Pull request: yes

This PR sets the unset loglevel to a `info` like the rest of flax examples where it wasn't done so.

Fixes a report in https://github.com/huggingface/transformers/pull/14909#issuecomment-1011467609

@patil-suraj, @sgugger 


Comments:
Thanks so much @stas00 , problem is solved now! Level is not correctly `INFO` during training!

Thank you for confirming that, @stefan-it.

Not the main person on the Flax examples so will let @patil-suraj or @patrickvonplaten chime in on the PR :-)
Dynamic Inputs for fx traced GPTNeoLM 
Author: sdtblck
Pull request: no

# 🚀 Feature request

I've been looking a little into using torch.fx + HF lately, and run into a problem that the traced GPTNeoLM can't take dynamic inputs - would love to know what work would be needed to enable that, and what the current blockers are - it would be great to be able to use the traced module for inference.

I'm assuming the problem has something to do with caching of keys/values?

## Your contribution

Given guidance, I could put some work into adding this feature, yes :) 

Comments:
cc @michaelbenayoun 

Hi @sdtblck ,
The issue with dynamic inputs is not related to the `past_key_values` but to dynamic control-flow in the model forward pass implementation. The PR #14321, that I plan to merge at some point (hopefully before the end of the year) solves this kind of issues for the supported models.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
bert-base-uncased weights for BertForPreTraining
Author: eldarkurtic
Pull request: no

Hi,

According to the model card: https://huggingface.co/bert-base-uncased , these weights are obtained by training on two objectives (MLM and NSP) but the `config.json` has only the `BertForMaskedLM` tag. So when initializing `BertForPreTraining` model with `bert-base-uncased` weights, the following warning is printed:
`Some weights of BertForPreTraining were not initialized from the model checkpoint at bert-base-uncased and are newly initialized: ['cls.predictions.decoder.bias']`.

(My use-case would be to load `bert-base-uncased` weights and run the same pretraining process as the one described in the model card: https://huggingface.co/bert-base-uncased)

@LysandreJik 

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
mBART support for run_summarization.py
Author: banda-larga
Pull request: yes

Added support for multilingual tokenizer and mBART for run_summarization.py

-> run_summarization.py not working with mBART

Comments:
So, it appears that this PR introduced a new issue by forcing all models to include `--lang`, @banda-larga would you like to fix it in a new PR and assert on lack of `--lang` only with mbart model type? see: https://github.com/huggingface/transformers/pull/15150#issuecomment-1012553262

and while at it to integrate a better error message as proposed here https://github.com/huggingface/transformers/pull/15150

will then also need to revert this https://github.com/huggingface/transformers/pull/15149 as part of the new PR.

and note to self: make sure 
```
RUN_SLOW=1 pytest tests/deepspeed/test_model_zoo.py::TestDeepSpeedModelZoo::test_zero_to_fp32_zero2_sum_pegasus
```
doesn't fail as this PR broke it. please tag me to the new PR and I will do the checking - you don't need to figure this part out.

Bonus points: adding a new examples test that should have failed with this PR - are we not testing `run_summarization.py` in torch_examples CI?

Thanks.

This is a tiny bit urgent, so not waiting to remove the two lines that break every existing command using `run_summarization`. I fixed that in [this commit](https://github.com/huggingface/transformers/commit/96881729ce83cfc8e5fa04c903ee4296ad17cfbb).

ok, so nothing else needs to be done. @banda-larga please ignore my comments above.
KeyError: 'gpt_neo' with EleutherAI/gpt-neo-1.3B
Author: fatihbeyhan
Pull request: no

I am trying to try new GPT-3 checkpoint however I am getting an error on local and Google Colab platform as well.

Google Colab:
```
!pip install transformers

from transformers import AutoTokenizer, AutoModelForCausalLM
  
tokenizer = AutoTokenizer.from_pretrained("EleutherAI/gpt-neo-1.3B")
model = AutoModelForCausalLM.from_pretrained("EleutherAI/gpt-neo-1.3B")
```

The error I get is:

```
KeyError                                  Traceback (most recent call last)

<ipython-input-5-333740565b3a> in <module>()
      3 from transformers import AutoTokenizer, AutoModelForCausalLM
      4 
----> 5 tokenizer = AutoTokenizer.from_pretrained("EleutherAI/gpt-neo-1.3B")
      6 model = AutoModelForCausalLM.from_pretrained("EleutherAI/gpt-neo-1.3B")

1 frames

/usr/local/lib/python3.7/dist-packages/transformers/models/auto/configuration_auto.py in from_pretrained(cls, pretrained_model_name_or_path, **kwargs)
    387         config_dict, _ = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
    388         if "model_type" in config_dict:
--> 389             config_class = CONFIG_MAPPING[config_dict["model_type"]]
    390             return config_class.from_dict(config_dict, **kwargs)
    391         else:

KeyError: 'gpt_neo'
```

Comments:
Hello! GPT Neo is available on the master branch, while you're installing the version v4.2.2.

You should change `pip install transformers` to `pip install git+https://github.com/huggingface/transformers` and reload your kernel

Doing so results in this error: 

```
>>> generator = pipeline('text-generation', model='EleutherAI/gpt-neo-2.7B')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\jerkm\AppData\Local\Programs\Python\Python38\lib\site-packages\transformers\pipelines\__init__.py", line 540, in pipeline
    framework, model = infer_framework_load_model(
  File "C:\Users\jerkm\AppData\Local\Programs\Python\Python38\lib\site-packages\transformers\pipelines\base.py", line 235, in infer_framework_load_model
    raise ValueError(f"Could not load model {model} with any of the following classes: {class_tuple}.")
ValueError: Could not load model EleutherAI/gpt-neo-2.7B with any of the following classes: (<class 'transformers.models.auto.modeling_tf_auto.TFAutoModelForCausalLM'>,).
```

I think this was resolved when I installed pytorch
Make sure all submodules are properly registered
Author: sgugger
Pull request: yes

# What does this PR do?

We have seen some mysterious bugs in Python 3.8 linked to the lazy init in Transformers, where some of the submodules are not properly registered as such because they don't appear in the `_import_structure` dict. This PR fixes that and adds a check to make sure newly added modules are always added to the init.

Comments:
[Fix doc example] - OpenAIGPTDoubleHeadsModel
Author: ydshieh
Pull request: yes

# What does this PR do?

This line fails

https://github.com/huggingface/transformers/blob/7b83feb50a8965e9d8f13b6c4042239710b97c76/src/transformers/models/openai/modeling_openai.py#L690

Should change from `lm_logits` to `logits` (See `OpenAIGPTDoubleHeadsModelOutput`)

## Who can review

@patrickvonplaten, @LysandreJik

Comments:
Fix QA trainer to properly log eval metrics
Author: nannullna
Pull request: yes

# What does this PR do?

This PR updates the custom `QuestionAnsweringTrainer` in the examples. Previously, the trainer used `self.log()`, which does not add a prefix `"eval"`. This was a problem especially when reporting metrics to WanDB. Without a prefix, WanDB assumes the metrics come from training data, not evaluation data.

This problem can be solved by simply changing `self.log(metrics)` to `self.log_metrics("eval", metrics)` in both `evaluate()` methods. 

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

Comments:
The prefix is added in the lines above.

Oh, I guess I used the older version of the QA trainer... 
I feel sorry for bothering you and not fully checking the code. Actually, it was my first public PR...
Anyway, I really appreciate your reply and your efforts to create and maintain this library.


No worries! Let us know if the bug persists with the last version.
Transformer-XL -100 label padding
Author: robflynnyh
Pull request: no

Transformers version: 4.12.5
model: Transformer-XL / transfo_xl
url: https://github.com/huggingface/transformers/blob/master/src/transformers/models/transfo_xl/modeling_transfo_xl.py#L1011
issue:
In the docstring it is specified that: All labels set to ``-100`` are ignored (masked). 
However when passed to the model the error: index -100 is out of bounds for dimension 1 with size {vocab size} is thrown
I assume this is because of the different softmax function used by this model


Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
TF Bert inference - support `np.ndarray` optional arguments
Author: gante
Pull request: yes

# What does this PR do?

This PR allows running inference with TF Bert when the optional arguments (such as `attention_mask`) are a `np.ndarray`. Although these changes were made in response to particular issues, which were related to TF Bert, other models have the same pattern (e.g. [here](https://github.com/huggingface/transformers/blob/master/src/transformers/models/gpt2/modeling_tf_gpt2.py#L413)), so these changes may positively impact other models too.

EDIT: see root cause [here](https://github.com/huggingface/transformers/pull/15074#issuecomment-1009133852)

Fixes #14346 and #14404 



Comments:
Thanks for your changes, @gante! I think here we should add a TensorFlow common test to ensure that all models pass it, rather than adding a test for BERT only

Any test defined in this class will be run by all models:
https://github.com/huggingface/transformers/blob/9fbf7c87c37e32152428f91038bd46e04a48868e/tests/test_modeling_tf_common.py#L103

Agree with @LysandreJik - good PR that fixes an important issue, but a common TF test is better than a BERT-specific test!

That was a great suggestion, as it led me to the root cause of the issue! We already had a general test for numpy inputs (as you can see in the new diff), which was passing, but it failed if we passed the same inputs as keyword arguments. 

After some digging, I found out it was because Keras converts whatever is in the first argument of a layer to a `tf.Tensor` ([here](https://github.com/keras-team/keras/blob/master/keras/engine/base_layer.py#L1041)), so the packed numpy input in the test was converted to a tensor -- in other words, for Bert, `attention_mask` was a tensor in this setting. With an unpacked input, only the first input was converted, and thus the error in the issues surfaced.

The updated PR tests the issue for all TF models (we might discover new issues 😬 ) and fixes the error related to `shape_list`, but we might want to reconsider how we handle inputs to HF Keras models in the future :) 

This seems to be another situation where there was a difference depending on whether the input was packed in the first argument or not (see [this issue](https://github.com/huggingface/huggingface_hub/issues/582#issuecomment-1008960808))

@Rocketknight1 / @LysandreJik -- can I get a review (or better yet, an approval)? :)

I just reviewed this - it looks great to me. Catching the issue with the conversion of the first argument was well done too.
Better dummies
Author: sgugger
Pull request: yes

# What does this PR do?

This PR reworks how the dummy objects work by using a metaclass to build them, and make sure that any classmethod used will raise the appropriate error message.

Fixes the bad error message when trying to do something like:
```python
FlaxVisionEncoderDecoderModel.from_encoder_decoder_pretrained("vit", "gpt2")
```
with Flax not installed.

Comments:
Yes, it will send the imprt error for any attribute, but at the same time the object should just not be used if the right frameworks are not installed.

Failure is non-related so merging.
Pretrain BERT with user defined vocabulary
Author: Jess0-0
Pull request: no
Labels: name: wontfix

# 🚀 Feature request

I'm wondering if there is a way to pretrain BERT with user-defined vocabulary, number of layers/heads, and use a WordLevel Tokenizer instead of a WordPiece Tokenizer current BERT uses. Thank you!

## Motivation

This could help the BERT mode to adapt to different tasks.



Comments:
Hi! Yes, you can definitely do so (not sure about the word level). 

You can define a custom model configuration:
```py
from transformers import BertConfig

config = BertConfig(vocab_size=xxx, num_attention_heads=xxx, ...)
```

You can then use that configuration with the `run_mlm.py` script to train your model.

Regarding the tokenizer, you could use a word-level, but you would need to manually modify the files for that. There exists a word-level tokenizer in the `tokenization_bert.py` file that you could leverage for this.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


> tokenization_bert.py

excuse me does this class have the functions that help in word-level, not sub-word level , please ?
Multiple answers for QA 
Author: anil8593
Pull request: no

Hi Team,

I would like to know if the current implementation for Question answering tasks on the squad data format considers multiple answers that can be passed in the format. Going through the code, i think it does not however wanted to know it from you.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Multilabel Token Classification in trainer
Author: hasansalimkanmaz
Pull request: no

# 🚀 Feature request

We need to be able to use the trainer for multilabel token classification problems.

## Motivation

Right now we create a custom model and a custom trainer class. The model has an additional layer for the second set of labels. Which makes the codebase complicated and harder to maintain. This feature may help to identify entity groups or roles as it is implemented in [RASA](https://rasa.com/blog/introducing-entity-roles-and-groups/)

## Your contribution

I can try to help if you direct me to the solution.


Comments:
I am unsure why the model would need an additional layer. Can't it all be treated by a proper loss function?

Thanks for the suggestion. After brainstorming with my team, it seems to be possible. I am closing this thread.
add embed_scale for bert
Author: xu-song
Pull request: yes

# What does this PR do?

`embed_scale`  has been widely used in many variation of bert and transformer.

This commit add `embed_scale` as an option feature for bert.

Existing implementions
- [fairseq roberta](https://github.com/pytorch/fairseq/tree/main/fairseq/models/roberta), 
- https://github.com/huggingface/transformers/search?q=embed_scale

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

- albert, bert, xlm: @LysandreJik




Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

1

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Bug in seq2seq fine tuning section of Automatic Speech Recognition Examples
Author: sekharvth
Pull request: no

## Environment info
- `transformers` version: 4.16.0.dev0
- Platform: Linux-5.4.144+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.12
- PyTorch version (GPU?): 1.10.0+cu111 (True)
- Tensorflow version (GPU?): 2.7.0 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help
@patrickvonplaten , @anton-l 

## Information

Model I am using (Bert, XLNet ...): Wave2vec2-2-BART fine tuning

The problem arises when using:
* [ ] the official example scripts: 

In the code snippet from this section : https://github.com/huggingface/transformers/tree/master/examples/pytorch/speech-recognition#sequence-to-sequence

In this line :
```model = SpeechEncoderDecoderModel.from_encoder_decoder_pretrained(encoder_id, decoder_id, encoder_add_adapter=True, encoder_feat_proj_dropout=0.0, encoder_layerdrop=0.0, max_length=200, num_beams=5)```

the arguments ```encoder_add_adapter=True, encoder_feat_proj_dropout=0.0, encoder_layerdrop=0.0``` throws an error with the traceback:

```TypeError: __init__() got an unexpected keyword argument 'add_adapter'```

When I remove those 3 arguments, it works and trains properly. 

Per the Readme, these arguments are added because`**by default a single output vector of wav2vec2-base has a receptive field of ca. 25ms (cf. with section 4.2 of the official Wav2Vec2 paper), which represents a little less a single character. BART on the other hand makes use of a sentence-piece tokenizer as an input processor so that a single hidden vector of bart-base represents ca. 4 characters. To better align the output of Wav2Vec2 and BART's hidden vectors for the cross-attention mechanism, we further subsample Wav2Vec2's output by a factor of 8 by adding a convolution-based adapter.**' 

It seems the accuracy would take a hit on removing those arguments. However, adding them raises exceptions. 

Could you please look into this? @patrickvonplaten 


Comments:
Thanks for reporting this. It should be fixed soon: https://github.com/huggingface/transformers/pull/15056 . I'll make sure to speed-up the process now - sorry for being so late on this.

No worries! Thanks for the prompt resolution