noamaanMulla-03/transformers-issues huggingface dataset:
How to generate sentences in batches, instead of generating sentences one by one
Author: SuHe36
Pull request: no
Labels: name: wontfix

After I finetune GPT-2, I want to use it to generate sentences in batches instead of one by one.

So I tried to modify the code of `examples/text-generation/run_generation.py`.


the code on line 239 in `run_generation.py` is:
`encoded_prompt = tokenizer.encode(prompt_text, add_special_tokens=False, return_tensors="pt")`
the `prompt_text` is a `str` type data, but when I modify it to `List[str]` type data, It always return `50256`.

But looking at the source code, the type of `prompt_text` can be `str` , `List[str]` or  `List[int]`.

I tested this example separately , and for `token_ids` it always returns 50256.

![image](https://user-images.githubusercontent.com/22442305/91297615-344f1300-e7d1-11ea-99d4-b0e28cd1ab14.png)

So, the `prompt_text` must be `str` type data?

What modifications should I make to generate sentences in batches using `examples/text-generation/run_generation.py`?

Looking forward to your reply!


Comments:
Hey @SuHe36,

I'm currently working on adding support for batched generation. At the moment, this is the best answer we can give you: https://github.com/huggingface/transformers/issues/3021#issuecomment-591236688

Thanks for your reply !

Hey, @patrickvonplaten is batch generation available for T5conditiongeneration?

Yes! Please take a look at this test, which does batch=4 generation for summarization using T5: https://github.com/huggingface/transformers/blob/55cb2ee62eb482787cff17585955f7193fe35dfa/tests/test_modeling_t5.py#L559

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Clear up confusing translation pipeline task naming
Author: klasocki
Pull request: no
Labels: name: wontfix

# 🚀 Feature request
Hello! 
I am using the translation pipeline, and I noticed that even though I have to specify the language when I create the pipeline, the passed model overwrites that. So pipeline created as
`nlp = pipeline('translation_en_to_de', 'Helsinki-NLP/opus-mt-en-jap')`
would translate english to japanese, in contrary to the task name. Is this the intended way of translating other languages, will it change in the future?

Would it be possible to just add a single 'translation' task for pipelines, which would then resolve the languages based on the model (which it seems to do anyway now) ?

## Motivation
It would clear up the current confusion, and make the `pipeline` function singature less prone to change.
It could also possibly reduce code duplication in https://github.com/huggingface/transformers/blob/master/src/transformers/pipelines.py
<!-- Please outline the motivation for the proposal. Is your feature request
     related to a problem? e.g., I'm always frustrated when [...]. If this is related
     to another GitHub issue, please link here too. -->

## My contribution
I'd love to help with a PR, though I'm confused: The `SUPPORTED_TASKS` dictionary in pipelines.py contains exactly the same entries for each translation pipeline, even the default model is the same, yet the specific pipelines actually translate to different languages 🤔 

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
2 slow TF T5 common tests failing on master
Author: sshleifer
Pull request: no
Labels: name: wontfix

name: TensorFlow

These should probably be run on CI somewhere.
Didn't know whether to assign @patrickvonplaten or @jplu.

These fail in both tf 2.2 and tf 2.3.


#### Command
```bash
RUN_SLOW=1 pytest tests/test_modeling_tf_t5.py -k saved
```
#### Traceback
```
================================================================================= FAILURES =================================================================================
__________________________________________________________ TFT5ModelTest.test_saved_model_with_attentions_output ___________________________________________________________
tests/test_modeling_tf_common.py:223: in test_saved_model_with_attentions_output
    self.assertEqual(len(outputs), num_out)
E   AssertionError: 5 != 4
_________________________________________________________ TFT5ModelTest.test_saved_model_with_hidden_states_output _________________________________________________________
tests/test_modeling_tf_common.py:185: in test_saved_model_with_hidden_states_output
    self.assertEqual(len(outputs), num_out)
E   AssertionError: 5 != 4
---------------------------------
```

Comments:
Yeah, that's a know failure and I didn't manage to make it work yet with the `cast_bool_to_primite(...)` function

This should be fixed in the next big TF rework.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Allow one to return encoder attentions in seq2seq generation
Author: aced125
Pull request: no
Labels: name: wontfix

# 🚀 Feature request

Please could we have the ability to return attention weights from the decoded generated tokens to the encoded source?

## Motivation

To attribute the decoded text. E.g in the summarization task we want to see where the decoder was paying attention to in the source.

## Your contribution

May be able to look into a PR but stretched for time at the minute.

FairSeq has implemented this capability I believe.

Comments:
Hi @aced125, I agree that this functionality should be provided :-) 

I think in the PR, one has to include a `output_attention` argument to the `generate()` function and then make sure that the output idx are correct! 
Before starting this PR, this PR should probably be solved before: https://github.com/huggingface/transformers/issues/3880

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


want to take a look at this soon

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


#6735 is a first step to allow for this feature

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
run_language_modeling.py TPU issue during evaluation
Author: Shiro-LK
Pull request: no
Labels: name: wontfix

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 3.3.1
- Platform: ubuntu
- Python version: 3.7
- PyTorch version (GPU?): xla-nightly
- Tensorflow version (GPU?):
- Using GPU in script?: no TPU
- Using distributed or parallel set-up in script?: yes

### Who can help
@sgugger @LysandreJik 
 


## Information

Model I am using (Bert, XLNet ...):

The problem arises when using:
[*] the official example scripts: (give details below)
run_language_modeling.py
The tasks I am working on is:
[*] my own task or dataset: (give details below)
txt file for pretraining Roberta model
## To reproduce
I am trying to launch a pretraining using run_language_modeling.py with TPU unfortunately I got some issue during the evaluation and logging_steps with error message  


1. `python run_language_modeling.py --model_name_or_path="roberta-base"  --model_type="roberta" --tokenizer_name="roberta-base" --do_train --evaluate_during_training --mlm --mlm_probability=0.15 --train_data_file="train.txt" --eval_data_file="val.txt" --do_eval --per_device_train_batch=8 --per_device_eval_batch=8 --output_dir="robertaweet" --max_steps=5000000 --logging_dir="log_bertweet" --logging_steps=20 --eval_steps=10 --save_steps=25 --dataloader_num_workers=0 --tpu_num_cores=8  --learning_rate=1e-4`
2.
3.

`020-10-05 15:36:48.946219: W   11372 tensorflow/core/distributed_runtime/rpc/grpc_remote_master.cc:160] RPC failed with status = "Unavailable: Socket closed" and grpc_error_string = "{"created":"@1601912208.946077389","description":"Error received from peer ipv4:10.255.226.90:8470","file":"external/com_github_grpc_grpc/src/core/lib/surface/call.cc","file_line":1056,"grpc_message":"Socket closed","grpc_status":14}", maybe retrying the RPC`
## Expected behavior

training without any error message


Comments:
This seems like more of a TPU issue than a `huggingface/transformers` issue. Do you mind copying the full output of your command? Maybe in a `pastebin` or a github gist if it doesn't fit here.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
TFEncoderDecoder
Author: discoveredcheck
Pull request: no
Labels: name: wontfix

# 🚀 Feature request

Tensorflow version for the `EncoderDecoder` class for sequence to sequence text generation.

## Motivation
I am replicating [this](https://arxiv.org/pdf/1907.12461.pdf) paper which studies several combinations of encoder and decoder with pretrained checkpoints. Most of it is implemented very nicely in the current API, but is not available for tensorflow. The closest thing I found was `TFT5ForConditionalGeneration` which works nicely. Are there any plans to extend this for other pretrained models such as Bert, Roberta, GPT(2)?

## Your contribution
Happy to work on a PR if someone can provide some pointers on where to start and best practices to include new models in the existing API.


Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
NER pipeline documentation example failing
Author: r-maheshh
Pull request: no
Labels: name: wontfix

Hello,

I am running the code through your documentation for named entity recognition and am trying to save this "ner" model locally:

https://huggingface.co/transformers/usage.html#named-entity-recognition

```
nlp = pipeline("ner")

sequence = "Hugging Face Inc. is a company based in New York City. Its headquarters are in DUMBO, therefore very" \
           "close to the Manhattan Bridge which is visible from the window."

nlp.save_pretrained("path to folder")
```


When going to load this model up and make predictions, I am getting the error: "IndexError: list index out of range" pointing the very last line below:

```
model = AutoModelForTokenClassification.from_pretrained("path to folder")
tokenizer = AutoTokenizer.from_pretrained("path to folder")

label_list = [
    "O",       # Outside of a named entity
    "B-MISC",  # Beginning of a miscellaneous entity right after another miscellaneous entity
    "I-MISC",  # Miscellaneous entity
    "B-PER",   # Beginning of a person's name right after another person's name
    "I-PER",   # Person's name
    "B-ORG",   # Beginning of an organisation right after another organisation
    "I-ORG",   # Organisation
    "B-LOC",   # Beginning of a location right after another location
    "I-LOC"    # Location
]

sequence = "Hugging Face Inc. is a company based in New York City. Its headquarters are in DUMBO, therefore very" \
           "close to the Manhattan Bridge."

# Bit of a hack to get the tokens with the special tokens
tokens = tokenizer.tokenize(tokenizer.decode(tokenizer.encode(sequence)))
inputs = tokenizer.encode(sequence, return_tensors="pt")

outputs = model(inputs)[0]
predictions = torch.argmax(outputs, dim=2)

print([(token, label_list[prediction]) for token, prediction in zip(tokens, predictions[0].tolist())])
```

I would like to get the entity for each token. I believe that the error is with "label_list" portion of the code, and I ran the following this has the token along with the prediction represented as integers:

`print([(token,prediction) for token, prediction in zip(tokens, predictions[0].tolist())])`

I am unable to recreate the output shown on the website due to that error. Any help would be much appreciated.

Comments:
How many labels does your model have? You can see with `print(model.config.num_labels)`. If it's larger than the length of your `label_list`, that could result in an index out of range error.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
TF Slow test CI
Author: sshleifer
Pull request: no
Labels: name: wontfix

name: Tests

I Don't think tf slow tests are run by circleci OR github actions.
Should they be @LysandreJik ?

Comments:
You're correct, it is not currently running as there were some issues setting up both the PT/TF test suites. Will look into it this afternoon.

The slow tests in TF take an absurdly long time. I had to stop them from running after ~3.5 hours as it was holding the whole test suite back. Will investigate more on a separate machine and try to skin it down.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
GPT2DoubleHeadsModel documentation example question (error in documentation)?
Author: h56cho
Pull request: no
Labels: name: wontfix

Hello,
I was reading the documentation for the GPT2DoubleHeadsModel, and I have a question.
In the documentation, the example shown is:
```python
import torch
from transformers import GPT2Tokenizer, GPT2DoubleHeadsModel
tokenizer = GPT2Tokenizer.from_pretrained('gpt2')
model = GPT2DoubleHeadsModel.from_pretrained('gpt2, return_dict=True)

# Add a [CLS] to the vocabulary (we should train it also!)
num_added_tokens = tokenizer.add_special_tokens({'cls_token': '[CLS]'})
embedding_layer = model.resize_token_embeddings(len(tokenizer))  # Update the model embeddings with the new vocabulary size
choices = ["Hello, my dog is cute [CLS]", "Hello, my cat is cute [CLS]"]
encoded_choices = [tokenizer.encode(s) for s in choices]
cls_token_location = [tokens.index(tokenizer.cls_token_id) for tokens in encoded_choices]
input_ids = torch.tensor(encoded_choices).unsqueeze(0)  # Batch size: 1, number of choices: 2
mc_token_ids = torch.tensor([cls_token_location])  # Batch size: 1
outputs = model(input_ids, mc_token_ids=mc_token_ids)
lm_logits = outputs.lm_logits
mc_logits = outputs.mc_logits
```
Now, I don't see any main question statement in this example, although I see two multiple-choice options. The way I used to use `GPT2DoubleHeadsModel` is I first did `tokenizer(question_statement, option_statement)` and do `encoded_dict['input_ids']` to extract `input_ids` and similarly `encoded_dict['token_type_ids']` to extract the `token_type_ids`. Has this changed? I am getting an impression that the example is wrong (maybe the example could apply for BERT, but not GPT2DoubleHeadsModel). Is this an error in the documentation? I thought that, since GPT-2 does causal language modeling, question statement and option statement has to be encoded together, and put the `[CLS]` token at the end (usually), so that GPT-2 can apply the causal language modeling process to solve the multiple-choice problem.

Reading this again, I think the examples for BERTForMultipleChoice and GPT2DoubleHeadsModel are flipped.

 Thanks,

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
SQuAD example docs inaccurately suggest settings for bert-large-uncased on a single V100
Author: nelson-liu
Pull request: no
Labels: name: wontfix

Hi!

In the SQuAD example docs, it's noted that ( https://github.com/huggingface/transformers/blob/3f42eb979f7bd20448ff6b15ab316d63f5489a6f/docs/source/examples.md#fine-tuning-bert-on-squad10 ) `This example code fine-tunes BERT on the SQuAD1.0 dataset. It runs in 24 min (with BERT-base) or 68 min (with BERT-large) on a single tesla V100 16GB.`

Could just be me, but I don't think the example as provided works with `s/bert-base-uncased/bert-large-uncased/`---even with FP16, you run into a GPU OOM.

It might be useful to revisit this recommendation and/or remove it.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Recording training loss and perplexity during training
Author: jasonyliang
Pull request: no
Labels: name: wontfix

I'm fine-tuning GPT-2 text generation with the following command on Colab:
python run_language_modeling.py \
    --output_dir=$OUTPUT_DIR \
    --model_type=gpt2 \
    --model_name_or_path=$MODEL_NAME \
    --do_train \
    --train_data_file=$TRAIN_FILE \
    --do_eval \
    --eval_data_file=$TEST_FILE \
    --per_gpu_train_batch_size=1 \
    --save_steps=-1 \
    --num_train_epochs=5

I was wondering if I can record the training loss, perplexity ... etc per epoch into a csv file or saved as a variable on Colab?
Thank you so much!

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
[NEW MODEL] Multilingual document embeddings: cT-LASER
Author: hypnopump
Pull request: no
Labels: name: wontfix

name: New model

# 🌟 New model addition

## Model description

Multilingual document embeddings by adapting the LASER architecture (which is based on BiLSTM) to transformer architectures. 

## Open source status

* [x] the model implementation is available: open source implementation is here: https://github.com/ever4244/tfm_laser_0520
* [ ] the model weights are available: i haven't found them
* [x] who are the authors: @ever4244


Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
can u help me out with how to input custom data files in RAG retriever and the data format
Author: Madhuri05thorat
Pull request: no
Labels: name: wontfix

sign me up, Sam

_Originally posted by @stas00 in https://github.com/huggingface/transformers/issues/7715#issuecomment-706769068_

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Batch and smart batch support for pipelines.
Author: PhilipMay
Pull request: no
Labels: name: wontfix

# 🚀 Feature request

## Motivation
I want to use `TextClassificationPipeline` to classify a large number of texts.
The naive approach is:

```python
model = AutoModelForSequenceClassification.from_pretrained(model_dir)
tokenizer = AutoTokenizer.from_pretrained(model_dir)
pipeline = TextClassificationPipeline(
    model=model,
    tokenizer=tokenizer,
    framework="pt",
    device=0,
)
results = pipeline(unlabeled_text_list)
```

But this gives me CUDA OOM error when `unlabeled_text_list` is long.

What about adding batch support that lets you specify the batch size and maybe also support for multiprocessing tokenization?

When possible smart batching would be nice. See this: https://github.com/UKPLab/sentence-transformers/issues/454#issuecomment-699496454

What do you think?

Comments:
I see the following requirements:

1. automatic batching
2. maybe smart batching
3. multi GPU support
4. Tokenization with multiple processes in parallel to the prediction
5. `max_length` and `truncation` support


This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Added TF TransfoXL Sequence Classification
Author: spatil6
Pull request: yes

This PR implements Sequence classification for TF TransfoXL model.
TFTransfoXLForSequenceClassification uses the last token in order to do the classification, as other causal models (e.g. GPT-1 ,GPT-2) do.

Fixes #7623

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?

 @LysandreJik @jplu

Comments:
Added TF CTRL Sequence Classification
Author: spatil6
Pull request: yes

This PR implements Sequence classification for TF CTRL model.
TFCTRLForSequenceClassification uses the last token in order to do the classification, as other causal models (e.g. Transformer XL ,GPT-2) do.

Fixes #7623

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you write any new necessary tests?


@jplu @LysandreJik 

Comments:
Added TF OpenAi GPT1 Sequence Classification
Author: spatil6
Pull request: yes

This PR implements Sequence classification for TF OpenAi GPT1 model.
TFOpenAIGPTForSequenceClassification uses the last token in order to do the classification, as other causal models (e.g. Transformer XL ,GPT-2) do.

Fixes #7623 


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?
@LysandreJik @jplu 


Comments:
Not sure, why test cases are failing for 'run_tests_tf'. @LysandreJik . 
Let me know any action require from my side.

Can you rebase on master, a fix has been recently merged.

The tests have already been fixed on `master`, merging! Thanks a lot @spatil6