noamaanMulla-03/transformers-issues huggingface dataset:
How to use hugging on onw embedding
Author: Yangxiaojun1230
Pull request: no

i already get a embedding，or i have a matrix for input. Then I want to use transformer to train on. For example use Longformer，i set input_ids to None，and set the input on parameter embed_input. Could it be OK？ anything else need to take care？ I also input the mask.

Comments:
Hello @Yangxiaojun1230 ,

From your message I assume you are trying to train a Longformer model using embeddings as the input of the model.

You can either use input_ids or inputs_embeds in the forward function of the models. I think your idea will work, please let us know if it does.

You can look at more information about your model in the documentation: https://huggingface.co/transformers/model_doc/longformer.html


@na

> Hello @Yangxiaojun1230 ,
> 
> From your message I assume you are trying to train a Longformer model using embeddings as the input of the model.
> 
> You can either use input_ids or inputs_embeds in the forward function of the models. I think your idea will work, please let us know if it does.
> 
> You can look at more information about your model in the documentation: https://huggingface.co/transformers/model_doc/longformer.html

Thank you for your answer, actually I try to train the model and meet an error.
My code:
att_mask[:,[100,300,500,800,1200,]]=2
logit=LongformerForSequenceClassification(inputs_embeds=embding,attention_mask=att_mask)

but error occurs:
pytorch/aten/src/THC/THCTensorIndex.cu:361: void indexSelectLargeIndex(TensorInfo<T, IndexType>, TensorInfo<T, IndexType>, TensorInfo<long, IndexType>, int, int, IndexType, IndexType, long) [with T = float, IndexType = unsigned int, DstDim = 2, SrcDim = 2, IdxDim = -2, IndexIsMajor = true]: block: [205,0,0], thread: [95,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
THCudaCheck FAIL file=/pytorch/aten/src/THC/THCReduceAll.cuh line=327 error=710 : device-side assert triggered

"/home/yangjeff/anaconda3/lib/python3.6/site-packages/transformers/modeling_longformer.py", line 272, in forward
    max_num_extra_indices_per_batch = num_extra_indices_per_batch.max()
RuntimeError: cuda runtime error (710) : device-side assert triggered at /pytorch/aten/src/THC/THCReduceAll.cuh:327

In the modeling_longformer.py:
after:extra_attention_mask = attention_mask > 0
I could not access to extra_attention_mask，
But in the example:
attention_mask = torch.ones(input_ids.shape, dtype=torch.long, device=input_ids.device) # initialize to local attention
        attention_mask[:, [1, 4, 21,]] = 2  # Set global attention based on the task. For example,
                                            # classification: the 
 token
                                            # QA: question tokens
                                            # LM: potentially on the beginning of sentences and paragraphs
        sequence_output, pooled_output = model(input_ids, attention_mask=attention_mask)
The only different in my code is I input inputs_embeds instead input_ids

I fixed finally
Refactor generation sampling parameters (e.g. top k, temperature) into "Sampling" classes 
Author: turtlesoupy
Pull request: yes
Labels: name: wontfix

#4164 has a full description of the intention here. Basically, to avoid exploding generate(...) with more arguments, I've added one generic Sampler parameter that allows for arbitrary transformations of the generation probability distribution conditioned on the past. This allows users to specify custom ways of sampling (e.g. insert a specific token after a previous one, etc.)

In the process, I've added some basic tests around these samplers; existing tests pass otherwise.

Comments:
@sshleifer thanks for taking a look. The run against the tests you mentioned (bart/t5/marian) passed when I gave them a kick. When you say performance, this approach should have the same amount of compute (each enabled Sampler runs once per generation loop) since it is just moving code around unless I missed something. Let me do a rebase and see if that CI failure goes away -- let me know if you have any other concerns! 

@turtlesoupy - thanks a lot for the PR! Cool design choice! 

The `generate` method definitely needs a bigger refactor sooner or later and this is a cool idea on how to make it easier to add new probability distribution wrap functions. With this design I'm a bit worried that we restrict beam search too much in a sense that only the log_softmax of the "next_tokens" distribution can "wrapped" but not the summed distribution of the `next_token_scorers + beam_scores`. Here this will break the beam search + sampling case (if I understood the code correctly).

I guess a method that adapts the `_beam_scores + next_token_scores` could also be used in "greedy" beam search in the future and this design choice would block us a bit. But I'm not sure whether there are many use cases one would like to adapt `_beam_scores + next_token_scores` before appling `top_k` for "greedy" beam search...what are your thoughts on this? @turtlesoupy @yjernite @sshleifer 

@patrickvonplaten I'm un-opinionated since my use cases weren't using beam search; the goal of this PR was so that I could introduce a my own sampler that enforced rules without having to fork the generate function.

For beam search, one approach could be to apply the warp to (`next_token_scores + beam_scores`) and then perform sampling afterwards. Then it is sampling from a consistent space and the hypothesis scores would be modified appropriately 

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
DPR model examples / notebook / pipeline
Author: flozi00
Pull request: no
Labels: name: wontfix

# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->
An notebook or own script in the examples folders would be pretty nice for the new DPR model, implemented by @lhoestq 
Maybe an implementation to the pipeline would be cool too

## Motivation

<!-- Please outline the motivation for the proposal. Is your feature request
     related to a problem? e.g., I'm always frustrated when [...]. If this is related
     to another GitHub issue, please link here too. -->

Such an notebook or script would make it easier to train an custom dpr model

## Your contribution

<!-- Is there any way that you could help, e.g. by submitting a PR?
     Make sure to read the CONTRIBUTING.MD readme:
     https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md -->

I did not read the paper and had no closer look to the code, but with some advice I could try to start an PR


Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
New run_language_modeling.py continuing trainng
Author: parmarsuraj99
Pull request: no

I tried using new script for Language Modeling to train model from scratch.

I was training model with old script. When I tried continuing using new one, there is no option for `--should_continue`.

Does `--overwrite_output_dir` makes it train from scratch if i use same directory for `--model_name_or_path` and `--output_dir`?

So, I am continuing training with old script.

Comments:
Yes, we removed the `--should_continue` flag. You can just use your latest checkpoint as `--model_name_or_path`

Let us know if this helps

Yup, The new script is really clean and understandable. To continue training, I should use `--overwrite_output_dir`?  Because, when I point to last checkpoint directory using `-- model_name_or_path`, It asks me for `--output_dir` which must be empty,  so I had to switch between two directories. 

Yes you would use `--overwrite_output_dir` (it was implicitly added before)

You would do:
```
--model_name_or_path ./model_name/checkpoint-9000
--output_dir ./model_name
--overwrite_output_dir
```

If this is a frequent request we could add it back but I feel like it was more confusing than anything.

Yes would like to get this feature back. 

I am using [TrainingArguments](https://huggingface.co/transformers/main_classes/trainer.html#trainingarguments) but couldn't continue training from last checkpoint. It is starting from step 0.  Any suggestion?

```
training_args = TrainingArguments(
    output_dir=last_checkpoint,
    overwrite_output_dir=True,
...
)
 trainer = Trainer( model=model, #roberta config model
           args=training_args,
...
)
```

e.g., `last_checkpoint="/saved_model/checkpoint-20000/"`
@julien-c
some sshleifer/xsum hub models  have bart-large-cnn task_specific_params['summarization']
Author: sshleifer
Pull request: no

S3 only fix! 


Comments:
Done. 

- [x] pegasus xsum students
- [x] uploaded distilled pegasus
- [x] re-evaluate updated layerdrop model  (te: 23.5291: quickly worsens)
- [x] re-evaluate bart-xsum pseudolabel model(s).
sts-b task score is far worse than other GLUE tasks
Author: sjpark9503
Pull request: no
Labels: name: wontfix

## ❓ Questions & Help

<!-- A clear and concise description of the question. -->
Hello,
I'm evaluating my GPT style model pretrained on TEXT8 dataset with GLUE.
Below is the evaluation result.

```
CoLA | SST-2 |     MRPC    |    QQP    |    STS-B    | MNLI | QNLI | RTE  | WNLI
19.1 |   85  | 82.5 / 71.6 | 78.4 / 82 | 41.6 / 39.4 | 62.8 | 74.6 | 57.4 | 56.3
```

Compare to other GLUE tasks, STS-B scores seem far worse than baseline in leader board.
Does anyone know why model works bad on STS? Thanks a lot.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Have you resolved this issue? I observed a much worse result.
[help] Add multigpu evalulation script for seq2seq
Author: sshleifer
Pull request: no
Labels: name: Help wanted

name: seq2seq

name: Examples

name: translation

### Goal
At the end the command,
```
cd examples/seq2seq
export DATA_DIR=test_data  # use wmt_en_ro for more data, download instructions in README.md
export save_dir=translations
python multigpu_eval.py Helsinki-NLP/opus-mt-en-ro $DATA_DIR/test.source $save_dir/test_translations.txt --reference_path $DATA_DIR/test.target --score_path $save_dir/test_bleu.json --fp16 --task translation --bs 16
```
On wmt_en_ro, thissShould get ~27.7 (same result as run_eval.py on 1 GPU) and should take less time.
On test_data I don't know the 1 GPU result.

side note: The motivation here is actually summarization, where the validation sets are larger and on 1 GPU for pegasus running eval takes an hour.


### Implementation

This looks similar, https://github.com/facebookresearch/ParlAI/blob/00efcbebb49524918692638ab580cadeebe70cf8/parlai/scripts/multiprocessing_eval.py#L34 but I'm open to many strategies. Lots of freedom. You will probably need to use one of the datasets in utils + an unshuffled dataloader of some sort. Try to keep the command line as similar to run_eval.py as possible, with the exception of a --gpus flag. Alternatively, use all available gpus and tell the user to use CUDA_VISIBLE_DEVICES=0,1 to restrict.


### Process

1. You can claim the bug here, but don't wait for me to reply to get started. This is hard enough that it is very unlikely that two people will be working on this at once. If I start working on this (unlikely) I will post here and remove the help-wanted label.
2. When you send a PR, tag me and link this issue. 
3. It will be difficult to test your solution in CI, but try to make the code work with 1 GPU so we can test it that way in CI. You can make a new test_.py file or add to test_seq2seq_examples.py
4. Report BLEU scores in PR description.

Happy hacking!


Comments:
🐛 Bart Tokenization difference between 2.11.0 and 3.0.2
Author: astariul
Pull request: no
Labels: name: wontfix

# 🐛 Bug

Running this code :

```python
from transformers import BartTokenizer
tokenizer = BartTokenizer.from_pretrained("facebook/bart-large")
print(tokenizer.batch_encode_plus(["This is an example"]))
```

in `transformers` `2.11.0` and `3.0.2` gives different results.

`transformers` `2.11.0` :

> {'input_ids': [[0, 152, 16, 41, 1246, 2]], 'attention_mask': [[1, 1, 1, 1, 1, 1]]}

`transformers` `3.0.2` :

>  {'input_ids': [[0, 713, 16, 41, 1246, 2]], 'attention_mask': [[1, 1, 1, 1, 1, 1]]}

---

Colab for reproducing :

* [`2.11.0`](https://colab.research.google.com/drive/1qwYkcZoD1JtuLLDjABngJFoDUD06RiXm?usp=sharing)
* [`3.0.2`](https://colab.research.google.com/drive/1qUWcCUYInpa9Lwy2Ur-t3N3hImF1grCT?usp=sharing)

---

Is it from the refactoring of `generation_utils.py` ?


Comments:
This line :

https://github.com/huggingface/transformers/blob/b42586ea560a20dcadb78472a6b4596f579e9043/src/transformers/tokenization_utils.py#L1709

was changed to :

https://github.com/huggingface/transformers/blob/b0892fa0e8df02d683e05e625b3903209bff362d/src/transformers/tokenization_utils.py#L505

---

In `2.11.0`, if `add_special_tokens` was `True` (which was the default value), then the RoBERTa tokenizer would add automatically the prefix space.

In `3.0.2`, `add_special_tokens` is still `True` by default, but is not passed to `tokenize()` anymore. RoBERTa tokenizer does not add a prefix space, which lead to the difference observed.

---

So the fixed code for `3.0.2` is :

```python
from transformers import BartTokenizer
tokenizer = BartTokenizer.from_pretrained("facebook/bart-large")
print(tokenizer.batch_encode_plus(["This is an example"], add_prefix_space=True))
```

---

_Not closing yet as I would like to know if this is an expected breaking changes or not._

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Demoing LXMERT with raw images by incorporating the FRCNN model for roi-pooled extraction and bounding-box predction on the GQA answer set.
Author: eltoto1219
Pull request: yes

This is a follow up PR from initially incorporating LXMERT. This PR includes the Faster-RCNN code to convert raw-images into usable roi-pooled features downstream in lxmert or any other suitable vision model. 

Comments:
Temporarily skip failing tests due to dependency change
Author: LysandreJik
Pull request: yes

cc @sgugger @stas00 

Comments:
@julien-c wrote that test
https://github.com/huggingface/transformers/pull/3800/files#diff-d524f2b47fd9e3af8aea82b9fac55079R88-R90
so it's probably a good idea to ping him instead, as I'm unfamiliar, yet, with that part of `transformers`.
ignore FutureWarning in tests
Author: stas00
Pull request: yes

As discussed in https://github.com/huggingface/transformers/pull/7033 we can't deal with transformers' `FutureWarning` in tests, since we have to keep those tests around until they become normal warnings and then the tests will get fixed/adjusted. So currently they just generate noise that can't be acted upon.

The only side-effect I can see is with other libraries' FutureWarnings, which now will be silenced too, but again we can easily fix those as soon as those aren't futuristic any more.





Comments:
Yes, I agree with this. Thanks @stas00!
[examples testing] restore code
Author: stas00
Pull request: yes

For some reason https://github.com/huggingface/transformers/pull/5512 re-added temp dir creation code that was removed by
https://github.com/huggingface/transformers/pull/6494 in the process undoing what the latter PR did those tests, leading to the temp dir created twice in a row.

I see now that it was an older PR that was committed much later, so that explains why the new code was not noticed.

@Joel-hanson

Comments:
Sorry, @stas00 I didn't notice that.

All is good, it was easy to miss, @Joel-hanson.
Support for Polyencoder and other retriever based models
Author: prakhar6sharma
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help

Is there any way I can load Polyencoder and other retriever based models from ParlAI in huggingface/transformers because as of now, there seems no support of loading huggingface/transformers models in ParlAI other than GPT.

Polyencoder: https://arxiv.org/abs/1905.01969
ParlAI Implementation : https://github.com/facebookresearch/ParlAI/blob/master/parlai/agents/transformer/polyencoder.py 


Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
prepare for the label for EncoderDecoderModel
Author: lonelydancer
Pull request: no

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     -->
When i using a EncoderDecoderModel for a translation task, do i have to right shift the target token to generate the label?
The [tutorial ](https://github.com/huggingface/transformers/tree/master/model_cards/patrickvonplaten/bert2bert-cnn_dailymail-fp16) not do that, and the source code in  forward  function do neither.

## Details
<!-- Description of your issue -->

<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on the forum/Stack Overflow**:

Comments:
In BertLMHeadModel ,
shift the label
*prediction_scores = prediction_scores[:, :-1, :].contiguous()
*labels = labels[:, 1:].contiguous()
BertForMaskedLM Loss function 
Author: lonelydancer
Pull request: no

# ❓ Questions & Help

https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_bert.py

1)In BertForMaskedLM, why the loss function don't ignore the not masked tokens?
loss_fct = CrossEntropyLoss()  # -100 index = padding token
masked_lm_loss = loss_fct(prediction_scores.view(-1, self.config.vocab_size), labels.view(-1))
2) do i have to set the not masked token to -100 ?


<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     -->

## Details
<!-- Description of your issue -->

<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on the forum/Stack Overflow**:

Comments:
torch.nn.CrossEntropyLoss(weight: Optional[torch.Tensor] = None, size_average=None, ignore_index: int = -100, reduce=None, reduction: str = 'mean')
Is Writing With Transform open source?
Author: DrAlta
Pull request: no
Labels: name: wontfix

I want to use Writing with Transform with my own model?  My first though was that I could just download it and modify the source to point to my model., but I can't find WWT anywhere in the repo. Do I have to publish my model then make a request for you to add it?

Comments:
If you add your model to the hub, you'll get the inference widget & API that you can use for demos or integration into your product: https://huggingface.co/docs

We don't currently have short-term plans to open source Write With Transformers's frontend

Following up on what @clmnt said, we have an option (currently in private beta) for GPU acceleration of the inference API which would let you built similar (fast!) applications.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Add quotes to paths in MeCab arguments
Author: polm
Pull request: yes

Without quotes directories with spaces in them will fail to be processed
correctly.

This bug was first reported in the fugashi repo [here](https://github.com/polm/fugashi/issues/24). It's not a bug in fugashi - fugashi handles quoted paths correctly, and the relevant dictionary packages quote paths in `MECAB_ARGS` to handle this case. Looks like it was missed when the dictionary handling code was added, since it doesn't use `MECAB_ARGS` but instead builds args up from component parts.

Comments:
Inconsistent parameter naming conventions in ModelConfigs
Author: arkadyark
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 3.0.2 (N/A)

### Who can help
 
@LysandreJik 

## Information

This isn't exactly a bug, but more a suggestion that would be helpful. Across several different models, there are config classes used to initialize a model (e.g. GPT2Config and BertConfig). Some of these configs have parameters that mean the same thing, but use different naming conventions for the parameters, for instance n_layer in GPT2Config is the same, conceptually, as num_hidden_layers for BertConfig.

For ease of development, I think it could be worthwhile to make these parameter names consistent across all of the transformer config classes.

I'd be happy to help out on this, it seems like it should be a relatively straightforward change, though it may break some backwards compatibility.

Comments:
Hi! As you've mentioned, this would break backwards compatibility which is not something we're willing to do for a name change. Furthermore, these values are named as such to be the same as the original implementation, therefore these will not be changed.

Thank you for opening an issue!