noamaanMulla-03/transformers-issues huggingface dataset:
How to return the word embeddings and how to understand the hidden_states in return?
Author: island99
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     -->

## Details
<!-- Description of your issue -->
I am new here, transformers is amazing. What I want is to get the embeddings of words with context , “logits” is obviously a bit thin, I think the hidden_states may represent the word embedding, but I can not understand the output of the embeddings and output of each layer.
Which one should I choose?
Thanks very much!

> hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) – Tuple of torch.FloatTensor (one for the output of the embeddings + one for the output of each layer) of shape (batch_size, sequence_length, hidden_size).

<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on the forum/Stack Overflow**:

Comments:
Well if you want the embeddings of words with their context, your best bet is to take the entire output of the model. If you take only the word embeddings, these are contextless. 

You should take the model with no head, i.e., the base model: `BertModel`, `GPT2Model`, etc. output to get this value.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
MASS : A generalization of BERT and GPT
Author: Jeevesh8
Pull request: no
Labels: name: wontfix

name: New model

# 🌟 New model addition

## Model description

MASS is a novel pre-training method for sequence to sequence based language generation tasks. It randomly masks a sentence fragment in the encoder, and then predicts it in the decoder. In this way, MASS can jointly train the encoder and decoder to develop the capability of representation extraction and language modeling. This pre-training is very helpful when the encoder and decoder are shared between multiple languages.

## Open source status

- [x] the model implementation is available :  The model is implemented upon fair-seq [here.](https://github.com/microsoft/MASS)
- [x] the model weights are available: Pre-trained model on various language pairs, for unsupervised translation, supervised translation and abstractive summarization are provided on the GitHub repo itself.
- [x] Kaitao Song, Xu Tan,  Tao Qin,  Jianfeng Lu,  Tie-Yan Liu  are the authors: ( @StillKeepTry , @tobyoup , @xutaatmicrosoftdotcom )


This is my first time contributing to this repository, so forgive me for any mistake . Please let know whether I should do it or not. Also, if anyone wants to come along and help, please let me know that too ! 😀

Comments:
Can also try MP-Net of theirs next . 

Sorry, just saw the request for MP-Net [here](https://github.com/huggingface/transformers/issues/4308) . Seems I was behind. So, shall I close this issue, or does anyone still want separate MASS model here ? @RyanHuangNLP

@RyanHuangNLP  @StillKeepTry , @tobyoup , @xutaatmicrosoftdotcom 

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Seq2SeqDataset experiment: try to use arrow datasets
Author: sshleifer
Pull request: no
Labels: name: Help wanted

name: wontfix

### High Level Goal
See if [`datasets/arrow_dataset.py`](https://github.com/huggingface/datasets/blob/master/src/datasets/arrow_dataset.py#L172) can improve `example/seq2seq/utils.py`'s [`Seq2SeqDataset`](https://github.com/huggingface/transformers/blob/master/examples/seq2seq/utils.py#L165)

This is more experimental/proof of concept.

### Spec
+ Generate a PR that adds `Seq2SeqArrowDataset` with a similar signature to `Seq2SeqDataset` but uses the arrow dataset as a backend.
+ This means that wherever `linecache` is used, arrow should be used.
+ You can flex the API as much as you'd like, but if you don't call `prepare_seq2seq_batch` you will have to implement lots of collate fns. (If you go this route, you can just implement 1)
+ Feel free to reduce the scope to either translation or summarization.

PR description should answer the following question:

- Can we generate similar batches with an ArrowDataset?
- Is the dataset faster?
- Does it consume as little RAM?
- Does it simplify the code?

Let me know if this is impossible or unclear!


Comments:
@sshleifer  could please assign me to this issue, would love to take a stab at this alongside `Seq2SeqTrainer` or once it's merged. ;)

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Add tokenizer file save in convert_graph_to_onnx.py 
Author: ierezell
Pull request: no
Labels: name: wontfix

# 🚀 Feature request

Allow saving the associated tokenizer files (to be loaded possibly with the tokenizer library) when running `convert_graph_to_onnx.py`  
## Motivation

Having the good vocab file next to the good onnx model to avoid confusion and ease the loading process by any other framework (because models and their tokenizers are deeply linked). 

## Your contribution

With [this comment from tokenizer repo](https://github.com/huggingface/tokenizers/issues/59#issuecomment-610645970) and the [convert_graph_to_onnx.py](https://github.com/huggingface/transformers/blob/master/src/transformers/convert_graph_to_onnx.py) one can simply add 

L.358
```
if tokenizer is None:
        download_vocab_files_for_tokenizer(nlp.tokenizer, model, output)
```
and before `if __name__ == "__main__"` 
```
def download_vocab_files_for_tokenizer(tokenizer, model_type, output_path):
    vocab_files_map = tokenizer.pretrained_vocab_files_map
    vocab_files = {}
    for resource in vocab_files_map.keys():
        download_location = vocab_files_map[resource][model_type]
        f_path = path.join(output_path.parent,
                           path.basename(download_location))
        urllib.request.urlretrieve(download_location, f_path)
```

I can do a PR for it if the interest is big enough. 

NB : This is a patch I added in my local lib but feel free to modify it or transform it as you want, it's just to give the idea


Comments:
cc. @mfuntowicz 

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
BERT uncased model outputs a tuple instead of a normal pytorch tensor 
Author: rajaswa
Pull request: no

While finetuning the BERT uncased model for sequence classification as follows:

```
config = BertConfig.from_pretrained('bert-base-uncased')
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
model = BertForSequenceClassification(config)

for layer, child in model.named_children():
    if layer not in ['classifier']:
        for param in child.parameters():
            param.requires_grad = False

optimizer = optim.SGD(model.parameters(), lr=1e-3)
criterion = nn.CrossEntropyLoss()

for (data, target) in (train_loader):
    data, target = data.to(device), target.to(device)   
    
    optimizer.zero_grad()
    output = model(data)
    target = target.float()
    loss = criterion(output, target)
    loss.backward()
    optimizer.step()

```
The following error comes up :
```
   1348         dim = _get_softmax_dim('log_softmax', input.dim(), _stacklevel)
   1349     if dtype is None:
-> 1350         ret = input.log_softmax(dim)
   1351     else:
   1352         ret = input.log_softmax(dim, dtype=dtype)

AttributeError: 'tuple' object has no attribute 'log_softmax'
```
Here are the model output and target tensors : 
```
--> output
(tensor([[-0.2530,  0.0788],
         [-0.1457, -0.0624],
         [-0.3478, -0.2125],
         [-0.1337,  0.2051],
         [ 0.0963,  0.3762],
         [-0.0910, -0.0527],
         [-0.1743,  0.2566],
         [-0.2223,  0.4083],
         [-0.1602, -0.0012],
         [-0.0059,  0.2334],
         [-0.3407, -0.1703],
         [-0.1359,  0.0776],
         [-0.2117,  0.1641],
         [-0.3365, -0.1266],
         [-0.1682,  0.0504],
         [-0.2346,  0.2380]], device='cuda:0', grad_fn=<AddmmBackward>),)

--> target

tensor([0., 0., 1., 1., 1., 0., 0., 0., 1., 1., 1., 0., 0., 0., 1., 0.],
       device='cuda:0')
```

Comments:
Hi, I was wondering how you managed to resolve this issue? I'm running into a similar problem. :) 

Hi, the model outputs are well documented, they're *always* tuples, even if there's a single return value. You can check the documentation [here](https://huggingface.co/transformers/main_classes/output.html).

@jacqueline-he  did you resolve the issue?

@WeeHyongTok Yes I have! I only needed to access what's inside the returned tuple. @LysandreJik's recommendation was very helpful.
Add instructions for syncing forked masters to avoid references
Author: bdalal
Pull request: no

# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->

## Motivation

<!-- Please outline the motivation for the proposal. Is your feature request
     related to a problem? e.g., I'm always frustrated when [...]. If this is related
     to another GitHub issue, please link here too. -->
Ticket created based on discussion https://github.com/lucidworks/transformers/commit/46b17d206529206848116fd6219643446bac938c#commitcomment-44356945
The problem is when someone on a forked repository decides to sync up their masters with upstream (HF master) using a branch and a PR, all the PR and issue references on the upstream will make their way into the forked PR's commit history, if that user creates a merge commit. 
Since GitHub autolinks issues and PRs on public forks, this will end up pinging the devs responsible for the referenced PRs creating unnecessary noise. The solution is to use a squashed merge.

One way to educate users with forked repos about this potential issue is to add instructions on how to do so to the `CONTRIBUTING.md` file.

## Your contribution

<!-- Is there any way that you could help, e.g. by submitting a PR?
     Make sure to read the CONTRIBUTING.MD readme:
     https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md -->
A PR with instructions to avoid this situation will be up shortly.

cc @stas00 

Comments:
[Help] GPU with query answering
Author: thiagomoeng
Pull request: no

I want to figure out some way to get faster results from a QA model. I did some tests on google cloud with different GPUs and got some results, those tests was made with different GPUs and same CPU using this code:

```
from transformers import AutoTokenizer, AutoModelForQuestionAnswering
from transformers.pipelines import pipeline

tokenizer = AutoTokenizer.from_pretrained("deepset/bert-large-uncased-whole-word-masking-squad2")
model = AutoModelForQuestionAnswering.from_pretrained("deepset/bert-large-uncased-whole-word-masking-squad2")
nlp_qa = pipeline('question-answering', model=model, tokenizer=tokenizer)
X = nlp_qa(context = text, question=queryy, topk = 3, device = 0, max_answer_len = 50)
```
Where context is just a long string and the question a simple query, and I got those results:

```
TESTE 1: **********
4 vCPUs 15Gb RAM 
NVIDIA TESLA P100X1
Tempo1: 1:45 min
Tempo2: 1:40 min
Tempo3: 1:45 min
***************
***************
TESTE 2: **********
4 vCPUs 15Gb RAM 
NVIDIA TESLA V100X1
Tempo1: 1:58 min
Tempo2: 1:58 min
Tempo3: 1:55 min
***************
***************
TESTE 3: **********
4 vCPUs 15Gb RAM 
NVIDIA TESLA K80X1
Tempo1: 2:06 min
Tempo2: 2:18 min
Tempo3: 2:20 min
***************
***************
TESTE 4: **********
4 vCPUs 15Gb RAM 
NVIDIA TESLA T4X1
Tempo1: 1:45 min
Tempo2: 1:50 min
Tempo3: 1:50 min
***************
***************
TESTE 5: **********
4 vCPUs 15Gb RAM 
NVIDIA NONE
Tempo1: 2:22 min
Tempo2: 1:57 min
Tempo3: 1:57 min
```

I want to know if I am  using GPU wrong, or is it normal to get almost same results with and without GPU on this set?.
Is there anyway to get faster results?

Comments:
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discusss.huggingface.co) instead?

Thanks!
AttributeError: type object 'T5ForConditionalGeneration' has no attribute 'from_config'
Author: rabeehk
Pull request: no

## Environment info
- `transformers` version: 3.5.1
- Platform: Linux
- Python version: 3.7
- PyTorch version (GPU?): 1.6
- Tensorflow version (GPU?): - 
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no 

### Who can help

 Model Cards: @julien-c
 Text Generation: @patrickvonplaten @TevenLeScao
 T5: @patrickvonplaten
 examples/seq2seq: @patil-suraj

## Information

Hi
I would like to use T5 untrained here is the command I try:

` model = T5ForConditionalGeneration.from_config(config=config)
`
I am getting this error, could you assist me please? thank you Looks weird error to me, since from_config should work based on documentations. thanks 

```
  File "finetune_t5_trainer.py", line 88, in main
    model = T5ForConditionalGeneration.from_config(config=config)
AttributeError: type object 'T5ForConditionalGeneration' has no attribute 'from_config'

```


## To reproduce
Please run the command given

## Expected behavior
load unpretrained model 



Comments:
I also tried to use automodel for this, I needed to modify the T5Config, I called it like this with automodel. thank you for your help. I need to make this work with not pretrained model. 

`  model = AutoModel.from_config(config)
`
I am getting this error:

```
Traceback (most recent call last):
  File "finetune_t5_trainer.py", line 236, in <module>
    main()
  File "finetune_t5_trainer.py", line 89, in main
    model = AutoModel.from_config(config)
  File "/idiap/user/rkarimi/libs/anaconda3/envs/internship/lib/python3.7/site-packages/transformers/modeling_auto.py", line 604, in from_config
    config.__class__, cls.__name__, ", ".join(c.__name__ for c in MODEL_MAPPING.keys())
ValueError: Unrecognized configuration class <class 'seq2seq.models.t5.configuration_t5.T5Config'> for this kind of AutoModel: AutoModel.
Model type should be one of RetriBertConfig, T5Config, DistilBertConfig, AlbertConfig, CamembertConfig, XLMRobertaConfig, BartConfig, LongformerConfig, RobertaConfig, LayoutLMConfig, SqueezeBertConfig, BertConfig, OpenAIGPTConfig, GPT2Config, MobileBertConfig, TransfoXLConfig, XLNetConfig, FlaubertConfig, FSMTConfig, XLMConfig, CTRLConfig, ElectraConfig, ReformerConfig, FunnelConfig, LxmertConfig, BertGenerationConfig, DebertaConfig, DPRConfig, XLMProphetNetConfig, ProphetNetConfig.



```

solved with model = T5ForConditionalGeneration(config=config)  syntax has been changed from 3.5.0 to 3.5.1 thanks
fix rag index names in eval_rag.py example
Author: lhoestq
Pull request: yes

There was a mistake in the eval_rag.py parameters choices.

As specified in the rag configuration (see [documentation](https://huggingface.co/transformers/model_doc/rag.html?highlight=rag#transformers.RagConfig)), one can choose between 'legacy', 'exact' and 'compressed'. The legacy index is the original index used for RAG/DPR while the other two use the `datasets` library indexing implementation.

This issue was reported on the forum https://discuss.huggingface.co/t/rag-retriever-hf-vs-legacy-vs-exact-vs-compressed/2135/5

Comments:
[core] implement support for run-time dependency version checking
Author: stas00
Pull request: yes

As discussed at https://github.com/huggingface/transformers/pull/8073#issuecomment-729181330 this PR:

* [x] adds mechanics for run-time dependency version checks (module with fixed and unversioned too)
* [x] adds thorough tests (this is where it's the easiest to see how things work)
* [x] creates one source for all dependency versions in setup.py - need to rerun setup.py on its update to re-generated  src/transformers/dependency_versions_table.py, which is then used by transformers
* [x] adds support and deploys python runtime version check
* [x] deploys runtime checks for versioned modules in setup.py's `install_requires` (i.e. must modules)
* [x] switches `examples/lightning_base.py` to a fatal-on-failure requirement check.
* [x] deploys the version lookup `setup.py`'s `extras` definitions and `install_requires`
* [x] adds new `Makefile` target `deps_table_update` that updates the dep table, and inserts it into `style/quality/fixup` targets so the sync shouldn't take too long if forgotten to be run explicitly

@sgugger, @LysandreJik, @patrickvonplaten, @thomwolf 


Comments:
@sgugger, wrt your comment in your [announcement](https://discuss.huggingface.co/t/transformers-v4-0-0-announcement/1990/1)
> => Resulting breaking change: some people will have to install sentencepiece explicitly while they didn’t have to before with the command pip install transformers[sentencepiece].

After this PR, you can just `require_version("sentencepiece", "pip install transformers[sentencepiece]")` in the code that needs it at run-time. You can expand the hint (second arg) as you wish to be self-explanatory.

@LysandreJik and @sgugger - it's ready to merge whenever you have a chance to review. Thank you.

Probably it is best to merge post v4-release, in case I missed something.

Will do a final review tomorrow!
Model can't be downloaded
Author: moniquebm
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 3.1.0
- Platform: Windows-10-10.0.19041-SP0
- Python version: 3.8.5
- PyTorch version (GPU?): 1.6.0 (True)
- Tensorflow version (GPU?): not installed (NA)

### Who can help
@julien-c 
 
## To reproduce

I've recently shared a finetunned model - monilouise/ner_pt_br - and despite its model card is already available at https://github.com/huggingface/transformers/tree/master/model_cards, the following error occurs when I try to download it:

OSError: Can't load config for 'monilouise/ner_pt_br'. Make sure that:

- 'monilouise/ner_pt_br' is a correct model identifier listed on 'https://huggingface.co/models'

- or 'monilouise/ner_pt_br' is the correct path to a directory containing a config.json file

BUT the model has config.json available at https://huggingface.co/monilouise/ner_pt_br/tree/main.

I used the following code to download the model:

```python
from transformers import BertForTokenClassification
model = BertForTokenClassification.from_pretrained('monilouise/ner_pt_br')
```

Is there anything else missing from the recommended sharing procedures?

Thanks in advance.


Comments:
Hey @moniquebm, 

I cannot reproduce this error on master.

it looks like you are working with an old version of transformers. Could you try updating `transformers` to `3.5.0` to see if the error persists?

Another reason why this doesn't work might be that you have a directory locally that is also called `monilouise/ner_pt_br`, so that instead of downloading from the model hub `from.pretrained()` tries to load a local model. You can check whether this might  be the problem by running the command from a different directory or checking whether you have a local dir called `monilouise`.

@patrickvonplaten is 100% right on his first guess that it's due to using transformers < `v3.5.x`

We backport new git-based models back to the previous S3 bucket (for models to be usable on previous versions of the library) automatically, however there was a hiccup yesterday that crashed the process (it's currently sync'ing again).

Hi @patrickvonplaten and @julien-c ,  

I've just updated transformers version and it worked.  

Thanks!
Create README.md
Author: mrm8488
Pull request: yes
Labels: name: model card

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSTM: @stas00
 -->


Comments: