noamaanMulla-03/transformers-issues huggingface dataset:
access to the embeddings for query and text used in a downstream NLP task
Author: mchari
Pull request: no

Newbie question -

is there any way to access the embeddings  that are generated for the passage(and query) that is fed to a BertForQuestionAnswering model ? 

Comments:
I realized that in QA, word/token embeddings are used, while I was looking for multi-sentence level embeddings. Pl. ignore my question.
The configuration of 3.0.2 and 3.1.0 is not compatible
Author: franciszzj
Pull request: no

The configuration of **3.0.2** and **3.1.0** is not compatible.

For example, in https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_bert.py#L388, `config.chunk_size_feed_forward` should change to `config.get('chunk_size_feed_forward', 0)`, because there is not `chunk_size_feed_forward` config in former version.

```
class BertLayer(nn.Module):
    def __init__(self, config):
        super().__init__()
        # self.chunk_size_feed_forward = config.chunk_size_feed_forward
        self.chunk_size_feed_forward = config.get('chunk_size_feed_forward', 0)
        self.seq_len_dim = 1
        self.attention = BertAttention(config)
        self.is_decoder = config.is_decoder
        self.add_cross_attention = config.add_cross_attention
        if self.add_cross_attention:
            assert self.is_decoder, f"{self} should be used as a decoder model if cross attention is added"
            self.crossattention = BertAttention(config)
        self.intermediate = BertIntermediate(config)
        self.output = BertOutput(config)
```

Comments:
issue #6950 is also this problem.

I see what you mean! From `3.1.0` onwards every configuration has a `config.chunk_size_feed_forward` parameter. So as far as I can see whenever a config is loaded (whether via `.from_pretrained()` or with `BertConfig(....)`, this parameter is part of the config...). Can you give me an example where this would not be the case? 

@patrickvonplaten Thanks for your replay.

Yes, I load the pretrained model with my own code, which cause this problem.
For some users who only use part of structures/classes/others in Transfomers, is it necessary to have certain compatibility?

> I see what you mean! From `3.1.0` onwards every configuration has a `config.chunk_size_feed_forward` parameter. So as far as I can see whenever a config is loaded (whether via `.from_pretrained()` or with `BertConfig(....)`, this parameter is part of the config...). Can you give me an example where this would not be the case?

I saw the `chunk_size_feed_forward` in BertConfig() after opened this issue.🤪

@patrickvonplaten 
In the future, I will make more normative to call modules.
Thanks for your replay.
I close this issue.
On Gpu sharing mechanism, specify model.to(' CPU '), but still use the Gpu
Author: changquanyou
Pull request: no

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     -->

## Details
<!-- Description of your issue -->
You know that The GPU device(K8s) only supports one container exclusive GPU, In the inferencing stage, it is extremely wasteful. For Tiny-Albert model,It's only using about 500MiB。We try to use  GPU share device, support more containers use one GPU device。We expect using  torch.cuda.is_available() to control Using CUDA or Not.
Here is My Code:
`
use_cuda = bool(os.environ.get("USE_CUDA"))
if use_cuda and torch.cuda.is_available():
    self._device = torch.device("cuda")
else:
   self._device = torch.device("cpu")
` 

1. USE_CUDA== False and  torch.cuda.is_available() is True:
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|============================================================|
|    0     11538      C   /usr/bin/python3                             **342MiB** |
+-----------------------------------------------------------------------------+
2. USE_CUDA== True and  torch.cuda.is_available() is True:
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|============================================================|
|    0     11538      C   /usr/bin/python3                             **342MiB** |
+-----------------------------------------------------------------------------+









<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on the forum/Stack Overflow**:

Comments:
It's my issue
can't get models directory after running python run_squad.py 
Author: guhuawuli
Pull request: no
Labels: name: wontfix

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...):
Bert

Language I am using the model on (English, Chinese ...):
English


The problem arises when using:
after running "python run_squad.py ", I didn't get models directory.
running time of my code in colab is only 20 minutes, I think the training process is not done, what's the problem? How to solve that?


The tasks I am working on is:
SQUaD

## To reproduce

Steps to reproduce the behavior:

1.https://qa.fastforwardlabs.com/pytorch/hugging%20face/wikipedia/bert/transformers/2020/05/19/Getting_Started_with_QA.html
2.click the "Open in Colab" button above

!python run_squad.py  \
    --model_type bert   \
    --model_name_or_path bert-base-uncased  \
    --output_dir models/bert/ \
    --data_dir data/squad   \
    --overwrite_output_dir \
    --overwrite_cache \
    --do_train  \
    --train_file train-v2.0.json   \
    --version_2_with_negative \
    --do_lower_case  \
    --do_eval   \
    --predict_file dev-v2.0.json   \
    --per_gpu_train_batch_size 2   \
    --learning_rate 3e-5   \
    --num_train_epochs 2.0   \
    --max_seq_length 384   \
    --doc_stride 128   \
    --threads 10   \
    --save_steps 5000 

## Expected behavior


## Environment info
colab GPU
     
- `transformers` version:
- Platform:colab
- Python version:3.6.9
- PyTorch version (GPU?):1.5.1+cu101
- Tensorflow version (GPU?):
- Using GPU in script?:
- Using distributed or parallel set-up in script?:


Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Bert-extractive-summaizer importing issue
Author: Vinu-4590
Pull request: no
Labels: name: wontfix

Hi ,

I am facing issues while import summarizer.

NameError                                 Traceback (most recent call last)
<ipython-input-6-3b4384c20fe2> in <module>()
----> 1 from summarizer import Summarizer
      2 
      3 body = 'Text body that you want to summarize with BERT'
      4 body2 = 'Something else you want to summarize with BERT'
      5 model = Summarizer()

~/anaconda3/envs/amazonei_tensorflow_p36/lib/python3.6/site-packages/summarizer/__init__.py in <module>()
----> 1 from summarizer.model_processors import Summarizer, SingleModel, TransformerSummarizer

~/anaconda3/envs/amazonei_tensorflow_p36/lib/python3.6/site-packages/summarizer/model_processors.py in <module>()
----> 1 from summarizer.bert_parent import BertParent
      2 from summarizer.cluster_features import ClusterFeatures
      3 from summarizer.sentence_handler import SentenceHandler
      4 from typing import List
      5 from abc import abstractmethod

~/anaconda3/envs/amazonei_tensorflow_p36/lib/python3.6/site-packages/summarizer/bert_parent.py in <module>()
      9 
     10 
---> 11 class BertParent(object):
     12 
     13     """

~/anaconda3/envs/amazonei_tensorflow_p36/lib/python3.6/site-packages/summarizer/bert_parent.py in BertParent()
     16 
     17     MODELS = {
---> 18         'bert-base-uncased': (BertModel, BertTokenizer),
     19         'bert-large-uncased': (BertModel, BertTokenizer),
     20         'xlnet-base-cased': (XLNetModel, XLNetTokenizer),

**NameError: name 'BertModel' is not defined**

Please help me on this issue

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Fix typo
Author: mrm8488
Pull request: yes

<!-- This line specifies which issue to close after the pull request is merged. -->
Fixes #{issue number}


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/6994?src=pr&el=h1) Report
> Merging [#6994](https://codecov.io/gh/huggingface/transformers/pull/6994?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/90ec78b5140251f093f658ebd4d2925e8c03f5e6?el=desc) will **decrease** coverage by `0.54%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/6994/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/6994?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #6994      +/-   ##
==========================================
- Coverage   80.58%   80.04%   -0.55%     
==========================================
  Files         161      161              
  Lines       30123    30123              
==========================================
- Hits        24276    24111     -165     
- Misses       5847     6012     +165     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/6994?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_tf\_electra.py](https://codecov.io/gh/huggingface/transformers/pull/6994/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9lbGVjdHJhLnB5) | `25.13% <0.00%> (-73.83%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/6994/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `86.97% <0.00%> (-0.33%)` | :arrow_down: |
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/6994/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `82.41% <0.00%> (-0.26%)` | :arrow_down: |
| [src/transformers/data/data\_collator.py](https://codecov.io/gh/huggingface/transformers/pull/6994/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9kYXRhL2RhdGFfY29sbGF0b3IucHk=) | `92.71% <0.00%> (+6.07%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_flaubert.py](https://codecov.io/gh/huggingface/transformers/pull/6994/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9mbGF1YmVydC5weQ==) | `88.34% <0.00%> (+63.80%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/6994?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/6994?src=pr&el=footer). Last update [90ec78b...09dda6e](https://codecov.io/gh/huggingface/transformers/pull/6994?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Huggingface "sentiment-analysis" pipeline always output "POSITIVE" label even for negative sentences
Author: abmitra84
Pull request: no

## Environment info
- `transformers` version: 3.1
- Platform: AWS , Colab etc.
- Python version: 3.6
- PyTorch version (GPU?): NA
- Tensorflow version (GPU?): NA
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help

 examples/token-classification: @stefan-it


## Information

Model I am using (Bert: https://huggingface.co/distilbert-base-uncased-finetuned-sst-2-english):

The problem arises when using:
* [T ] the official example scripts: (give details below)
from transformers import pipeline
classifier = pipeline('sentiment-analysis')

classifier("I hate this thing") -->Returns Postive label
In fact when tested with a large bacth it never returned Negative label at all

* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Get transformer 3.1
2. Use pipeline("sentiment-analysis")
3. Get label and score

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

from transformers import pipeline
classifier = pipeline('sentiment-analysis')

classifier("I hate this thing") -->Returns Negative label

Updates post some more experiments:

- I am using  distilbert-base-uncased-finetuned-sst-2-english following instruction from :https://huggingface.co/distilbert-base-uncased-finetuned-sst-2-english
- When I download the model and save it in local before using it through AutoModel and AutoTokenizer the score for sentence nlp("i dislike it") is 0.5453
- when I directly use it without manually downloading the model to local the same score jumps to 0.99 [same score in hosted modelhub inference api as well]
- another observation is that the tokenizer_config.json has both special_tokens_map_file and full_tokneizer_file as Null. Not sure if it supposed to be like this
- for my purpose I have to use the downloaded and manually loaded style of inferencing

<!-- A clear and concise description of what you would expect to happen. -->


Comments:
Upon further updates, I think the issue was with using wrong AutoModel function. I replaced it with AutoModelforSequenceClassification at the time of download and it worked.
PyTorch (with GPU) Trainer leaks CPU memory on Google Colab
Author: rajaswa
Pull request: no

## Environment info
- `transformers` version: 3.1.0
- Platform: Linux-4.19.112+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.6.0+cu101 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No


### Who can help
Trainer: @sgugger

## Information

Model I am using `RoBERTaForMaskedLM`:
`
config = RobertaConfig(
    vocab_size=32_000,
    max_position_embeddings=256+2,
    num_attention_heads=12,
    num_hidden_layers=6,
    type_vocab_size=1,
)
`
The problem arises when using:
* [ ] [my own modified scripts](https://colab.research.google.com/drive/1bVv6V9IIhTNXvWbpUvQyPyRpK3LOgFfz?usp=sharing): I am trying to follow the official blog tutorial on training a language model from scratch. I have made a few changes from the official script (using Marathi OSCAR corpus, changed model config, and vocabulary size, and am fetching the sentences in the dataset on the fly

The tasks I am working on is:
* [ ] my own task or dataset: Masked Language Modelling with RoBERTa on Marathi Oscar Corpus

## To reproduce

Steps to reproduce the behavior:

1. Run this [colab notebook](https://colab.research.google.com/drive/1bVv6V9IIhTNXvWbpUvQyPyRpK3LOgFfz?usp=sharing).

## Expected behavior
The RAM consumption starts rising towards the end of the first epoch, ultimately crashing the entire session due to full memory consumption (12.72GB of Colab RAM)


Comments:
Add support for Funnel-Transformer
Author: stefan-it
Pull request: no
Labels: name: New model

# 🌟 New model addition

## Model description

The recently introduced Funnel-Transformer architecture and models would be a great feature for Transformers:

>Funnel-Transformer is a new self-attention model that gradually compresses the sequence of hidden states to a shorter one and hence reduces the computation cost. More importantly, by re-investing the saved FLOPs from length reduction in constructing a deeper or wider model, Funnel-Transformer usually has a higher capacity given the same FLOPs. In addition, with a decoder, Funnel-Transformer is able to recover the token-level deep representation for each token from the reduced hidden sequence, which enables standard pretraining.

The paper can be found [here](https://arxiv.org/abs/2006.03236).

## Open source status

* [x] the model implementation is available: [official GitHub repo](https://github.com/laiguokun/Funnel-Transformer)
* [x] the model weights are available: [Google Cloud Bucket](https://github.com/laiguokun/Funnel-Transformer/blob/master/download_all_ckpts.sh)
* [x] who are the authors: @zihangdai and @laiguokun


Comments:
Will start to look into this.

@sgugger Any updates on this? Thanks! 

The first models are uploaded and the base models are available in PyTorch (`FunnelModel` has encoder + decoder and `FunnelBaseModel` just the encoder, for sequence classification and multiple choice) in [this branch](https://github.com/huggingface/transformers/tree/funnel_transformer). Should have all checkpoints on the HuggingFace S3 and all PyTorch models on the same branch by the end of this week.

Note that there might be some changes in the names as this goes under review once it's ready.
[Community notebooks] Add notebook on fine-tuning GPT-2 Model with Trainer Class
Author: philschmid
Pull request: yes

Adding a link to a community notebook containing an example of fine-tuning a German GPT-2 Model with the Trainer Class

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/7005?src=pr&el=h1) Report
> Merging [#7005](https://codecov.io/gh/huggingface/transformers/pull/7005?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/c18f5916a03d0a161d003e95ffff8120d8addc0c?el=desc) will **decrease** coverage by `0.59%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/7005/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/7005?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #7005      +/-   ##
==========================================
- Coverage   80.63%   80.03%   -0.60%     
==========================================
  Files         161      161              
  Lines       30123    30123              
==========================================
- Hits        24289    24109     -180     
- Misses       5834     6014     +180     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/7005?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_tf\_electra.py](https://codecov.io/gh/huggingface/transformers/pull/7005/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9lbGVjdHJhLnB5) | `25.13% <0.00%> (-73.83%)` | :arrow_down: |
| [src/transformers/data/data\_collator.py](https://codecov.io/gh/huggingface/transformers/pull/7005/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9kYXRhL2RhdGFfY29sbGF0b3IucHk=) | `91.90% <0.00%> (-0.81%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/7005/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `86.97% <0.00%> (-0.33%)` | :arrow_down: |
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/7005/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `82.66% <0.00%> (+0.25%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_flaubert.py](https://codecov.io/gh/huggingface/transformers/pull/7005/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9mbGF1YmVydC5weQ==) | `88.34% <0.00%> (+63.80%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/7005?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/7005?src=pr&el=footer). Last update [c18f591...0d1e538](https://codecov.io/gh/huggingface/transformers/pull/7005?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).