noamaanMulla-03/transformers-issues huggingface dataset:
Wrong tokenizer configuration in sentiment-analysis pipeline
Author: leonhardhennig
Pull request: no
Labels: name: wontfix

name: Core: Pipeline

# 🐛 Bug

## Information

When following the Pipelines Notebook 03-pipelines.ipynb, Sentiment Analysis tasks gives wrong result ("NEGATIVE") for example 'Such a nice weather outside !'. 
```
nlp_sentence_classif = pipeline('sentiment-analysis')
nlp_sentence_classif('Such a nice weather outside !')
[{'label': 'NEGATIVE', 'score': 0.97545063}]
```

Probable reason: pipelines.py configuration uses uncased model, but cased tokenizer. Tokenizer should probably be 'distilbert-base-uncased'. 

```
"sentiment-analysis": {
        "impl": TextClassificationPipeline,
        "tf": TFAutoModelForSequenceClassification if is_tf_available() else None,
        "pt": AutoModelForSequenceClassification if is_torch_available() else None,
        "default": {
            "model": {
                "pt": "distilbert-base-uncased-finetuned-sst-2-english",
                "tf": "distilbert-base-uncased-finetuned-sst-2-english",
            },
            "config": "distilbert-base-uncased-finetuned-sst-2-english",
            "tokenizer": "distilbert-base-cased",
        },
    },
``` 

Model I am using (Bert, XLNet ...): distilbert-base-uncased-finetuned-sst-2-english (preconfigured sentiment-analysis pipeline)

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [x] the official example scripts: 03-pipelines.ipynb
* [ ] my own modified scripts: (give details below)


## Expected behavior

Example sentence should be labeled as POSITIVE.

## Environment info
     
- `transformers` version: 2.8.0
- Platform: Linux Mint
- Python version: 3.7
- PyTorch version (GPU?): 1.4.0 (no)
- Tensorflow version (GPU?): 2.1.0 (no)
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no


Comments:
I'm having the same issue

> I'm having the same issue

I got this working by using the following code:
```
 # Allocate a pipeline for sentiment-analysis
    nlp = pipeline("sentiment-analysis")
    nlp.tokenizer = transformers.DistilBertTokenizer.from_pretrained("**distilbert-base-uncased**")
```

Thanks for pointing me in the right direction LysandreJik!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Albert pretrained weights change across runs. 
Author: CVxTz
Pull request: no

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): TFAlbertModel 

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [X] my own modified scripts: (give details below)
```
import tensorflow as tf
from transformers import AlbertTokenizer, TFAlbertModel

tokenizer = AlbertTokenizer.from_pretrained('albert-base-v2')
model = TFAlbertModel.from_pretrained('albert-base-v2')
model.summary()

print(len(model.trainable_weights))

print(model.trainable_weights[23])

input_ids = tf.constant(tokenizer.encode("Hello, my dog is cute"))[None, :]

outputs = model(input_ids)

print(outputs[0].shape, outputs[1].shape, len(outputs))
last_hidden_states = outputs[0]

print(last_hidden_states)
```
The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [X] my own task or dataset: (give details below)
Trying to load pre-trained weights
## To reproduce
Run the code above two times and you will see that the weights of the model are not the same across the two runs
Steps to reproduce the behavior:

1. Run the code the first time and log the output
2. Run the code a second time and log the output
3. Check that the two logs are not the same.

## Expected behavior

Since the model is loading pre-trained weights the results should be the same across runs.

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 2.11.0
- Platform: Linux-4.4.0-179-generic-x86_64-with-debian-stretch-sid
- Python version: 3.6.9
- PyTorch version (GPU?): 1.4.0 (True)
- Tensorflow version (GPU?): 2.0.1 (True)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

I apologize if the issue is due to me misusing your library, first time using Albert.

Comments:
I just did the same experiment with Roberta weights and did not have the same issue.

Hi, I can reproduce. This is due to the archive maps not being available anymore, and therefore the wrong ALBERT models are linked.

Thanks for raising the issue, this is quite a bug.

cc @julien-c 

My bad! It's my fault. I added a warning to the release notes about this: https://github.com/huggingface/transformers/releases/tag/v2.11.0

Is there a plan to fix this? Looks like the issue is that the "real" model we want is named `with-prefix-tf_model.h5`, which needs to be renamed to `tf_model.h5`. https://huggingface.co/albert-base-v2#list-files

This should work now, the weights have been changed to use the `with-prefix` weights.

Thanks !!
What do the following parameters mean during the initialization of T5 model?
Author: abhisheknovoic
Pull request: no

Hello,

  I am aware of the general Transformer model and I believe it is the same model used in T5 architecture. 

I know we have the input vocab_size which is the total number of vocabulary size. Besides this, the important parameters would be the embedding size (the size of the embedding of each token), the number of layers, the number of heads and others.

Particularly, looking at the T5Config class that is used to initialize a T5 model, what is the d_kv and d_model parameter? 

Is d_model the size of the embedding and in which case what is the d_kv? The docstring is not really clear for me.

Thanks for your help.

Comments:
I added a description of `d_kv`. `d_model` is as you say the size of the embedding. `d_kv` is the size of the key, query, value projections. 

If you look at this blog post: http://jalammar.github.io/illustrated-transformer/

`d_model` corresponds to the size of a vector `x_1` and `d_kv` of a vector `q_1, k_1, v_1`
Model card for t5-base-finetuned-emotion (recognition)
Author: mrm8488
Pull request: yes
Labels: name: model card


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5179?src=pr&el=h1) Report
> Merging [#5179](https://codecov.io/gh/huggingface/transformers/pull/5179?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/eb0ca71ef6772a1dd16ac152e1f7a07a9e1e6fda&el=desc) will **increase** coverage by `0.08%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/5179/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/5179?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #5179      +/-   ##
==========================================
+ Coverage   77.98%   78.06%   +0.08%     
==========================================
  Files         138      138              
  Lines       23710    23710              
==========================================
+ Hits        18490    18510      +20     
+ Misses       5220     5200      -20     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/5179?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/data/processors/squad.py](https://codecov.io/gh/huggingface/transformers/pull/5179/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9kYXRhL3Byb2Nlc3NvcnMvc3F1YWQucHk=) | `35.03% <0.00%> (+6.36%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/5179?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5179?src=pr&el=footer). Last update [eb0ca71...a3d99bd](https://codecov.io/gh/huggingface/transformers/pull/5179?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Cleaner warning when loading pretrained models
Author: thomwolf
Pull request: yes

Give more explicit logging messages when using the various `from_pretrained` methods in the lib.
Also makes these messages as `logging.warning` because it's a common source of silent mistakes.

cc @BramVanroy 

Happy to improve the language further if people have advice.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4557?src=pr&el=h1) Report
> Merging [#4557](https://codecov.io/gh/huggingface/transformers/pull/4557?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/a34a9896ac2a4a33ff9cd805c76eed914c8d8965&el=desc) will **increase** coverage by `0.00%`.
> The diff coverage is `86.95%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4557/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4557?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4557   +/-   ##
=======================================
  Coverage   77.87%   77.87%           
=======================================
  Files         123      123           
  Lines       20566    20580   +14     
=======================================
+ Hits        16016    16027   +11     
- Misses       4550     4553    +3     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4557?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_tf\_pytorch\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4557/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9weXRvcmNoX3V0aWxzLnB5) | `89.30% <78.57%> (-0.63%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4557/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `88.70% <100.00%> (+0.03%)` | :arrow_up: |
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4557/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `90.56% <100.00%> (+0.02%)` | :arrow_up: |
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4557/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `73.44% <0.00%> (-0.42%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4557?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4557?src=pr&el=footer). Last update [a34a989...0323876](https://codecov.io/gh/huggingface/transformers/pull/4557?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Good stuff!
Quick tour
Author: sgugger
Pull request: yes

This PR replaces the old quickstart guide by splitting it in two:
- a quick tour that is before the installation page,
- the philosophy page, after the installation.

It also renames Usage to Task summary since it's really what it is, and renames the files usage to task_summary, summary to model_summary.

The quick tour tries to go over all the API introduced in Transformers at a high level, pointing out tutorials (existing or upcoming) for each of them.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5145?src=pr&el=h1) Report
> Merging [#5145](https://codecov.io/gh/huggingface/transformers/pull/5145?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/f45e873910e60d89511ae0193711e71c5c710468&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/5145/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/5145?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #5145   +/-   ##
=======================================
  Coverage   77.19%   77.19%           
=======================================
  Files         133      133           
  Lines       22233    22233           
=======================================
  Hits        17163    17163           
  Misses       5070     5070           
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/5145?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5145/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `75.00% <0.00%> (-0.41%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5145/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `85.96% <0.00%> (+0.14%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/5145?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5145?src=pr&el=footer). Last update [f45e873...ae14c1a](https://codecov.io/gh/huggingface/transformers/pull/5145?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


That's awesome, I think this PR is really cool! I wonder whether A short section showing a super simple training example should already be in the quicktour

For the quick training example, I have it in mind but for when the integration with nlp is smooth: gathering the data takes a bit too long in code right now, and we have to talk about some obscure function that prepares it, which doesn't really fit the rest.
[Benchmark] Add optimization notebook
Author: pommedeterresautee
Pull request: no

# 🖥 Benchmarking `transformers`

## Benchmark

This notebook is about benchmarking model training with/without dynamic padding optimization. 
https://github.com/ELS-RD/transformers-notebook 

**Would it be possible to add it to the [community notebook](https://github.com/huggingface/transformers/tree/master/notebooks) list?** (a link to the Google collab version is provided) @julien-c @patrickvonplaten

## Set-up

GPU : Nvidia P100 provided by Google Collab

## Results

Using dynamic padding on MNLI provides a **4.7 times training time reduction**, with max pad length set to 512. The effect is strong because few examples are >> 400 tokens in this dataset. IRL, it will depend of the dataset, but it always bring improvement and, after more than 20 experiments listed in this [article](https://towardsdatascience.com/divide-hugging-face-transformers-training-time-by-2-or-more-21bf7129db9q-21bf7129db9e?source=friends_link&sk=10a45a0ace94b3255643d81b6475f409), it seems to not hurt performance.


Comments:
That'd be great! Do you want to open a PR? :-) 
I would use this line as the github line: https://github.com/ELS-RD/transformers-notebook/blob/master/Divide_Hugging_Face_Transformers_training_time_by_2_or_more.ipynb

and this one as the colab notebook line:
https://colab.research.google.com/drive/1CBfRU1zbfu7-ijiOqAAQUA-RJaxfcJoO?usp=sharing



Haha, I thought you were managing the page (special order of article or whatever)... :-)
So the PR is done and waiting for your validation.
Create README.md
Author: aodiniz
Pull request: yes
Labels: name: model card

Creating README.md file for model on Community contribution.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5152?src=pr&el=h1) Report
> Merging [#5152](https://codecov.io/gh/huggingface/transformers/pull/5152?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/5ed94b231269972c59d53bf4134a842c2273e814&el=desc) will **decrease** coverage by `0.38%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/5152/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/5152?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #5152      +/-   ##
==========================================
- Coverage   78.32%   77.93%   -0.39%     
==========================================
  Files         137      137              
  Lines       23472    23472              
==========================================
- Hits        18385    18294      -91     
- Misses       5087     5178      +91     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/5152?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/data/processors/squad.py](https://codecov.io/gh/huggingface/transformers/pull/5152/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9kYXRhL3Byb2Nlc3NvcnMvc3F1YWQucHk=) | `28.66% <0.00%> (-28.03%)` | :arrow_down: |
| [src/transformers/tokenization\_bert.py](https://codecov.io/gh/huggingface/transformers/pull/5152/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fYmVydC5weQ==) | `90.45% <0.00%> (-0.83%)` | :arrow_down: |
| [src/transformers/tokenization\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5152/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fdXRpbHMucHk=) | `94.81% <0.00%> (-0.38%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/5152?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5152?src=pr&el=footer). Last update [5ed94b2...9874049](https://codecov.io/gh/huggingface/transformers/pull/5152?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Create README.md
Author: aodiniz
Pull request: yes
Labels: name: model card

Creating README.md file for model on Community contribution.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5160?src=pr&el=h1) Report
> Merging [#5160](https://codecov.io/gh/huggingface/transformers/pull/5160?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/68e19f1c228c92d5d800533f558faff24b57127a&el=desc) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/5160/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/5160?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #5160      +/-   ##
==========================================
- Coverage   77.93%   77.92%   -0.01%     
==========================================
  Files         137      137              
  Lines       23475    23475              
==========================================
- Hits        18295    18294       -1     
- Misses       5180     5181       +1     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/5160?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5160/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `85.81% <0.00%> (-0.30%)` | :arrow_down: |
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5160/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `91.28% <0.00%> (+0.12%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/5160?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5160?src=pr&el=footer). Last update [68e19f1...2bf6959](https://codecov.io/gh/huggingface/transformers/pull/5160?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Passing in own embeddings for image input
Author: andrewlee98
Pull request: no

# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->

I am trying to use your pretrained BERT/RoBERTa models to write multimodal classifiers such as the bitransformer (see diagram at top of page 2):
https://arxiv.org/pdf/1909.02950.pdf

and Visual BERT:
https://arxiv.org/pdf/1908.03557.pdf

It would be nice to be able to change the input layer of the transformer models to handle custom input embeddings rather than just input_ids. If there is already a way to do this, it would be great to get some instructions on how. I have tried printing the layers of the pretrained RoBERTa model, but it only gives:
`<transformers.modeling_tf_roberta.TFRobertaMainLayer object at 0x7f79fc04dcf8>`

Edit: I actually see that there is a MMBT model mentioned in the documentation, but I cannot find the pre-trained model or any examples with it. Please help. Thank you.

## Motivation

<!-- Please outline the motivation for the proposal. Is your feature request
     related to a problem? e.g., I'm always frustrated when [...]. If this is related
     to another GitHub issue, please link here too. -->

I would like to be able to use your pretrained models to handle image embeddings as well. I am working on a project with this multimodal dataset: 
https://gombru.github.io/2019/10/09/MMHS/

Any help would be greatly appreciated. Thanks!






Comments:
Check out the `inputs_embeds` parameter.