noamaanMulla-03/transformers-issues huggingface dataset:
Only init encoder_attention_mask if stack is decoder
Author: rlouf
Pull request: yes

We currently initialize `encoder_attention_mask` when it is `None`,
whether the stack is that of an encoder or a decoder. Since this
may lead to bugs that are difficult to tracks down later, I added a condition
that assesses whether the current stack is a decoder.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/1770?src=pr&el=h1) Report
> Merging [#1770](https://codecov.io/gh/huggingface/transformers/pull/1770?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/1c542df7e554a2014051dd09becf60f157fed524?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `90%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/1770/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/transformers/pull/1770?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #1770      +/-   ##
==========================================
+ Coverage   84.03%   84.03%   +<.01%     
==========================================
  Files          94       94              
  Lines       14032    14034       +2     
==========================================
+ Hits        11792    11794       +2     
  Misses       2240     2240
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/1770?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [transformers/modeling\_bert.py](https://codecov.io/gh/huggingface/transformers/pull/1770/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL21vZGVsaW5nX2JlcnQucHk=) | `87.82% <90%> (+0.04%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/1770?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/1770?src=pr&el=footer). Last update [1c542df...cd286c2](https://codecov.io/gh/huggingface/transformers/pull/1770?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Ok great @rlouf!

I wonder why encoder_attention_mask use decoder inputs shape when it's not specified. 
`encoder_attention_mask = torch.ones(input_shape, device=device)`

Indeed, wanna have a look @rlouf?

That's a very good point, thanks @efeiefei ! See #2107
Encoding special tokens
Author: pglock
Pull request: no

## 🐛 Bug

<!-- Important information -->
In version 2.2.1 encoding special tokens changed.

```python
from transformers import BertTokenizer

tokenizer = BertTokenizer.from_pretrained("bert-base-uncased")
tokenizer.decode(tokenizer.encode("[CLS] hello world [SEP]", add_special_tokens=False))
```
output: `'[ cls ] hello world [ sep ]'`

For version `transformers==2.2.0` the output is: `'[CLS] hello world [SEP]'`




Comments:
I got the same issue for version 2.2.1. 

I also meet this issue and you may check out the possible root cause from #2052. 

My workaround is backoff to 2.1.1 version.


Should have been fixed with https://github.com/huggingface/transformers/pull/2051
Potential bug with gradient clipping when using gradient accumulation in examples
Author: Mathieu-Prouveur
Pull request: no
Labels: name: wontfix

## ❓ Questions & Help

Hi developpers,

Thanks for the awesome package. I have a question related to the recent major from pytorch_pretrained_bert to pytorch_transformers.

Gradient clipping used to be done inside the optimizer BertAdam and is now done at the same time as gradient computation in `run_squad.py` : https://github.com/huggingface/pytorch-transformers/blob/7729ef738161a0a182b172fcb7c351f6d2b9c50d/examples/run_squad.py#L156

It seems to me like the first accumulated gradients might get clipped several times hence giving more weight to last accumulated gradients : 

As an example of my thought here is what happens if we compare what we get if we clip to 1 at each accumulation step instead of at the end of the accumulation for the two gradients [2,0] and [0,2]:

```python
In [1]: import torch
   ...: from torch.nn.utils import clip_grad_norm_
   ...: from torch.autograd import Variable
   ...: 
   ...: x = Variable(torch.FloatTensor([[0],[0]]), requires_grad=True)
   ...: 
   ...: grad1 = torch.FloatTensor([[2],[0]])
   ...: grad2 = torch.FloatTensor([[0],[2]])
   ...: 
   ...: x.grad = grad1
   ...: clip_grad_norm_(x, 1)
   ...: print(x.grad)
   ...: 
   ...: x.grad += grad2
   ...: clip_grad_norm_(x, 1)
   ...: print(x.grad)
   ...: 
   ...: grad1 = torch.FloatTensor([[2],[0]])
   ...: grad2 = torch.FloatTensor([[0],[2]])
   ...: 
   ...: x.grad = grad1 + grad2
   ...: clip_grad_norm_(x, 1)
   ...: print(x.grad)
   ...: 
   ...: 
tensor([[1.0000],
        [0.0000]])
tensor([[0.4472],
        [0.8944]])
tensor([[0.7071],
        [0.7071]])```

We can see that clipping  at each step biased gradient towards the gradient of the second batch:
`python tensor([[0.4472], [0.8944]])` 
Instead of getting the balanced expected result : `python tensor([[0.7071], [0.7071]])` 

So either I missed something or I think the fix would be to simply move gradient clipping before the call to the optimizer

Comments:
Hi, indeed we could move the gradient clipping just before the call to the optimizer.
Do you want to send a PR to fix that on `run_squad` and `run_glue`?

Hi, was this ever implemented? I think it makes the most sense to clip right before an optimizer step. Right now it's implemented in two different ways in [run_lim_finetuning](https://github.com/huggingface/transformers/blob/master/examples/run_lm_finetuning.py) and [run_glue](https://github.com/huggingface/transformers/blob/master/examples/run_glue.py)

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Unable to load weights properly from tf checkpoint
Author: aavshr
Pull request: no
Labels: name: wontfix

The  function ``` load_tf_weights_in_bert ``` in ``` modeling_bert.py ``` is buggy and throws a lot of attribute errors because of what seems as the pointer pointing to the entire model.

For instance for the variable ```bert/encoder/layer_0/attention/output/dense/kernel ``` it throws an attribute error along the lines of ```Bert model has no attribute weight ``` because the pointer is the model ```bert``` itself whereas the pointer should be ```bert.encoder.layer.0.attention.output.dense```.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


any updates on this?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
create encoder attention mask from shape of hidden states
Author: rlouf
Pull request: yes

As noted by @efeiefei (#1770) we currently create masks on the encoder hidden states (when they're not provided) based on the shape of the inputs to the decoder. This is obviously wrong; sequences can be of different lengths. We now create the encoder attention mask based on the `batch_size` and `sequence_length` of the encoder hidden states.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/2107?src=pr&el=h1) Report
> Merging [#2107](https://codecov.io/gh/huggingface/transformers/pull/2107?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/0cb163865a4c761c226b151283309eedb2b1ca4d?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/2107/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/transformers/pull/2107?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #2107      +/-   ##
==========================================
+ Coverage   82.67%   82.67%   +<.01%     
==========================================
  Files         111      111              
  Lines       16162    16164       +2     
==========================================
+ Hits        13362    13364       +2     
  Misses       2800     2800
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/2107?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [transformers/modeling\_bert.py](https://codecov.io/gh/huggingface/transformers/pull/2107/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL21vZGVsaW5nX2JlcnQucHk=) | `87.72% <100%> (+0.04%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/2107?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/2107?src=pr&el=footer). Last update [0cb1638...3520be7](https://codecov.io/gh/huggingface/transformers/pull/2107?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


👍
How to fine tune small dataset?
Author: g-jing
Pull request: no
Labels: name: wontfix

## ❓ Questions & Help

<!-- A clear and concise description of the question. -->
Most people test bert on large dataset, but when it comes to small dataset, I assume the fine tune process and batch size maybe different. Besides, the dataset domain is twitter domain, which is kind of different from BERT pretrained corpus.
Could anyone gives some suggestions on finetuning BERT on small twitter dataset? Thanks in advance for any help.

Comments:
Usually, you should train more epochs. Optimal batch size from 8 to 16. I'm not sure but maybe learning rate should be lower in this case. You can try

Thanks for your suggestion. I will try that. Besides, do you think I should modify the warmup step or just set it as 10% of total step just like in the original paper? 

Influence of warmup steps is not clear for me. it looks like it's not so important for final quality but can speed up training a bit

@avostryakov  Thanks for your help!

@avostryakov  I wonder if I could ask another question. I know in BERT, weight decay of some layer is set to be 0 while others are set to be 0.01. So how to set the weight decay for other layer like the linear layer after bert output in finetuning?  

First of all, I'm not sure that weight decay is really important. I tried 0.1 and 0.01 for RoBERTa without a difference in quality. But ok, you can look in run_glue.py for this code:

```
no_decay = ['bias', 'LayerNorm.weight']
    optimizer_grouped_parameters = [
        {'params': [p for n, p in model.named_parameters() if not any(nd in n for nd in no_decay)], 'weight_decay': args.weight_decay},
        {'params': [p for n, p in model.named_parameters() if any(nd in n for nd in no_decay)], 'weight_decay': 0.0}
        ]
    optimizer = AdamW(optimizer_grouped_parameters, lr=args.learning_rate, eps=args.adam_epsilon)```

As you can see you can set any decay weight for any parameter or layer, you need to know it's name.

@avostryakov  Thanks very much, maybe the learning rate is more important than weight decay 

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Hey @RoderickGu, Do you have any intuition now? If so could you share them with public?  

@ereday  First I think you should follow the most methods in BERT fine-tune process, such as use adamw. Besides, you could use a small batch size. Since dataset is small, I also suggested you to run it several times for the best learning rate. Hope this helps.
Running run_lm_finetuning.py within python
Author: aclifton314
Pull request: no

## Setup
* Model: roberta-base
* Language: english
* OS: Ubuntu 18.04.3
* Python version: 3.7.3
* PyTorch version: 1.3.1+cpu
* PyTorch Transformers version (or branch): 2.2.0
* Using GPU ? No
* Distributed of parallel setup ? No
* Script inputs:
```
python run_lm_finetuning.py \
--output_dir=$OUTPUT_DIR \
--model_type=roberta \
--model_name_or_path=roberta_base \
--do_train \
--train_data_file=$TRAIN_FILE \
--do_eval \
--eval_data_file=$TEST_FILE \
--mlm \
--no_cuda
```
## ❓ Questions & Help
Is there a way to run the above within python? Said differently, if I want to call `run_lm_finetuning.py` from within one of my own python scripts using the above configurations, how would I best go about doing that?

Thanks in advance!

Comments:
You have two choices:
- transform the code into `run_lm_finetuning.py` as Python functions and use them (I think it's **the most elegant solution**). In order to do that, you've to convert the `main` method with `argparse` arguments to a method without `argparse` and after that you can use the script as given
- call `run_lm_finetuning.py` from your Python script with e.g. [subprocess](https://docs.python.org/3.6/library/subprocess.html)

> ## Setup
> * Model: roberta-base
> * Language: english
> * OS: Ubuntu 18.04.3
> * Python version: 3.7.3
> * PyTorch version: 1.3.1+cpu
> * PyTorch Transformers version (or branch): 2.2.0
> * Using GPU ? No
> * Distributed of parallel setup ? No
> * Script inputs:
> 
> ```
> python run_lm_finetuning.py \
> --output_dir=$OUTPUT_DIR \
> --model_type=roberta \
> --model_name_or_path=roberta_base \
> --do_train \
> --train_data_file=$TRAIN_FILE \
> --do_eval \
> --eval_data_file=$TEST_FILE \
> --mlm \
> --no_cuda
> ```
> 
> ##  Questions & Help
> Is there a way to run the above within python? Said differently, if I want to call `run_lm_finetuning.py` from within one of my own python scripts using the above configurations, how would I best go about doing that?
> 
> Thanks in advance!

@TheEdoardo93 roger that. Thanks!
Remove misplaced summarization documentation
Author: rlouf
Pull request: yes

Documentation for the previous version of abstractive summarization is still present in the repository:

https://twitter.com/DavidMezzetti/status/1204123548966621184

This PR removes it.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/2122?src=pr&el=h1) Report
> Merging [#2122](https://codecov.io/gh/huggingface/transformers/pull/2122?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/e57d00ee108595375504eb21c230ce35428aae5e?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/2122/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/transformers/pull/2122?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #2122   +/-   ##
=======================================
  Coverage   80.08%   80.08%           
=======================================
  Files         112      112           
  Lines       16862    16862           
=======================================
  Hits        13504    13504           
  Misses       3358     3358
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/2122?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/2122?src=pr&el=footer). Last update [e57d00e...a5fa0de](https://codecov.io/gh/huggingface/transformers/pull/2122?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Tokenization in quickstart guide fails
Author: yenicelik
Pull request: no

## 🐛 Bug

<!-- Important information -->

The same issue as in #226 re-appears in transformers==2.2.1 (it works on 2.1!)

I just encountered the same issue as @dhirajmadan1 with `transformers==2.2.1`. Is this expected somehow?

I am following the quickstart guide: https://huggingface.co/transformers/quickstart.html


## To Reproduce

Steps to reproduce the behavior:

```
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')

# Run an example text through this:
text = "[CLS] Who was Jim Henson ? [SEP] Jim Henson was a puppeteer [SEP]"
tokenized_text = tokenizer.tokenize(text)

masked_index = 8
tokenized_text[masked_index] = '[MASK]'
predicted_tokenized_sentence = ['[CLS]', 'who', 'was', 'jim', 'henson', '?', '[SEP]', 'jim', '[MASK]', 'was', 'a', 'puppet', '##eer', '[SEP]']
        
```

## Expected behavior

This should not fail:

```assert tokenized_text == ['[CLS]', 'who', 'was', 'jim', 'henson', '?', '[SEP]', 'jim', '[MASK]', 'was', 'a', 'puppet', '##eer', '[SEP]']```


## Environment

* OS: Mac
* Python version: 3.6
* PyTorch version: 1.3.1
* PyTorch Transformers version (or branch): 2.1 (latest-minor)
* Using GPU no
* Distributed of parallel setup no
* Any other relevant information:

## Additional context

<!-- Add any other context about the problem here. -->


Comments:
Oops, that appears to be my fault. Should be a quick fix though, so I'll try to make a PR on it right away. Sorry about that! :grimacing: 

Thanks man! :) yeah no worries, thought it may be a good idea to report haha
`bert-base-uncased` tokenizer broke around special tokens in v2.2.1
Author: dirkgr
Pull request: no

In `v2.2.1`, the `bert-base-uncased` tokenizer changed in a way that's probably not intentional:

```
Python 3.7.5 (default, Oct 25 2019, 10:52:18)
[Clang 4.0.1 (tags/RELEASE_401/final)] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from transformers.tokenization_auto import AutoTokenizer
To use data.metrics please install scikit-learn. See https://scikit-learn.org/stable/index.html
Neither PyTorch nor TensorFlow >= 2.0 have been found.Models won't be available and only tokenizers, configurationand file/data utilities can be used.
>>> t = AutoTokenizer.from_pretrained("bert-base-uncased"); t.encode_plus(text='A, [MASK] AllenNLP sentence.')
{
    'input_ids': [101, 1037, 1010, 1031, 7308, 1033, 5297, 20554, 2361, 6251, 1012, 102],
    'token_type_ids': [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
}
```

In `v2.2.0`:
```
Python 3.7.5 (default, Oct 25 2019, 10:52:18)
[Clang 4.0.1 (tags/RELEASE_401/final)] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from transformers.tokenization_auto import AutoTokenizer
To use data.metrics please install scikit-learn. See https://scikit-learn.org/stable/index.html
Neither PyTorch nor TensorFlow >= 2.0 have been found.Models won't be available and only tokenizers, configurationand file/data utilities can be used.
>>> t = AutoTokenizer.from_pretrained("bert-base-uncased"); t.encode_plus(text='A, [MASK] AllenNLP sentence.')
{
    'special_tokens_mask': [1, 0, 0, 0, 0, 0, 0, 0, 0, 1],
    'input_ids': [101, 1037, 1010, 103, 5297, 20554, 2361, 6251, 1012, 102],
    'token_type_ids': [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
}
```
(indented the results for clarity)

The key difference is that in `v2.2.0`, it recognizes the `[MASK]` token as a special token and gives it token id `103`. In `v2.2.1`, this no longer happens. The behavior of `bert-base-cased` has not changed, so I don't think this is an intentional change.



Comments:
`git bisect` says the commit introducing this problem is 7246d3c2f93c4461f3ec8ada7a26a002d8f196ea.

Any way you could run the same test on `master`? It might have been fixed since.

I did. It was not fixed in master.

It only affects the [MASK] token.

On Tue, Dec 10, 2019, 16:25 Julien Chaumond <notifications@github.com>
wrote:

> Any way you could run the same test on master? It might have been fixed
> since.
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/2132?email_source=notifications&email_token=AAHAYPUUEHE5JTMYXUYYXHLQYAXQHA5CNFSM4JZGP2YKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGRNYOY#issuecomment-564321339>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAHAYPRQDOQTJNFD6ULI4JDQYAXQHANCNFSM4JZGP2YA>
> .
>


I screwed up. It is fixed in `master` after all.

Good to hear! We'll push a new release soon, cc @LysandreJik
"Write With Transformer" interface returning 502 on gpt2/xl model
Author: scottlingran
Pull request: no

## 🐛 Bug

The "Write With Transformer" interface is returning a `502` when the API calls the gpt2/xl model.

See: https://transformer.huggingface.co/doc/gpt2-xl

## To Reproduce

Steps to reproduce the behavior just using the API request:

```
curl 'https://transformer.huggingface.co/autocomplete/gpt2/xl' --data-binary '{"context":"See how a modern ","model_size":"gpt2/xl","top_p":0.9,"temperature":1,"max_time":1}' --compressed
```

## Expected behavior

Expecting autocomplete results, but getting this `502` response instead.
```
<html>
<head><title>502 Bad Gateway</title></head>
<body bgcolor="white">
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx/1.14.2</center>
</body>
```



Comments:
is the tokenization broken for bert?
Author: AdityaSoni19031997
Pull request: no

## 🐛 Bug

<!-- Important information -->

Model I am using is `bert-base-uncased`:

Language I am using the model on (English):

## To Reproduce

Steps to reproduce the behavior:

1. Just Ran the example from the docs

```
import torch
from transformers import BertTokenizer, BertModel, BertForMaskedLM

# OPTIONAL: if you want to have more information on what's happening under the hood, activate the logger as follows
import logging
logging.basicConfig(level=logging.INFO)

# Load pre-trained model tokenizer (vocabulary)
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased', force_download=True)

# Tokenize input
text = "[CLS] Who was Jim Henson ? [SEP] Jim Henson was a puppeteer [SEP]"
tokenized_text = tokenizer.tokenize(text)

# Mask a token that we will try to predict back with `BertForMaskedLM`
masked_index = 8
tokenized_text[masked_index] = '[MASK]'
print(tokenized_text)
assert tokenized_text == ['[CLS]', 'who', 'was', 'jim', 'henson', '?', '[SEP]', 'jim', '[MASK]', 'was', 'a', 'puppet', '##eer', '[SEP]']

>INFO:transformers.tokenization_utils:loading file https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-uncased-vocab.txt from cache at /Users/1570137/.cache/torch/transformers/26bc1ad6c0ac742e9b52263248f6d0f00068293b33709fae12320c0e35ccfbbb.542ce4285a40d23a559526243235df47c5f75c197f04f37d1a0c124c32c9a084
```

The assertion fails and the `print(tokenized_text)` returns this actually,

`['[', 'cl', '##s', ']', 'who', 'was', 'jim', 'henson', '[MASK]', '[', 'sep', ']', 'jim', 'henson', 'was', 'a', 'puppet', '##eer', '[', 'sep', ']']`

## Extra Details

>! pip show transformers
```
Name: transformers
Version: 2.2.1
Summary: State-of-the-art Natural Language Processing for TensorFlow 2.0 and PyTorch
Home-page: https://github.com/huggingface/transformers
Author: Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Google AI Language Team Authors, Open AI team Authors, Facebook AI Authors, Carnegie Mellon University Authors
Author-email: thomas@huggingface.co
License: Apache
Location: /Users/1570137/anaconda3/envs/my_env/lib/python3.7/site-packages
Requires: sacremoses, numpy, tqdm, sentencepiece, regex, requests, boto3
Required-by: flair
```

I am on MacOS, No GPU.
Also is that behaviour expected?
Thanks.

Comments:
This should be fixed in the current master but not in a release AFAIK. See https://github.com/huggingface/transformers/issues/2132 and close this issue please.

Okay thanks!