noamaanMulla-03/transformers-issues huggingface dataset:
GPT2TokenizerFast object has no attribute 'with_pre_tokenizer'
Author: armheb
Pull request: no
Labels: name: wontfix

## 🐛 Bug

<!-- Important information -->

Model I am using (Bert, XLNet....): GPT2

Language I am using the model on (English, Chinese....): English

The problem arise when using:
tokenizer = GPT2TokenizerFast.from_pretrained('gpt2')

The tasks I am working on is:

* [ ] my own task or dataset:

## To Reproduce

Steps to reproduce the behavior:

1.tokenizer = GPT2TokenizerFast.from_pretrained('gpt2')
2.
3.

<!-- If you have a code sample, error messages, stack traces, please provide it here as well. --><ipython-input-97-b0a0cde738fe> in <module>
----> 1 tokenizer = GPT2TokenizerFast.from_pretrained('gpt2')

/media/bahram/New Volume/Projects/Python_LM/transformers/tokenization_utils.py in from_pretrained(cls, *inputs, **kwargs)
    307 
    308         """
--> 309         return cls._from_pretrained(*inputs, **kwargs)
    310 
    311     @classmethod

/media/bahram/New Volume/Projects/Python_LM/transformers/tokenization_utils.py in _from_pretrained(cls, pretrained_model_name_or_path, *init_inputs, **kwargs)
    459         # Instantiate tokenizer.
    460         try:
--> 461             tokenizer = cls(*init_inputs, **init_kwargs)
    462         except OSError:
    463             raise OSError(

/media/bahram/New Volume/Projects/Python_LM/transformers/tokenization_gpt2.py in __init__(self, vocab_file, merges_file, unk_token, bos_token, eos_token, pad_to_max_length, add_prefix_space, max_length, stride, truncation_strategy, **kwargs)

AttributeError: 'Tokenizer' object has no attribute 'with_pre_tokenizer'

## Expected behavior

<!-- A clear and concise description of what you expected to happen. -->
AttributeError: 'Tokenizer' object has no attribute 'with_pre_tokenizer'
## Environment

* OS: Ubuntu 18
* Python version: 3.7
* PyTorch version:
* PyTorch Transformers version (or branch):
* Using GPU yes
* Distributed or parallel setup No
* Any other relevant information:

## Additional context

<!-- Add any other context about the problem here. -->


Comments:
Hi, could you specify your versions of `tokenizers` and `transformers`? I believe you're running on an older version of `transformers`, could you install from source: `pip install git+https://github.com/huggingface/transformers` and let me know if it fixes this issue? Thank you.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Remove MD emojis
Author: mrm8488
Pull request: yes
Labels: name: model card


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4602?src=pr&el=h1) Report
> Merging [#4602](https://codecov.io/gh/huggingface/transformers/pull/4602?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/5ddd8d6531c8c49fdd281b55b93f6c81c9826f4b&el=desc) will **increase** coverage by `0.08%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4602/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4602?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #4602      +/-   ##
==========================================
+ Coverage   78.03%   78.11%   +0.08%     
==========================================
  Files         124      124              
  Lines       20647    20647              
==========================================
+ Hits        16111    16128      +17     
+ Misses       4536     4519      -17     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4602?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4602/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `73.44% <0.00%> (-0.42%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4602/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `88.66% <0.00%> (+0.16%)` | :arrow_up: |
| [src/transformers/data/processors/squad.py](https://codecov.io/gh/huggingface/transformers/pull/4602/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9kYXRhL3Byb2Nlc3NvcnMvc3F1YWQucHk=) | `34.07% <0.00%> (+5.41%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4602?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4602?src=pr&el=footer). Last update [5ddd8d6...a5049de](https://codecov.io/gh/huggingface/transformers/pull/4602?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Create README.md
Author: mrm8488
Pull request: yes
Labels: name: model card


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4591?src=pr&el=h1) Report
> Merging [#4591](https://codecov.io/gh/huggingface/transformers/pull/4591?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/b86e42e0ac1b59f21f0eccf351d3346bbe3ed4eb&el=desc) will **increase** coverage by `0.00%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4591/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4591?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4591   +/-   ##
=======================================
  Coverage   78.09%   78.09%           
=======================================
  Files         123      123           
  Lines       20624    20624           
=======================================
+ Hits        16106    16107    +1     
+ Misses       4518     4517    -1     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4591?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4591/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `88.66% <0.00%> (+0.16%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4591?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4591?src=pr&el=footer). Last update [b86e42e...1e39227](https://codecov.io/gh/huggingface/transformers/pull/4591?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Finetuning error: RuntimeError: Error(s) in loading state_dict for GPT2LMHeadModel: Missing key(s) in state_dict: “transformer.h.0.attn.masked_bias”, “transformer.h.1.attn.masked_bias”
Author: kiranikram
Pull request: no

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     
     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question 
     that you posted.
     -->

## Details
<!-- Description of your issue -->

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**:


Comments:
I think we're missing some info here, can you add your complete code snippet to the issue's title? 

### 🤣🤣 

You can even add some binary data in there for good measure.

Hehe I'm just kidding :trollface: 

Managed: I just didn’t account for the May 7 transformers release! 

Sent from my iPhone

> On 12 May 2020, at 2:08 PM, Julien Chaumond <notifications@github.com> wrote:
> 
> ﻿
> I think we're missing some info here, can you add your complete code snippet to the issue's title?
> 
> 🤣🤣
> 
> You can even add some binary data in there for good measure.
> 
> Hehe I'm just kidding 
> 
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub, or unsubscribe.


The same issue in Colab. I use DialoGPT repo but issue is the same

Help, please . Anyone. I can't load on other devices. I have copied the model from the device where it was trained on GPU, now I cannot load on other device with CPU.

It's hard to help with this much information. Can you open a new issue, respecting the template, letting us know what's your issue?

https://github.com/huggingface/transformers/commit/05deb52dc170226fcad899c240ee9568986d8d5a#diff-8530ad4c95b70580d10d594b7411a8be seemed to break this pretrained model, which was last modified on February 18.  There masked_bias was introduced.  Try reverting to a version before that, like cec3cdd.
❓ [BART] Different embedding sizes between pre-trained / fine-tuned checkpoint
Author: astariul
Pull request: no

# ❓ Questions & Help

Running this code :

```python
from transformers import BartModel

x = BartModel.from_pretrained('bart-large')
x2 = BartModel.from_pretrained('bart-large-cnn')
print(x.shared)
print(x2.shared)
```

Gives :

>Embedding(50265, 1024, padding_idx=1)
Embedding(50264, 1024, padding_idx=1)

---

Why the vocabulary size is different ? Isn't it supposed to be the same ? Is it just from the original authors' checkpoint ?  

@sshleifer 


Comments:
Good catch. There is no mask token in the second checkpoint. I believe that is the same as the authors' implementation.

Completely off topic: if you still have the xsum data you used I would love a copy. I'm sam [at] huggingface.co . 

Thanks for your fast answer ! 

Do you know why there is no mask token in the second checkpoint ? And if it has any impact on score ?

I have a hunch the there is no `<mask>` token because of fairseq's `--find-unused-parameters` clarg, but I'm not certain.

I would guess no impact on score because `<mask>` does not show up in the finetuning data.
AttributeError: 'Namespace' object has no attribute 'to_json_string'
Author: manhlab
Pull request: no

trainer.train()
don't know how to set parameters self.args.to_json_string()


Comments:
I'm not sure I understand what you're trying to do. Do you mind explaining? Showing the code you're using would help as well.

```
training_args = dict( num_cores=  8, model_name_or_path= 't5-base',
  max_len= 512 ,target_max_len= 2, output_dir= './models',
  overwrite_output_dir= True,
  per_gpu_train_batch_size= 8,
  per_gpu_eval_batch_size= 8,
  gradient_accumulation_steps= 4,
  learning_rate= 1e-4,
  tpu_num_cores= 8,
                logging_dir='/log',
  do_train= True, weight_decay=0.00,
                     device='xla',local_rank=-1,
                     max_steps=10000, adam_epsilon=1e-8,
    warmup_steps=0,
    train_batch_size=8,
    eval_batch_size=8,
    num_train_epochs=1,
    early_stop_callback=False,
    fp_16=False, # if you want to enable 16-bit training then install apex and set this to true
    opt_level='O1', # you can find out more on optimisation levels here https://nvidia.github.io/apex/amp.html#opt-levels-and-properties
    max_grad_norm=1.0, # if you enable 16-bit training then set this to a sensible value, 0.5 is a good default
    seed=42, fp16=False, n_gpu=0,SummaryWriter=None)
       from transformers import Trainer
trainer = Trainer(
        model=model,
        args=argparse.Namespace(**training_args),
        train_dataset=train_dataset,
        data_collator=T2TDataCollator(),
        prediction_loss_only=True, tb_writer=None
    )

trainer.train()

````

here is my code and i got 
AttributeError: 'Namespace' object has no attribute 'to_json_string'

Trainer's args should be a `TrainingArguments` instance, not a dict or a Namespace.

Try:
```python
from transformers import TrainingArguments
```
bert uncased models needs lower casing option turned on in the example code.
Author: rsanjaykamath
Pull request: yes

The command to replicate bert-base-uncased model and bert-large-uncased-whole-word-masking model results on Squad requires --do_lower_case to be used. If not, the result is low.

Comments:
Those models should contain config that initializes their tokenizer with the correct `do_lower_case=True` option. Is this not the case?

i.e. we are moving the tokenizer configuration out of those example scripts (see updated scripts for `Trainer` at https://github.com/huggingface/transformers/tree/master/examples).

I haven't gone through the recent update sorry. 
It looks like the [examples page](https://github.com/huggingface/transformers/tree/master/examples/) has completely changed now. 
But in the folder of [question-answering](https://github.com/huggingface/transformers/tree/master/examples/question-answering), the configuration to replicate results still remains the same. 

Shall I modify this there and create a request ? 
Because the  `do_lower_case` argument is a `store_true` argument and not a `boolean` with default as `true`. 

I spent 3 runs to realise the error because it worked correctly with cased models. 

Reference https://github.com/huggingface/transformers/blob/4658896ee105c504eba23db2efefc5fe14191b81/examples/question-answering/run_squad.py#L579




Closing this b/c #4245 was merged

(we still need to investigate why the lowercasing is not properly populated by the model's config)
why squad.py did not reproduce squad1.1 report result?
Author: yyHaker
Pull request: no
Labels: name: Migration

# 📚 Migration

## Information

<!-- Important information -->

Model I am using (Bert, XLNet ...):

Language I am using the model on (English...):

The problem arises when using:
* [x] the official example scripts: (give details below)
examples/question-answering/run_squad.py
* [x] my own modified scripts: (give details below)
'''
CUDA_VISIBLE_DEVICES=5 python examples/question-answering//run_squad.py \
    --model_type bert \
    --model_name_or_path bert-large-uncased-whole-word-masking \
    --do_train \
    --do_eval \
    --data_dir EKMRC/data/squad1.1 \
    --train_file train-v1.1.json \
    --predict_file dev-v1.1.json \
    --per_gpu_eval_batch_size=4   \
    --per_gpu_train_batch_size=4  \
    --gradient_accumulation_steps=6 \
    --save_steps 3682 \
    --learning_rate 3e-5 \
    --num_train_epochs 2 \
    --max_seq_length 384 \
    --doc_stride 128 \
    --output_dir result/debug_squad/wwm_uncased_bert_large_finetuned_squad/ \
    --overwrite_output_dir
'''

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: (give the name)

## Details
But I did not reproduce the result reported, the repository say get result bellow:
```
python $SQUAD_DIR/evaluate-v1.1.py $SQUAD_DIR/dev-v1.1.json ../models/wwm_uncased_finetuned_squad/predictions.json
{"exact_match": 86.91579943235573, "f1": 93.1532499015869}
```
my result is below:
```
python $SQUAD_DIR/evaluate-v1.1.py $SQUAD_DIR/dev-v1.1.json ../models/wwm_uncased_finetuned_squad/predictions.json
{"exact_match": 81.03, "f1": 88.02}
```

## Environment info
<!-- You can run the command `python transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
 
- `transformers` version:
- Platform: Linux gpu19 3.10.0-1062.4.1.el7.x86_64 #1 SMP Fri Oct 18 17:15:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
- Python version: python3.6
- PyTorch version (GPU?): 1.4.0
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: parallel

<!-- IMPORTANT: which version of the former library do you use? -->
* `pytorch-transformers` or `pytorch-pretrained-bert` version (or branch):
current version of transformers.

## Checklist

- [yes ] I have read the migration guide in the readme.
 ([pytorch-transformers](https://github.com/huggingface/transformers#migrating-from-pytorch-transformers-to-transformers);


Comments:
I have just solved this problem.
You have to set an additional flag: --do_lower_case.
I wonder why the run_squad.py behaves differently than run_glue.py, etc. Is there is a code improve on the way?

You shouldn't have to set `--do_lower_case` as it should be lowercased by default for that model.

> You shouldn't have to set `--do_lower_case` as it should be lowercased by default for that model.

I thought it is and it should be, but it isn't

Closing this b/c #4245 was merged

(we still need to investigate why the lowercasing is not properly populated by the model's config)
[testing] LanguageModelGenerationTests require_tf or require_torch
Author: sshleifer
Pull request: yes


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4616?src=pr&el=h1) Report
> Merging [#4616](https://codecov.io/gh/huggingface/transformers/pull/4616?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/a9aa7456ac824c9027385b149f405e4f5649273f&el=desc) will **increase** coverage by `0.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4616/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4616?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #4616      +/-   ##
==========================================
+ Coverage   78.02%   78.04%   +0.01%     
==========================================
  Files         124      124              
  Lines       20626    20626              
==========================================
+ Hits        16093    16097       +4     
+ Misses       4533     4529       -4     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4616?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4616/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `90.53% <0.00%> (+0.11%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4616/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `88.83% <0.00%> (+0.32%)` | :arrow_up: |
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4616/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `73.85% <0.00%> (+0.41%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4616?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4616?src=pr&el=footer). Last update [a9aa745...4d147b2](https://codecov.io/gh/huggingface/transformers/pull/4616?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Create README.md
Author: kldarek
Pull request: yes
Labels: name: model card

Model card for cased model

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4607?src=pr&el=h1) Report
> Merging [#4607](https://codecov.io/gh/huggingface/transformers/pull/4607?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/a9aa7456ac824c9027385b149f405e4f5649273f&el=desc) will **increase** coverage by `0.00%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4607/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4607?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4607   +/-   ##
=======================================
  Coverage   78.02%   78.02%           
=======================================
  Files         124      124           
  Lines       20626    20626           
=======================================
+ Hits        16093    16094    +1     
+ Misses       4533     4532    -1     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4607?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4607/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `88.66% <0.00%> (+0.16%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4607?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4607?src=pr&el=footer). Last update [a9aa745...efcdd13](https://codecov.io/gh/huggingface/transformers/pull/4607?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Great – [model page](https://huggingface.co/dkleczek/bert-base-polish-cased-v1)