noamaanMulla-03/transformers-issues huggingface dataset:
Can't down models from huggingface.cn!
Author: havetry
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version:
- Platform:
- Python version:
- PyTorch version (GPU?):
- Tensorflow version (GPU?):
- Using GPU in script?:
- Using distributed or parallel set-up in script?:

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.
 
 albert, bert, GPT2, XLM: @LysandreJik 
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 TextGeneration: @TevenLeScao 
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten @TevenLeScao
 Blenderbot: @patrickvonplaten
 Bart: @patrickvonplaten
 Marian: @patrickvonplaten
 Pegasus: @patrickvonplaten
 mBART: @patrickvonplaten
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao
 RAG: @patrickvonplaten, @lhoestq
 FSMT: @stas00
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->

## Information

Model I am using (Bert, XLNet ...):

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1.
2.
3.

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->
I want to down model from huggingface.cn, but I can't find the models to down.
![down](https://user-images.githubusercontent.com/49902228/98777922-8bf23780-242c-11eb-90b9-02d86cefac34.png)
But I found the model had been download many times in noverber 9. Something happened I didn't know about? And what should I do to get models that I need?
![have_down](https://user-images.githubusercontent.com/49902228/98778321-23578a80-242d-11eb-9503-c46f3cde25f3.png)


Comments:
Duplicate of #8449
login to huggingface forum
Author: srulikbd
Pull request: no

hey.
I'm not sure where to post it, so my apologise in advance.

I'm trying to login to huggingface forum, but it return me the message: "Unauthorized".
what can I do?
(I created a user..)

thanks.

Comments:
Maybe @julien-c or @Pierrci knows!

can you try again, just in case it was an intermittent issue?

Wonderful, i login with another email and it works. Thanks.
multiple hard-coded paths in transformers/file_utils.py
Author: rabeehkarimimahabadi
Pull request: no

Hi
I need to run the codes on a machine not having access to internet, I am running finetune_trainer.py on a dataset from datasets repo, due to these hardcoded paths I cannot get the code running without access to internet, could you please make any hard-coded path a parameter, so if user do not have access to internet, they could download the data and set the paths? here is the full path to the file I am mentioning 

/idiap/user/rkarimi/libs/anaconda3/envs/internship/lib/python3.7/site-packages/datasets/utils/file_utils.py

thanks.
Best
Rabeeh 

Comments:
will report this in datasets repo.
Replaced unnecessary iadd operations on lists in tokenization_utils.py with proper list methods
Author: bombs-kim
Pull request: yes

# Replaced unnecessary iadd operations on lists in tokenization_utils.py with proper list methods

@mfuntowicz

Previously, unnecessarily many list objects are created because of list updates through iadd operations.
This is bad for the following reasons.
* It slows down the program.
* It's a substandard style.

Regarding the slowing down, please see the following snippets.
```
l = []
for i in range(10**6):
    l.append(i)
# Takes 0.13282 seconds, on average, on my machine
```

```
l = []
for i in range(10**6):
    l += [i]  # this creates a new list [i] every iteration
# Takes 0.14698 seconds, on average, on my machine
```

The previous style is considered bad since it's confusing. It is easy to think that `l += [i]` has the same semantics as `l = l + [i]`, which is not at all the case. To see this, run the following code.
```
 l = []
for i in range(10**6):
    l = l + [i]  # This replaces the existing list with a new list (l + [i]) every iteration
```

The fact that the existing list is mutated is more clearly expressed in the new code, and, to my best knowledge, all the python standard library code prefer the style of the new code.


Comments:
Thanks for the quick reviews both of you!

@LysandreJik 
I just searched the whole code and I found many more list append operations than updates through `__iadd__`. So I actually think that for consistency over the whole project scope, it may be better to use list append whenever it's applicable. If you approve, I would like to fix the other similar cases, too.

Regarding tuples, the current style seems perfectly okay. Tuples are immutable and `__iadd__` does not update existing tuple but replace it with a newly created one, i.e., `tup += ('a',)` is equivalent to `tup = tup + ('a',)`. So they should be treated differently from lists.

Here is a demonstration that shows the different effects of `__iadd__` on lists and tuples.
```
l = ['a']
print(id(l))  # 4327718256
l += ['b']
print(id(l))  # 4327718256


tup = ('a',)
print(id(tup)) # 4326480784
tup += ('b',)
print(id(l))   # 4327718256  (ID changed!)
```
trainer.evaluate returns 'epoch' from training
Author: PhilipMay
Pull request: no

I am training a BERT model: `trainer.train()`

Then I call `evaluate_result = trainer.evaluate(labeled_dataset_test)`

The value of `evaluate_result` looks like this:

```python
{'eval_loss': 0.5908029079437256, 
'eval_acc': 0.8282828282828283, 
'eval_bac': 0.8243021346469622, 
'eval_mcc': 0.7422526698197041, 
'eval_f1_macro': 0.826792009400705, 
'epoch': 3.0, 
'total_flos': 1373653507542624}
```

IMO the dict should not contain `'epoch': 3.0,`. That is the number of epochs from training. It has nothing to do with evaluation...

Comments:
You can easily  ignore that value though.

The problem is that you won't have it at each eval during the training loop if we don't include it. There could be something smarter there, but it would take time for something that is just purely cosmetic.

I added a small PR for improved documentation about this: #8273

Closing this since PR was merged.
`do_predict` option of `TrainingArguments` - but no way to pass test set.
Author: PhilipMay
Pull request: no

The `TrainingArguments` class has the option to pass `do_predict=True`. The doc sais: "Whether to run predictions on the test set or not."

But there is no way to pass a test set to the trainer. At least I can not find it in the documentation...

Can you please clarify / fix this?

Many thanks
Philip

Comments:
The `do_predict` argument (like `do-train` and `do-eval` is not used by `Trainer`), just by the training scripts provided as examples.

Getting predictions on a test set is done with `trainer.predict(test_dataset)`.

Should corresponding documentation be added?

Sure, do you want to take a stab at it?

@sgugger I can do a PR if you want. But...

... for me it smells like a design flaw when this is only for CLI usage and has no meaning for the "normal use".
Should we consider just removing it?

How would a documentation look like? _"This field is just a workaround for CLI value storage for the example code and has no meaning for normal usage."_?

It's not `TrainerArguments` but `TrainingArguments`, so I don't see the problem with some of those arguments being only for CLI usage. Besides, removing them would break existing code so it would do more harm than good IMO.

For the documentation itself, it's not just a workaround. Something along the line of
```
This argument is not directly used by :class:`~transformers.Trainer`, it's intended to be used by your training/evaluation scripts instead. See the `example scripts <https://github.com/huggingface/transformers/tree/master/examples>`___ for more details.
```
would sound better.

> ```
> This argument is not directly used by :class:`~transformers.Trainer`, it's intended to be used by your training/evaluation scripts instead. See the `example scripts <https://github.com/huggingface/transformers/tree/master/examples>`___ for more details.
> ```
> 
> would sound better.

PR has been created: #8270

closing this since PR was merged
Text-to-SQL Query
Author: thiagomoeng
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help

Hello everyone, I got a task that I want to use NLP for convert text into a SQL query. Anyone knows how to do this or got any suggestion? Thanks.

Comments:
pinging @mrm8488 .

You can try this model: https://huggingface.co/mrm8488/t5-base-finetuned-wikiSQL
Thanks for ping me @patil-suraj 

If you go to the model hub and type 'SQL', you get [this](https://huggingface.co/models?search=Sql). There are currently 3 variants of the T5 model fine-tuned on WikiSQL (a large dataset that contains sentence - SQL pairs).

Hi @mrm8488  I am having some issue with "torch_xla" module, is there any way to run this model on local windows, without this TPU module? Thanks.

hi @thiagomoeng, can you try setting the `xla_device` argument in `config.json` to `False`.

```python3
config = T5Config.from_pretrained("mrm8488/t5-base-finetuned-wikiSQL")
config.xla_device = False
model = T5ForConditionalGeneration.from_pretrained("mrm8488/t5-base-finetuned-wikiSQL", config=config)

Ofc, I can but the basic structure band the most of the code is from the public colab about fine tuning T5 on TPU by @patil-suraj 

Hi @mrm8488  I got a SQL data on Oracle, do you can give me some way of how to prepare this sql data to finetune on your model? I am beginner on training models.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


@thiagomoeng  if you want to convert natural language to sql , here is one implementation : https://github.com/abhijithneilabraham/tableQA

Drop me any comments on the slack channel in the readme there.
Fine tuning a classification model with engineered features
Author: rogeriobromfman
Pull request: no

# 🚀 Feature request

When fine-tuning a BERT model for text classification, it would be useful to be able to add some engineered features to improve accuracy. 

## Motivation

For example: 

- if there are any dates on the text (based on NER); 
- if the text starts with a punctuation;
- if the font size is larger or smaller than the rest of the document, etc.

These things can help the model make better predictions as to the class of the text.

## Contribution

At the moment, I've been adding these features as custom tokens to the end of the text. E.g "\<DATES\>", "\<PUNCT\>"


Comments:
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discuss.huggingface.co) instead?

Thanks!
automodel
Author: RochelleChoenni
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version:
- Platform:
- Python version:
- PyTorch version (GPU?):
- Tensorflow version (GPU?):
- Using GPU in script?:
- Using distributed or parallel set-up in script?:

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.
 
 albert, bert, GPT2, XLM: @LysandreJik 
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 TextGeneration: @TevenLeScao 
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten @TevenLeScao
 Blenderbot: @patrickvonplaten
 Bart: @patrickvonplaten
 Marian: @patrickvonplaten
 Pegasus: @patrickvonplaten
 mBART: @patrickvonplaten
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao
 RAG: @patrickvonplaten, @lhoestq
 FSMT: @stas00
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->

## Information

Model I am using (Bert, XLNet ...):

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1.
2.
3.

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->


Comments:
Support for SuperGLUE fine-tune/eval?
Author: julien-c
Pull request: no
Labels: name: wontfix

## 🚀 Feature

https://super.gluebenchmark.com/

Current canonical implem is https://github.com/nyu-mll/jiant/

## Motivation

https://twitter.com/_florianmai/status/1177489945918722050



Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


So is HuggingFace going to write the finetuning implementation for SuperGlue?

Hi @jiachangliu, did you have any news about support for superglue?

> Hi @jiachangliu, did you have any news about support for superglue?

No I have not heard any HugginFace support on SuperGlue. It was not urgent for me to run those experiments. However, if you want to run SuperGlue, I guess you need to install JIANT, which uses the model structures built by HuggingFace.

> > Hi @jiachangliu, did you have any news about support for superglue?
> 
> No I have not heard any HugginFace support on SuperGlue. It was not urgent for me to run those experiments. However, if you want to run SuperGlue, I guess you need to install JIANT, which uses the model structures built by HuggingFace.

Thank you !!
Example NER script predicts on tokenized dataset
Author: sarnoult
Pull request: yes

The new run_ner.py script (relying on datasets) tries to run prediction on the input test set `datasets["test"]`, but it should really input the tokenized set `tokenized_datasets["test"]`

# What does this PR do?
Fix an error with run_ner.py at the prediction step on a custom dataset 



<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSTM: @stas00
 -->


Comments:
Fix typo in roberta-base-squad2-v2 model card
Author: antoniolanza1996
Pull request: yes
Labels: name: model card

# What does this PR do?
Simply adding `-v2` for Haystack API model loading. Furthermore, I've also changed `model` in `model_name_or_path` due to breaking change in Haystack (https://github.com/deepset-ai/haystack/pull/510).

## Who can review?

 Model Cards: @julien-c 

Comments:
I meet the zero gradient descent
Author: Sniper970119
Pull request: no

I want use transformers to do  text classification, I want code myself rather than use `TFBertForSequenceClassification`,so I write the model with `TFBertModel` and `tf.keras.laters.Dense`,but this is no gradient descent in my code, I try to find what wrong with my code but I can't. So I submit this issues to ask for some help.
my code is here:

Model:
![图片](https://user-images.githubusercontent.com/30463691/98935026-e15a4180-251d-11eb-87a5-0d23bcb7cf4a.png)

![图片](https://user-images.githubusercontent.com/30463691/98935038-e4edc880-251d-11eb-9c3d-ee32df987590.png)


and I know train data is test data,just for quick debug.
and when I train this model , 
![图片](https://user-images.githubusercontent.com/30463691/98934626-5ed18200-251d-11eb-9eb3-c0330a6398bf.png)



Comments:
Hi @Sniper970119 do you mind posting this on the forum rather? It's here: https://discuss.huggingface.co

We are trying to focus the issues on bug reports and features/model requests.

Thanks a lot.

> 
> 
> Hi @Sniper970119 do you mind posting this on the forum rather? It's here: https://discuss.huggingface.co
> 
> We are trying to focus the issues on bug reports and features/model requests.
> 
> Thanks a lot.

ok,I just post it on the forum.Thank for your reply.
CUDA out of memory (ALBERT)!!
Author: ppyu
Pull request: no

# ❓ Questions & Help
While using `albert-base-v2` to train my model, I got the following problem:

```
#first call
outputs = self.albert(input_ids,
                              attention_mask=attention_mask,
                              token_type_ids=token_type_ids,
                              position_ids=position_ids,
                              head_mask=head_mask,
                              inputs_embeds=inputs_embeds)

sequence_output = outputs[0]

context_mask = token_type_ids * attention_mask
question_mask = ((1 - context_mask) * attention_mask)

#second call
question, _ = self.albert(input_ids, attention_mask=question_mask, token_type_ids=token_type_ids)

#third call
context, _ = self.albert(input_ids, attention_mask=context_mask, token_type_ids=token_type_ids)
```

While calling `self.albert()` thrice, the memory it consumes will multiply by 3.
So that I must change my batch_size to 4, it's so bad!
Is it a BUG or a feature?
Even `albert-base` so large?


Comments:
Hi there. Questions like this should be asked on the forum. In your code example, you are using different variable names for each of your three call, so it's logical to get more memory consumption. Python will only release the memory if you reuse the same variable name.

Hi,friends.
`you are using different variable names for each of your three call`
what do you mean? 
My problem is that:
when I first call `outputs = self.albert()` , the memory is almost 4G
And I second call `question, _ = self.albert()` , the memory increase to almost 8G.
But the variable `question` should not take up so much memory.

> Hi there. Questions like this should be asked on the forum. In your code example, you are using different variable names for each of your three call, so it's logical to get more memory consumption. Python will only release the memory if you reuse the same variable name.

Hi,friends.
`you are using different variable names for each of your three call`
what do you mean? 
My problem is that:
when I first call `outputs = self.albert()` , the memory is almost 4G
And I second call `question, _ = self.albert()` , the memory increase to almost 8G.
But the variable `question` should not take up so much memory.

Agree with @sgugger. Question like this should be asked on https://discuss.huggingface.co

We are tying to keep the issues for bug reports and new features/model requests.

Closing this for now.
Question template
Author: sgugger
Pull request: yes

# What does this PR do?

This PR updates the question template to insist a bit more on users using the forum for questions.

Comments:
<img width="1402" alt="image" src="https://user-images.githubusercontent.com/7353373/98936078-c3bcb600-24e4-11eb-9ae3-6894af553004.png">

For some reason I don't see the "Question and Help" option when I try to open an issue @sgugger @LysandreJik 

Mmm, guess the metadata at the top doesn't like the link.
Fix on "examples/language-modeling" to support more datasets
Author: zeyuyun1
Pull request: yes

# What does this PR do?

Fix on "run_clm.py", "run_mlm.py",  "run_plm.py", so that they can support datasets with more than one features. Before they will fail on datasets with more than one features. 

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).


## Who can review?

 @sgugger


Comments:
Great fix, thanks!