noamaanMulla-03/transformers-issues huggingface dataset:
support batch-sampler in trainer 
Author: dorooddorood606
Pull request: no

# 🚀 Feature request

Hi
Currently Trainer class  only supports sampler, but not batch-sampler, (the sampling strategies based on batches, which are a group of samplers in torch), if a user wants to use batch-sampler, currently Trainer class introduces unwanted bugs, but not setting epoch for this type of samplers.  If a user is careful enough, he/she still need to overwrite the whole ```train``` function, which is a big chunk of code, could you make this line a function, so the user easily overwrite it for the case of batch-sampler? This part of trainer also introduces bugs specially, if a user use user-defined samplers as well, and not being careful to set epoch for them. 

```
 if isinstance(train_dataloader.sampler, DistributedSampler):
                    train_dataloader.sampler.set_epoch(epoch)
```               
so the user might need to set it to:

```
 train_dataloader.batch_sampler.set_epoch(epoch)
```

thanks 


## Motivation

supporting all type of samplers in Trainer, or making set_epoch a function so user can overwrite it nicely 

@sgugger 


Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
cannot import name 'set_seed' from 'transformers'
Author: andy311p
Pull request: no

i run with transformers==4.5.1 and get the following error:
![image](https://user-images.githubusercontent.com/68938613/116115472-18cf3b80-a6c3-11eb-9a2d-f3c9ad1d5588.png)

do you know how to resolve this issue?

thanks

Comments:
Try the following code:
`from transformers.trainer_utils import set_seed`
Let me know if it works!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
梯度爆炸问题
Author: JiTingyu
Pull request: no

我在运行使用apex后用bert进行微调训练,最后报错提示"No module named 'amp_C'",并且还会梯度爆炸,请问是什么原因,该如何解决呢?
错误如下:

Selected optimization level O1: Insert automatic casts around Pytorch functions and Tensor methods.
Defaults for this optimization level are:
enabled : True
opt_level : O1
cast_model_type : None
patch_torch_functions : True
keep_batchnorm_fp32 : None
master_weights : None
loss_scale : dynamic
Processing user overrides (additional kwargs that are not None)...
After processing overrides, optimization options are:
enabled : True
opt_level : O1
cast_model_type : None
patch_torch_functions : True
keep_batchnorm_fp32 : None
master_weights : None
loss_scale : dynamic
Warning: multi_tensor_applier fused unscale kernel is unavailable, possibly because apex was installed without --cuda_ext --cpp_ext. Using Python fallback. Original ImportError was: ModuleNotFoundError("No module named 'amp_C'",)
Gradient overflow. Skipping step, loss scaler 0 reducing loss scale to 32768.0
Gradient overflow. Skipping step, loss scaler 0 reducing loss scale to 16384.0
Gradient overflow. Skipping step, loss scaler 0 reducing loss scale to 8192.0
Gradient overflow. Skipping step, loss scaler 0 reducing loss scale to 8192.0
Gradient overflow. Skipping step, loss scaler 0 reducing loss scale to 4096.0
Gradient overflow. Skipping step, loss scaler 0 reducing loss scale to 8192.0
Gradient overflow. Skipping step, loss scaler 0 reducing loss scale to 4096.0
Gradient overflow. Skipping step, loss scaler 0 reducing loss scale to 8192.0
Gradient overflow. Skipping step, loss scaler 0 reducing loss scale to 4096.0

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
[Deepspeed] Assert on mismatches between ds and hf args
Author: stas00
Pull request: yes

This is another iteration to make things less prone to errors when dealing with the complex space of partially overlapping DS and HF Trainer configs.

- validate params and assert on mismatch - revamps how the config massage is done
- add a test
- add docs
- this new code uncovered a config mismatch in the deepspeed tests thanks to the new validation, so fixed that too.

As example on a really bad mismatch (as in the new test), the user gets an assert with:
```
Please correct the following DeepSpeed config values that mismatch TrainingArguments values:
- ds train_micro_batch_size_per_gpu=4 vs hf per_device_train_batch_size=2
- ds gradient_accumulation_steps=4 vs hf gradient_accumulation_steps=2
- ds train_batch_size=1000 vs hf train_batch_size (calculated)=4
- ds gradient_clipping=1.1 vs hf max_grad_norm=1.0
- ds optimizer.params.betas=[0.8, 0.89] vs hf adam_beta1+adam_beta2=[0.9, 0.99]
- ds fp16.enabled=False vs hf fp16+fp16_backend(amp)=True
The easiest method is to set these DeepSpeed config values to 'auto'.
```

Fixes: https://github.com/microsoft/DeepSpeed/issues/1107

@sgugger 

Comments:
@stas00 - this looks great.  Thanks for the added bonus of showing the conflicting names, values, and sources!

I'm glad to hear you found it useful, @rfernand2. 

This is all new and experimental so if you find other suggestions for improvements please don't hesitate to file an issue. Thank you!
[TrainerArguments] format and sort __repr__, add __str__
Author: stas00
Pull request: yes

This PR:
- makes the `TrainerArguments` log dump more readable - by sorting and formatting the output
- `__repr__` wasn't actually used in examples, but `__str__` was needed - so fixing that buglet

New dump looks like:

```
06/03/2021 17:05:12 - INFO - __main__ -   Training/evaluation parameters Seq2SeqTrainingArguments(
_n_gpu=1,
adafactor=False,
adam_beta1=0.9,
adam_beta2=0.999,
adam_epsilon=1e-06,
dataloader_drop_last=False,
dataloader_num_workers=0,
dataloader_pin_memory=True,
ddp_find_unused_parameters=None,
debug=[],
deepspeed=None,
disable_tqdm=False,
do_eval=True,
do_predict=False,
do_train=False,
eval_accumulation_steps=None,
eval_steps=2500,
evaluation_strategy=IntervalStrategy.STEPS,
fp16=False,
fp16_backend=auto,
fp16_full_eval=True,
fp16_opt_level=O1,
gradient_accumulation_steps=1,
greater_is_better=None,
group_by_length=False,
ignore_data_skip=False,
label_names=None,
label_smoothing_factor=0.1,
learning_rate=3e-05,
length_column_name=length,
load_best_model_at_end=False,
local_rank=-1,
log_on_each_node=True,
logging_dir=runs/Jun03_17-05-12_hope,
logging_first_step=True,
logging_steps=500,
logging_strategy=IntervalStrategy.STEPS,
lr_scheduler_type=SchedulerType.LINEAR,
max_grad_norm=1.0,
max_steps=-1,
metric_for_best_model=None,
mp_parameters=,
no_cuda=False,
num_train_epochs=1.0,
output_dir=output_dir,
overwrite_output_dir=True,
past_index=-1,
per_device_eval_batch_size=16,
per_device_train_batch_size=8,
predict_with_generate=True,
prediction_loss_only=False,
push_to_hub=False,
remove_unused_columns=True,
report_to=['tensorboard'],
resume_from_checkpoint=None,
run_name=output_dir,
save_steps=500,
save_strategy=IntervalStrategy.STEPS,
save_total_limit=None,
seed=42,
sharded_ddp=[],
skip_memory_metrics=True,
sortish_sampler=True,
tpu_metrics_debug=False,
tpu_num_cores=None,
use_legacy_prediction_loop=False,
warmup_ratio=0.0,
warmup_steps=50,
weight_decay=0.0,
)
```

@sgugger 

Comments:
Seq2SeqTrainer: cannot set max length when we evaluate/(generate) during training
Author: allanj
Pull request: no

## Environment inf
- `transformers` version: 4.6.1

### Who can help

- trainer: @sgugger 


## Information
Seq2SeqTrainer: cannot set max length when we evaluate/(generate) during training.

I know we can set the max length during the actual evaluation here: https://github.com/huggingface/transformers/blob/cbe63949d7/examples/seq2seq/finetune_trainer.py#L321

But if we want to set the max length during the evaluation in training:
https://github.com/huggingface/transformers/blob/cbe63949d7/src/transformers/trainer.py#L924
I can see, currently, there is nothing I can pass in. 
I found the solution can be `model.config.max_length`, but is there a more explicit argument that I can pass in?



Also, if I want to get the logits as well during training, is there anything I can do here?
https://github.com/huggingface/transformers/blob/cbe63949d7/src/transformers/trainer.py#L1505


## Expected behavior

Set max length, and output logits, during the evaluation in training


Comments:
The Seq2SeqTrainer also accepts the `max_length` argument in its [evaluate method](https://github.com/huggingface/transformers/blob/1f335aef3bb5382b5cfd7adbe5861ed4979dd98d/src/transformers/trainer_seq2seq.py#L41).

Yeah. But the `Seq2SeqTrainer` extends `Trainer`, which implements the actual`train` function.

https://github.com/huggingface/transformers/blob/cbe63949d7/src/transformers/trainer.py#L924
And it is "fixed" as in no argument will be passed in.

Ah yes, for this you need to set the parameters in the config then.
I cannot import deepsepped
Author: Arij-Aladel
Pull request: no

```
from transformers.file_utils import   CONFIG_NAME


from transformers.deepspeed import deepspeed_config, is_deepspeed_zero3_enabled
```

why I am getting this error ? even the file_utils and deepspeed are in the same directory I can import the first one but not the second which is not understandable for me


> ---------------------------------------------------------------------------
> ModuleNotFoundError                       Traceback (most recent call last)
> File /data/home/admin/.conda/envs/cmr_env/lib/python3.8/site-packages/IPython/core/interactiveshell.py, in run_code:
> Line 3437:  exec(code_obj, self.user_global_ns, self.user_ns)
> 
> In  [12]:
> Line 1:     from transformers.deepspeed import deepspeed_config, is_deepspeed_zero3_enabled
> 
> ModuleNotFoundError: No module named 'transformers.deepspeed'
> ---------------------------------------------------------------------------

Comments:
Sorry my bad I have not noticed that thee was changes on transformers and should upgrade the installation
pipelines should allow passing in tokenizer arguments
Author: EtienneT
Pull request: no

# 🚀 Feature request

It should be possible to pass in additional arguments for the tokenizer in the pipeline constructor.

Something like this:

```
from transformers import pipeline
classifier = pipeline('sentiment-analysis', padding=True, truncation=True, max_length=512, device=0)
```

## Motivation

For example for a sentiment-analysis pipeline, if the model has a maximum number of tokens and you pass-in larger text than that to the pipeline it will make the pipeline crash.  It would be really nice to be able to provide additional arguments for the tokenizer like padding=True, truncation=True, max_length=512 for example.  The only workaround I found was to create the tokenizer and model separately and provide the arguments to the tokenizer directly.

Here is how I do it right now:

```
pt_batch = tokenizer(texts, padding=True, truncation=True, max_length=512, return_tensors="pt")
    for x in pt_batch.keys():
        pt_batch[x] = pt_batch[x].to('cuda')
    pt_outputs = model(**pt_batch)
```

and how I would prefer to be able to do it instead:

```
from transformers import pipeline
classifier = pipeline('sentiment-analysis', padding=True, truncation=True, max_length=512, device=0)
preds = classifier(texts)
```

Comments:
Hi,

you can pass a 2-tuple `(tokenizer_name, tokenizer_kwargs)` to achieve this. E.g.:
```python
from transformers import pipeline
classifier = pipeline('sentiment-analysis', tokenizer=(tokenizer_name, {"padding": True, "truncation": True, "max_length": 512}), device=0)
```
@patrickvonplaten Any particular reason why this is not documented? 

@mariosasko thanks for your response! I tried this, but unfortunately I get the same error.
```
from transformers import pipeline
model_name = 'distilbert-base-uncased-finetuned-sst-2-english'
classifier = pipeline('sentiment-analysis', model=model_name, tokenizer=(model_name, {"padding": True, "truncation": True, "max_length": 512}), device=0)
```

The error:

> Token indices sequence length is longer than the specified maximum sequence length for this model (1055 > 512). Running this sequence through the model will result in indexing errors

Thanks,

My bad. Just checked the source. This should work:
```python
from transformers import pipeline
classifier = pipeline('sentiment-analysis', device=0)
classifier(texts, padding=True, truncation=True, max_length=512)
```

This works!  I was sure I had tried that, but it seems not.

Thank you!
Move integrations imports before any ML framework imports
Author: dsblank
Pull request: yes

## Fixes

Current transformers breaks compatibility with comet_ml because it needs to be imported before any ML frameworks (such as torch). This PR simply moves the imports earlier in the flow.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

- integrations and imports: @sgugger


Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Electra model vocabulary
Author: avinashsai
Pull request: no

1. Electra model vocabulary doesn't show the vocabulary words unlike other models where vocabulary words can be clearly seen. 

2.  In this link (https://huggingface.co/google/electra-base-discriminator/resolve/main/vocab.txt) strangely all words are [unused0] barring [PAD], [CLS] and few other special tokens.

3. How can I seen vocabulary words for Electra tokenizer??

Comments:
Small error in documentation / Typo
Author: ashkamath
Pull request: no

The documentation for BART decoder layer mentions that it expects hidden states as well as the encoder hidden states to be expected in "(seq_len, batch, embed_dim)" instead of the "(batch, seq_len, embed_dim)" that is actually is expected. Lead to a bit of confusion so would be great if it were corrected! :)

Relevant lines:
https://github.com/huggingface/transformers/blob/996a315e76f6c972c854990e6114226a91bc0a90/src/transformers/models/bart/modeling_bart.py#L373
https://github.com/huggingface/transformers/blob/996a315e76f6c972c854990e6114226a91bc0a90/src/transformers/models/bart/modeling_bart.py#L376

@patrickvonplaten 

Comments:
Thanks for spotting. Feel free to open a PR to fix this :) 

By the way, I see you're the main author of MDETR (amazing work!). I'm currently adding DETR to the repo (see #11653), so if you are up to help me add MDETR to the repo, feel free to reach out :) 


Ooh thanks :D That sounds great, I'd be happy to help :) Will send you an email.
Fixed Typo in modeling_bart.py
Author: ceevaaa
Pull request: yes

# What does this PR do?

Fixes #11895 
Fixed Typo `(seq_len, batch, embed_dim)` to `(batch, seq_len, embed_dim)`  in line 
[373](https://github.com/huggingface/transformers/blob/996a315e76f6c972c854990e6114226a91bc0a90/src/transformers/models/bart/modeling_bart.py#L373) and [376](https://github.com/huggingface/transformers/blob/996a315e76f6c972c854990e6114226a91bc0a90/src/transformers/models/bart/modeling_bart.py#L376) as discussed [here](https://github.com/huggingface/transformers/issues/11895).

@NielsRogge 

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Comments:
Thanks a lot for fixing this!

Could you run `make fix-copies` and then push again?
FAILED tests/test_tokenization_deberta_v2.py::DebertaV2TokenizationTest::test_tokenizer_integration
Author: PhilipMay
Pull request: no

This integration (`@slow`) test fails:

see https://github.com/huggingface/transformers/runs/2723622794?check_suite_focus=true#step:7:2663

I will provide a PR

Comments:
> I will provide a PR

see #12017
long text classification
Author: gogokre
Pull request: no
Labels: name: wontfix

I want to binary classify documents with max length greater than 512 with bert.
Is there any reference or code for how to do this?

Comments:
Following!

> Following!

I don't know what it means.


Use transformer-XL

Longformer is exactly designed for your use case (https://github.com/allenai/longformer)

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Reformer and longformer are being worked on to be included in this library.

Looks like they've got longformer now. 

https://huggingface.co/transformers/model_doc/longformer.html

Is longformer a multilingual model? or are there options to work with longer texts that are not in English? 

You can also use Text Guide, a clever text truncation method and use a transformer model with a standard 512 limit.
And if you have extremely long text instances (longer than 4096 == Longformer model limit) you can also use this approach to further improve your results.

Paper: https://arxiv.org/abs/2104.07225
Code: https://github.com/krzysztoffiok/TextGuide
A brief description: https://www.quora.com/If-I-have-a-long-text-say-10-paragraphs-how-can-I-use-BERT-or-other-newer-and-better-models-such-as-RoBERTa-for-feature-extraction-that-represents-the-entire-document-Seems-like-BERT-has-limits-Are-there-packages
Using the  latest DPR checkpoint available with HuggingFace DPR class
Author: shamanez
Pull request: no

Hi,
There's a new DPR checkpoint available in [DPR Orignal repositary](https://github.com/facebookresearch/DPR#new-march-2021-retrieval-model), which shows nice improvements.  I did convert the DPR checkpoint using the conver_dpr.py (did a minor modification) and it is working fine.  

I have one question regarding the correct tokenizer that should use for the question_encder and context_enocder.  Since I can load the tokenizer from the following paths (AutoTokenizer.from_pretrained),  I assumed both of these tokenizers behave in the same way (PreTrainedTokenizerFast).
1.  **facebook/dpr-question_encoder-multiset-base** 
2.  **facebook/dpr-question_encoder-single-nq-base**

So, there won't be any problem if I use a tokenizer loaded from any of the above paths with the new checkpoint right (since DPR uses HuggingFace Tokenizers)?

@lhoestq 


Comments:
Hi ! Yes you should be fined. Under the hood it's actually the same tokenizer as `bert-base-uncased`.

Also it would be nice to add the new DPR checkpoints on the Hub as well.

What changes did you have to do in the convert_dpr.py file ?

Perfect and thanks @lhoestq 

 Actually, it was a very minor change.  With the current transformer version, it gives an error [in this line](https://github.com/huggingface/transformers/blob/finalize_rag/src/transformers/convert_dpr_original_checkpoint_to_pytorch.py#L71), saying no positional_id key in the state_dict.  It simply worked with setting the **strict= False**.
 
 But for clarity I changed as follows (check newly added line :)):
 
```
 class DPRQuestionEncoderState(DPRState):
    def load_dpr_model(self):
        model = DPRQuestionEncoder(DPRConfig(**BertConfig.get_config_dict("bert-base-uncased")[0]))
        print("Loading DPR biencoder from {}".format(self.src_file))
        saved_state = load_states_from_checkpoint(self.src_file)
        encoder, prefix = model.question_encoder, "question_model."
        model_state_dict = encoder.state_dict()
        state_dict = {}

        for key, value in saved_state.model_dict.items():
            if key.startswith(prefix):
                key = key[len(prefix) :]
                if not key.startswith("encode_proj."):
                    key = "bert_model." + key
                state_dict[key] = value

       #newly added
        for k , v in model_state_dict.items():
            if not k in state_dict:
                print("warnning can't find key:",k)
                state_dict[k]=v

        #encoder.save_pretrained(save_directory='./', state_dict=state_dict) #no need to modify
        encoder.load_state_dict(state_dict)
        return model
```