noamaanMulla-03/transformers-issues huggingface dataset:
如何将xlnet作为嵌入层置于其他模型前
Author: yangbin-Neil
Pull request: no

### Feature request

我想将预训练模型xlnet作为嵌入层添加到bilstm-CRF前，如何作为一个嵌入层加入到模型的构建当中。比如说，我现在用bert4keras框架搭建好了bert-Bilstm-Crf模型，我想将bert替换为xlnet，将xlnet作为模型的嵌入层，这样可不可以？？？
-------------------------

    def bert_bilstm_crf(config_path,checkpoint_path,latm_units,drop_rate,learning_rate):      # 构建模型的结构
       -----------------------------------------------------------------------------------以下关于bert的内容是否可以替换
         bert=build_transformer_model(                              #BERT加载权重
                    config_path=config_path,
                    checkpoint_path=checkpoint_path,
                    model='bert',
                    return_keras_model=False
                )
        x=bert.model.output       #[batchsize,seq_len,768] 
        x=keras.layers.Bidirectional(
            keras.layers.LSTM(         #加入lstm模型
                latm_units,
                kernel_initializer='he_normal',    #初始化规则
                return_sequences=True
        )
        )(x)    #[batchsize,seq_len,lstm_units*2]

### Motivation

修改关于ner方面的问题

### Your contribution

解决模型构建的问题

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Added the timeout variable in training args to avoid socket timeouts in DDP calls
Author: dvlshah
Pull request: yes

Overriding the default timeout defined by PyTorch in **torch.distributed.init_process_group** calls by introducing a timeout argument and prevents Socket Timeouts

Added a custom timeout argument in **src/transformers/training_args.py**. It can be used to override the timeout argument in the **init_process_group** function call to avoid socket timeouts for mapping or tokenizing huge datasets that may take a longer time.

The timeout variable is a **int** type and default value is set to **1800(s)** which is default value in torch.distributed.init_process_group fn. 

The timeout datatype used in torch.distributed.init_process_group is a **datetime.timedelta** object that expects this timeout variable.

# What does this PR do?

Fixes #18054 #17106


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_18081). All of your documentation changes will be reflected on that endpoint.

Hey @dvlshah, thank you for your PR! Would this parameter need to be used on https://github.com/huggingface/transformers/blob/ac98a88fbc6377f93e8b7fbd244b0c3331bb82a0/src/transformers/training_args.py#L1310?

cc @sgugger 

> Hey @dvlshah, thank you for your PR! Would this parameter need to be used on
> 
> https://github.com/huggingface/transformers/blob/ac98a88fbc6377f93e8b7fbd244b0c3331bb82a0/src/transformers/training_args.py#L1310
> 
> ?
> cc @sgugger

The idea is provide the option to use this parameter if they want in **torch.distributed.init_process_group(backend=self.xpu_backend, rank=rank, world_size=size)**  fn call.

Thanks, but I'm not sure I follow the PR: why add a new `TrainingArguments` if it's not used anywhere?

@sgugger I need to add the timeout var in the torch.distributed.init_process_group call. Forgot the push the change in the PR.

Hey @dvlshah, did you push the changes?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Add `in_chans` to `DetrModel`
Author: Zhylkaaa
Pull request: no

# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->
Add `in_chans` parameter to `DetrConfig` and corresponding models

## Motivation
I find myself wanting to test DETR on data that have only one channel (it can be grey scale images or spectrograms in my case). But to make it work I have to use hacks, that may eventually work purely.
<!-- Please outline the motivation for the proposal. Is your feature request
     related to a problem? e.g., I'm always frustrated when [...]. If this is related
     to another GitHub issue, please link here too. -->

## Your contribution
I will be happy to open PR with this if this is as useful as I see it. Maybe some other improvements that can be incorporated? 
UPADTE: I think this will involve also adding parameters to disable `replace_batch_norm` and very specific part of setting `parameter.requires_grad_(False)` to all parameters in resnet except layer2-4

<!-- Is there any way that you could help, e.g. by submitting a PR?
     Make sure to read the CONTRIBUTING.MD readme:
     https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md -->


Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

I also think it could be useful. I don't want to waste memorys by convert 1 channel image to 3 channels.
PyTorch Cuda Tensors not supported in DETR model
Author: dennisushi
Pull request: no

https://github.com/huggingface/transformers/blob/db7d6a80e82d66127b2a44b6e3382969fdc8b207/src/transformers/models/detr/feature_extraction_detr.py#L568

The feature extractor attempts to convert the tensors to PIL images, which is only possible for `t.detach().cpu()` tensors. 

The origin of the error is `ImageFeatureExtractorMixin.to_pil_image()`

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

I'm having same problem here. Did you solve it?
`AttributeError: 'BertTokenizer' object has no attribute 'tokens_trie'
Author: roetezadi
Pull request: no
Labels: name: bug

### System Info

Google colab

### Who can help?

Anyone

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Hi, 
I have trained a model with `transformers==4.9.1`. 
Now I want to run this saved model with the new version of transformers and I get this error: `AttributeError: 'BertTokenizer' object has no attribute 'tokens_trie'`. 
As I looked at transformers codes `token_trie` has been added which was not in the previous versions if I am correct. How can I solve this problem from my side? and is there any possibilities that this compatibility problem can be handled in the newer versions of transformers?

### Expected behavior

None

Comments:
@Narsil is the one knows the best.

This is added in the PR #13220

@roetezadi How did you solve this ?
Fix failure on DeBERTa(base/v2/sew_d) fp16 training with ONNX Runtime
Author: JingyaHuang
Pull request: yes

## Context

It was reported in optimum https://github.com/huggingface/optimum/issues/305 that the mixed-precision training on DeBERTa with optimum.onnxruntime.ORTTrainer is broken.

After investigation, the break comes from mismatched inputs dtype for some Matmul nodes. In #18272, some sqrt results are cast to fp32, and they need to be re-casted to fp16 before Matmul ops, and this PR is supposed to correct the dtype.

Besides, this PR also fix the tracing of DeBERTa which haven't been fixed in #18272 

Fixes #https://github.com/huggingface/optimum/issues/305
Fixes #18199 

Who can review?

@michaelbenayoun @LysandreJik @sgugger 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix ORTTrainer failure on DeBERTa(base/v2/sew_d) fp16 training
Author: JingyaHuang
Pull request: yes

# What does this PR do?

__Context__

It was reported in optimum https://github.com/huggingface/optimum/issues/305 that the training on DeBERTa with optimum.onnxruntime.ORTTrainer is broken. 
After investigation, the break comes from two causes:
* At that time `XDropOut` didn't have a symbolic function. And it has been implemented by @garymm in https://github.com/huggingface/transformers/pull/17502 and has been merged to the main of transformers.
* The implementation of DeBERTa have some numpy/math operations that led to incorrect export. This will be fixed in https://github.com/huggingface/transformers/pull/18272. 

However with those two fixes, the fp32 training will work, but the mixed-precision training will fail due to mismatched inputs dtype for some `Matmul` nodes. In https://github.com/huggingface/transformers/pull/18272, some `sqrt` results are cast to `fp32`, and they need to be re-casted to fp16 before `Matmul` ops, and this PR is supposed to add the re-cast part.

Fixes #https://github.com/huggingface/optimum/issues/305

## Who can review?

@LysandreJik @patrickvonplaten @lewtun 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_18529). All of your documentation changes will be reflected on that endpoint.

close as it turned to be too messy even after rebasing.
Add missing tokenizer tests - Longformer
Author: tgadeliya
Pull request: yes

# What does this PR do?

This PR add tests for Longformer tokenizer copying tests from Roberta tokenizer's test suite, because those tokenizers are absolutely identical. 

Fixes #16627

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [X] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [X] Did you write any new necessary tests?


## Who can review?
@SaulLu @LysandreJik 



Comments:
_The documentation is not available anymore as the PR was closed or merged._

I read discussion in merged tokenizers' tests PRs and post [~~Don't~~ Repeat Yourself*](https://huggingface.co/blog/transformers-design-philosophy) on HF blog and I manually add "the copying mechanism". But I don't understand how it is work, so I tried not to change copied test code from Roberta tokenizer tests. If code modification is not a problem, I would like to add some minor changes, e.g. delete commented code and split big test into smaller one.
Could describe "copying mechanism" works in more details?

Thanks a lot for working on this @tgadeliya!! 

As far as I know, there are no identified "practices" for this case (cc @LysandreJik in case you have another opinion). Nevertheless, if changes are relevant, they are obviously welcome. For example, it is possible to indicate the changes made as here: 
https://github.com/huggingface/transformers/blob/d95a32cc60e5d92b4bf08cd805c6b0db7b4100cc/src/transformers/models/deberta/modeling_deberta.py#L308-L309
If the differences are too long to list perhaps the message can just explain why it diverged from the originally copied and pasted code.

Does this help you?

@SaulLu, Sorry for the late reply. Summer is ending :) 

Thanks for your comment. Now it is clear for me. Actually, I came to the conclusion, that code cleaning not so necessary considering all pros and cons. So this PR can be reviewed and merged 

@SaulLu  I refreshed this PR, so now it is ready to merge

Thanks @tgadeliya :hugs:
The problem in BATCH generation of GPT model
Author: yupei9
Pull request: no

When I tried to use GPT model (including GPT-2, GPT-NEO-2.7B, GPT-J-6B, GPT-NEOX) to generate text, I found some strange results.

When I set the batch size to 1, all results are normal. 
BUT when I set the batch_size more than 1, such as 4, 8, ..., the generated text on GPT-J-6B,  GPT-NEOX is abmoral, which contains a large number of repeated consecutive letters or words, for example "AAAAAAAAAAAAAAA" or "The The The The The The The The The".

I can not find the root of this question. Could you please give some suggestions to solve this problem? Thank you!



Comments:
Hi,

See this thread for batched generation: https://github.com/huggingface/transformers/pull/7552#issue-714062850

> Hi,
> 
> See this thread for batched generation: [#7552 (comment)](https://github.com/huggingface/transformers/pull/7552#issue-714062850)

Thanks a lot! 
And could you please tell me whether the current version supports correct sampling generation with batched setting? Thanks! 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Closing this as the issue seems resolved.
Please Add a Fast Flaubert_tokenizer class as well to leverage fast_tokenizer methods
Author: datatales-with-pankaj
Pull request: no
Labels: name: bug

### System Info

transformers version: 4.20.1
GPU: Nvidia Titan T4



### Who can help?

@LysandreJik I feel that you can help best since Flaubert Tokenizer class is directly inherited from XLM's Tokenizer which also does not have a fast tokenizer class too!



### Information

- [x] My own modified scripts
- [ ] The official example scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction


This method is not working since my AutoTokenizer Class is taking the default FlaubertTokenizer class and since we don't have a Flaubert_tokenizer_fast class here; we are not able to leverage fast_tokenizer functionalities!
<img width="1245" alt="Screenshot 2022-07-14 at 4 39 38 PM" src="https://user-images.githubusercontent.com/78647606/179044514-e6081be4-591a-48bb-bf10-56ab90628ab0.png">


### Expected behavior

If FlaubertTokenizer has a faster version class like BertToeknizerFast; We will be able to use class methods like word_ids, word_to_tokens, etc Specifically for the TokenClasssifcation task.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
update no trainer scripts to remove check for main process while initiating trackers
Author: Gladiator07
Pull request: yes

# What does this PR do?

This PR updates no_trainer examples with tracking enabled to remove the check for `is_main_process` while initiating trackers by `accelerator.init_trackers()` as this method automatically initializes the trackers on the main process. This behavior was merged in Accelerate in this [PR](https://github.com/huggingface/accelerate/pull/642)

- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@muellerzr @sgugger 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Unpin detectron2
Author: ydshieh
Pull request: yes

# What does this PR do?

Unpin `detectron2` that are introduced in #18701 and #18680, as the issue was fixed on its side

https://github.com/facebookresearch/detectron2/commit/3cfda4704cb3b23255f29d683cf05cbaf10daaa4

The test `run_tests_layoutlmv2_and_v3` is OK now:

https://app.circleci.com/pipelines/github/huggingface/transformers/46171/workflows/c82c09a9-f6f7-4e21-8732-6681083960ff/jobs/541901

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_18727). All of your documentation changes will be reflected on that endpoint.
[Improvements] Enable `git push` without requiring login when uploading model
Author: xhluca
Pull request: no

# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->

Is there a way to allow `git push` a new model version without requiring login? 

`transformers-cli` will auto generate a key for the user. Is there a way to leverage this key?

## Motivation

GitHub allows you to push in VSCode without explicitly login in each time.

<!-- Please outline the motivation for the proposal. Is your feature request
     related to a problem? e.g., I'm always frustrated when [...]. If this is related
     to another GitHub issue, please link here too. -->


Comments:
Yes. Two ways:

- you can add your credentials when cloning, e.g. `git clone https://username:password@huggingface.co/user/model_id` (you can also use your token instead of your password if you know it).
- but the cleaner way is to have a credential store set-up in git (https://git-scm.com/book/en/v2/Git-Tools-Credential-Storage) which means git will only ask you once. GitHub's doc is good as well (and not specific to GitHub): https://docs.github.com/en/free-pro-team@latest/github/using-git/caching-your-github-credentials-in-git

Let me know if this helps

Thanks! I'll try this out and reopen this issue if any problem arises!

Is there any documentation for "How to set the credential helper for Hugging Face URLs without changing the helper used for all other repos?"

you can set the helper for a specific repo clone, but I'm not sure if you can set the helper system-wide for specific hosts
improve `add_tokens` docstring
Author: SaulLu
Pull request: yes

# What does this PR do?

This PR proposes to clarify the fact that added tokens and tokens of the vocabulary of the tokenization algorithm are not treated the same way by detailing the docstring of the `add_tokens` method.

Fixes #18662


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_18687). All of your documentation changes will be reflected on that endpoint.
CLI: Don't check the model head when there is no model head
Author: gante
Pull request: yes

# What does this PR do?

As the title says -- when there is no model head, don't check it :)

When `architectures` is not defined, it is implied that there is no model head in the weights.

Comments:
cc @ChrisFugl (related to #18678 )

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_18733). All of your documentation changes will be reflected on that endpoint.

OK, so it was empty dict, and we can't take max of the values, right? LGTM, thank you @gante 

nit: I would probably check if the dict is empty or not, instead of check `architectures` is None.

For example, `BertModel` has `pooler_output`.

@ydshieh that is a great point! 

Going to change it

@ydshieh added your suggestion (plus a check -- a model specified head should have outputs beyond `hidden_...`)