noamaanMulla-03/transformers-issues huggingface dataset:
Why the output of DistilBertModel is inconsistent with BertModel?!
Author: amirj
Pull request: no

[The output of DistilBertModel](https://github.com/huggingface/transformers/blob/be916cb3fb4579e278ceeaec11a6524662797d7f/transformers/modeling_distilbert.py#L468) does not contain [pooled_output as available in BERT model](https://github.com/huggingface/transformers/blob/be916cb3fb4579e278ceeaec11a6524662797d7f/transformers/modeling_bert.py#L632).
I'm going to replace BERT with DistilBert in a classification task, if pooled_output is not a proper way in DistillBertModel? -Currently, I'm using pooled_output of BERT in my experiments.

Comments:
Hello @amirj,

The "pooled_output" is the hidden state of the `[CLS]`. It is this hidden state that is used for classification tasks for instance (see DistilBertForSequenceClassification). So you could retrieve it by filtering out `hidden_states `.

The reason why there is no linear transformation in the pooler of DistilBERT is because I'm removing the next sentence prediction (see RoBERTa which also do the same). However, this linear transformation is still in `DistilBertForSequenceClassification` so that the classification heads for DistilBERT and BERT have the same number of parameters.

I hope it answers your question.
Victor

Hello @VictorSanh,
DistilBertForSequenceClassification is the rescue.
Thanks.
Amir

def forward(self, input_ids, attention_mask, labels=None):
    output = self.bert(input_ids, attention_mask = attention_mask)
    output = self.classifier(output.hidden_states) 
    output = torch.sigmoid(output)    
    loss = 0
    if labels is not None:
        loss = self.criterion(output, labels)
    return loss, output

for BERT model self.classifier will take output.pooler_output as its input
But for DistilBERT this doesnt happen.
I used "DistilBertForSequenceClassification " and replaced pooler_output with hidden_states, still it doesnt work

Error I get is: linear(): argument 'input' (position 1) must be Tensor, not NoneType
How to save the fine-tuned model
Author: yz-qiang
Pull request: no

Hi, 
I save the fine-tuned model with the `tokenizer.save_pretrained(my_dir)` and `model.save_pretrained(my_dir)`. Meanwhile, the model performed well during the fine-tuning(i.e., the loss remained stable at **0.2790**). And then, I use the `model_name.from_pretrained(my_dir)` and ` tokenizer_name.from_pretrained(my_dir)` to load my fine-tunned model, and test it  in the training data. The value of loss surprised me because it was so high, i.e., **4.7**. I don't know why. Can you help me?  Please reply to me at your convenience. Thank you very much. :)

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
huggingface_pytorch-pretrained-bert_bert.ipynb -- RuntimeError: Cannot find callable bertTokenizer in hubconf
Author: neqkir
Pull request: no

## Environment info
Google Colab
https://colab.research.google.com/github/pytorch/pytorch.github.io/blob/master/assets/hub/huggingface_pytorch-pretrained-bert_bert.ipynb#scrollTo=bb1RXLUYj_Wv

- Using GPU in script?: yes

Models:

BERT

## To reproduce

Steps to reproduce the behavior:

```
### First, tokenize the input
import torch
tokenizer = torch.hub.load('huggingface/pytorch-pretrained-BERT', 'bertTokenizer', 'bert-base-cased', do_basic_tokenize=False)

# Tokenized input
text = "[CLS] Who was Jim Henson ? [SEP] Jim Henson was a puppeteer [SEP]"
tokenized_text = tokenizer.tokenize(text)
indexed_tokens = tokenizer.convert_tokens_to_ids(tokenized_text)
```

Get error stack

```
Downloading: "https://github.com/huggingface/pytorch-pretrained-BERT/archive/master.zip" to /root/.cache/torch/hub/master.zip
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-2-0cb4dd772680> in <module>()
      1 ### First, tokenize the input
      2 import torch
----> 3 tokenizer = torch.hub.load('huggingface/pytorch-pretrained-BERT', 'bertTokenizer', 'bert-base-cased', do_basic_tokenize=False)
      4 
      5 # Tokenized input

2 frames
/usr/local/lib/python3.7/dist-packages/torch/hub.py in load(repo_or_dir, model, source, force_reload, verbose, skip_validation, *args, **kwargs)
    397         repo_or_dir = _get_cache_or_reload(repo_or_dir, force_reload, verbose, skip_validation)
    398 
--> 399     model = _load_local(repo_or_dir, model, *args, **kwargs)
    400     return model
    401 

/usr/local/lib/python3.7/dist-packages/torch/hub.py in _load_local(hubconf_dir, model, *args, **kwargs)
    425     hub_module = import_module(MODULE_HUBCONF, hubconf_path)
    426 
--> 427     entry = _load_entry_from_hubconf(hub_module, model)
    428     model = entry(*args, **kwargs)
    429 

/usr/local/lib/python3.7/dist-packages/torch/hub.py in _load_entry_from_hubconf(m, model)
    233 
    234     if func is None or not callable(func):
--> 235         raise RuntimeError('Cannot find callable {} in hubconf'.format(model))
    236 
    237     return func

RuntimeError: Cannot find callable bertTokenizer in hubconf

```

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Logits warper for batch generation
Author: bnurbekov
Pull request: no

# 🚀 Feature request
Is there a way to modify `logits_warper`, so that it will apply different parameters (`top_p`, `top_k`, `temperature`) for each item in a batch instead of applying the same parameters for all items in the batch:

https://github.com/huggingface/transformers/blob/f25a9332e8d091398ce96c462e02a467943c8eb9/src/transformers/generation_utils.py#L1562

## Motivation

It will very useful for inference because it will increase the throughput. Currently, you cannot really use #7552 for inference because the same parameters apply to the entire batch.

## Your contribution

Can help with implementing this feature and reviewing the PR.

Comments:
Hey @bnurbekov,

Sorry for replying so late. We recently allowed to pass customized logits wrapper to `generate()`. Could you maybe try to build a custom wrapper for your purpose this way? 

https://github.com/huggingface/transformers/pull/14779

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Add conversion from TF to PT for Tapas retrieval models
Author: bogdankostic
Pull request: yes

# What does this PR do?

Table Retrieval models based on Tapas as described [here](https://arxiv.org/pdf/2103.12011.pdf) just got published in the [Tapas repository](https://github.com/google-research/tapas). The existing conversion function does not work with the retrieval models, so I added support to convert them to Pytorch.
Unfortunately, this only converts the language model without the down projection layer. However, I think this might still be useful to some people who, for instance, want to fine-tune the pre-trained models. 
Unfortunately, I do not have the time at the moment to add the down projection layer myself.


## Who can review?

@NielsRogge 


Comments:
Thanks for this, it's certainly something I'd like to add in the future. The TAPAS team seems quite active, they released [yet another paper involving TAPAS](https://arxiv.org/abs/2106.00479) (to work on larger tables).

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

@NielsRogge I just quickly reviewed this script and it looks fine. Is it possible to know why this PR became stall and was automatically closed? Is there anything wrong with the script I should be aware of? 

I'm currently working on finetuning the TAPAS retrieval model for a research project, just wanted to have your thoughts on this before running the script and uploading the model to the Huggingface hub.

@jonathanherzig Just wanted to confirm with you, in this case `bert` is the table encoder and `bert_1` is the question encoder, right? 

Hi @xhlulu ,
Sorry, but I not familiar with the implementation details in this version of TAPAS... probably @NielsRogge  can help.

Best,
Jonathan

No worries, thanks Jonathan!
502 Server Error: Bad Gateway for url: https://huggingface.co/api/models/t5-base
Author: EdieLu
Pull request: no

Saw similar issue in this forum before, and it seems that a reboot is needed for the data centre? 

It happens when I try this - 
t5_tokenizer = T5Tokenizer.from_pretrained("t5-base")

Comments:
I have the same issue when I try to train a distilbert model:

502 Server Error: Bad Gateway for url: https://huggingface.co/distilbert-base-uncased/resolve/main/config.json

Currently, it seems that the huggingface server is down. 

it's back up now
Porting Compressive Transformer to Huggingface
Author: vilmarzti
Pull request: no
Labels: name: New model

# 🌟 New model addition

As part of my Master-Thesis I will implement or adapt different transformer architecture for LM that are specifically designed for long-context situation. As part of this I started with porting the Compressive Transformer to the huggingface interface. And will probably do the same for other architectures in the future. Let me know if you're interested for a pull-request.

## Model description
[Paper](https://arxiv.org/pdf/1911.05507.pdf)

The Compressive Transformer is an extension of the Transformer-XL architecture with an additional compressed memory. Memories that would get discarded in the Transformer-XL get compressed and added to the compressed memory. 
The compression function can take different forms but the best performance on word-level LM is a Conv1d compression. 
Training of the C-Transformer happens the same way to how the Transformer-XL architecture is trained. But additional to that there is an "attention-reconstruction-loss" which compares the attention that we get from using the memory with the attention we get from its compressed counter-part. Using the MSE-loss we can perform gradient updates on the compression function


## Open source status

* [x] the model implementation is available: 
https://nn.labml.ai/transformers/compressive/index.html Is an open source implemention under the MIT license
https://github.com/deepmind/pg19 The data-set used in parts of the experiment by the Authors
https://github.com/vilmarzti/long_context_transformers/blob/main/longcontext/transformers/compressive_transformer.py My humble start of porting the architecture to the huggingface-format.
* [x] the model weights are available: 
None that I could find.
Weights (for wikitext-2 and 103) might become available as my thesis progresses and I start training
* [x] who are the authors: 
Jack W. Rae (https://github.com/dm-jrae)
Anna Potapenko (https://github.com/AnyaP)
Siddhant M. Jayakumar (github-profile not found)
Chloe Hillier (github profile not found)
Timothy P. Lillicrap (github profile not found)


Comments:
Handle PyTorch to Flax conversion of 1D convolutions
Author: sanchit-gandhi
Pull request: yes

# What does this PR do?

Currently, only 2-dimensional convolutional layers are renamed and reshaped in the PyTorch to Flax conversion script. This PR handles the case of 1-dimensional convolutions layers, in an entirely equivalent way to their 2-dimensional counterparts.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

I'm a bit surprised that we needed that. We already had 1D Conv layers in Flax in Wav2Vec2 and the conversion worked
Revert "Handle PyTorch to Flax conversion of 1D convolutions"
Author: patrickvonplaten
Pull request: yes

Reverts huggingface/transformers#15519

@patil-suraj @sanchit-gandhi - please see comment on PR. Let me know if I'm mistaken there and we actually do need this new weight conversion, but I really don't see why this would be the case.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
How to boost the speed of one sentence Marian Translation(no batches)?
Author: AbdallahHefny
Pull request: no

I am using a Marian Model for translating from English to Arabic. I want to use this translation per sentence (no batching). 

I am using this simple code:

```
en_ar_tokenizer = AutoTokenizer.from_pretrained("Helsinki-NLP/opus-mt-en-ar") 
en_ar_model = AutoModelForSeq2SeqLM.from_pretrained("Helsinki-NLP/opus-mt-en-ar")

def transformer_en_ar(sen):
    tokens = en_ar_tokenizer('>>ara<<' + sen, return_tensors="pt")
    translated = en_ar_model.generate(**tokens)
    decoded = en_ar_tokenizer.decode(translated[0], skip_special_tokens=True)
    return decoded

queries = [ 'some dummy sentence1',  'some dummy sentence2']
for query in queries:
    translation = transformer_en_ar(query)
```

Is there a way to increase the speed of inferring one sentence? Is there any software or hardware solution?

For example:
Shall I get rid of the for loop of the queries? How?
Will changing either (or both) interop and intraop threads have an effect?
Using GPU did not boost the speed of one sentence at a time? Am I doing something wrong?

Thanks

Comments:
Not sure why you don't want to leverage batching, because it's always faster than passing each sentence at a time through the model:

```
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
import torch

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

tokenizer = AutoTokenizer.from_pretrained("Helsinki-NLP/opus-mt-en-ar") 
model = AutoModelForSeq2SeqLM.from_pretrained("Helsinki-NLP/opus-mt-en-ar")

model.to(device)

sentences = ["Hello world",
            "How are you doing?", # use different length sentences to test batching
            ]
inputs = tokenizer(sentences, padding=True, return_tensors="pt")

output = model.generate(
    input_ids=inputs['input_ids'],
    attention_mask=inputs['attention_mask'],
)
tokenizer.batch_decode(output, skip_special_tokens=True)
```

If you're interested in optimizing the inference speed on CPU, take a look at [ONNX](https://huggingface.co/docs/transformers/serialization) or [Torchscript](https://huggingface.co/docs/transformers/serialization#torchscript). We [recently added support](https://github.com/huggingface/transformers/pull/14586) for ONNX for MarianMT.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
No module named 'transformers.models.fnet.modeling_fnet'
Author: lonngxiang
Pull request: no

transformers            4.12.5

from transformers import AutoModelForTokenClassification,AutoTokenizer,pipeline
model = AutoModelForTokenClassification.from_pretrained('uer/roberta-base-finetuned-cluener2020-chinese')
tokenizer = AutoTokenizer.from_pretrained('uer/roberta-base-finetuned-cluener2020-chinese')
ner = pipeline('ner', model=model, tokenizer=tokenizer)
print(ner("江苏警方通报特斯拉冲进店铺"))

![image](https://user-images.githubusercontent.com/40717349/147812866-c3215989-c7f3-4eb5-bbad-af9681cc2fe1.png)



Comments:
Hey @lonngxiang, can you double check your transformers package version? I'm unable to reproduce on a fresh [Colab notebook](https://colab.research.google.com/drive/1-FUJSSFAl88foTbyqGHyFtwgvHMXTyBV?usp=sharing).

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
to use FX - your torch version must be *exactly* 1.9, even though fx also works in later versions
Author: sdtblck
Pull request: no

[this line](https://github.com/huggingface/transformers/blob/3977b58437b8ce1ea1da6e31747d888efec2419b/src/transformers/file_utils.py#L375) checks that the torch version matches 1.9 exactly, even though fx is available in 1.9+ (i.e torch 1.10).

To use it in torch 1.10, i had to do the following, and the symbolic trace worked just fine, so i'm not sure that this strict versioning is necessary:

```python
from transformers import GPTNeoForCausalLM
import torch
from transformers.models.gpt_neo.modeling_gpt_neo import GPTNeoSelfAttention
from transformers.utils.fx import symbolic_trace
import transformers 

transformers.utils.fx.is_torch_fx_available = lambda: True

model = GPTNeoForCausalLM.from_pretrained("EleutherAI/gpt-neo-125M")

g = symbolic_trace(model)

print(g)
```

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

cc @michaelbenayoun, what's the status of our support of torch fx?

Currently working on a PR (#14321), that should be merged soon, it should allow to use symbolic trace for torch 1.10 and above.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
[JAX/FLAX]: CLM Tokenizer Training confusion
Author: stefan-it
Pull request: no
Labels: name: WIP

Hi,

after looking at the current readme of the [CLM tokenizer training example](https://github.com/huggingface/transformers/tree/master/examples/flax/language-modeling#train-tokenizer-1), there's something strange in the model configuration:

The `config.json` file looks like this:

```json
GPT2Config {
  "_name_or_path": "./",
  "activation_function": "gelu_new",
  "architectures": [
    "GPT2LMHeadModel"
  ],
  "attn_pdrop": 0.0,
  "bos_token_id": 50256,
  "embd_pdrop": 0.0,
  "eos_token_id": 50256,
  "initializer_range": 0.02,
  "layer_norm_epsilon": 1e-05,
  "model_type": "gpt2",
  "n_ctx": 1024,
  "n_embd": 768,
  "n_head": 12,
  "n_inner": null,
  "n_layer": 12,
  "n_positions": 1024,
  "reorder_and_upcast_attn": false,
  "resid_pdrop": 0.0,
  "scale_attn_by_inverse_layer_idx": false,
  "scale_attn_weights": true,
  "summary_activation": null,
  "summary_first_dropout": 0.1,
  "summary_proj_to_labels": true,
  "summary_type": "cls_index",
  "summary_use_proj": true,
  "task_specific_params": {
    "text-generation": {
      "do_sample": true,
      "max_length": 50
    }
  },
  "transformers_version": "4.16.0.dev0",
  "use_cache": true,
  "vocab_size": 50257
}

```

Vocab size is 50257, and `eos_token_id` is set to 50256. I think that setting `eos_token_id` is wrong, because of the following example:

```bash
In [10]: tokenizer.convert_ids_to_tokens([1797, 705, 225, 50256])
Out[10]: ['hal', 'lo', 'Ġ', 'Ġgeestigheid']
```

Id *50256* is originally set to `'Ġgeestigheid'`. I'm not 100% sure, but it should be set to 50257 (and thus outside the vocabulary), because of:

```bash
In [7]: tokenizer.encode("hallo 
")
Out[7]: [1797, 705, 225, 50257]
```

It shows that `eos_token` is set to `
` and from the tokenizer part, `eos_token_id` then should be set to `50257`?!

Now I'm using the official GPT-2 model as reference:

It uses `"eos_token_id": 50256` in the `config.json` file, some tokenizer tests:

```bash
In [6]: tokenizer.eos_token
Out[6]: '
'

In [7]: tokenizer.eos_token_id
Out[7]: 50256

In [8]: tokenizer.encode("Hello 
")
Out[8]: [15496, 220, 50256]
```

Which is correct.

And there's another issue: after looking at the `tokenizer.json` file for GPT-2, the following entry exists:

```bash
"
":50256}
```

which is perfect, but: for the own trained vocab this entry does not exist! I'm not sure if this is a bug in the Tokenizers library or intended :thinking: 

Comments:
Add ConvNeXT
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR adds [ConvNeXT](https://github.com/facebookresearch/ConvNeXt) to the library, a convnet inspired by Transformers.

To do:

- [x] rename nielsr to facebook to fix tests
- [x] remove inference.py script
- [x] remove PushToHubMixin hack from feature extractor
- [x] fix "gamma" in `from_pretrained` method. Right now, if a model includes a parameter named "gamma", they are not initialized from the hub, due to [this line](https://github.com/huggingface/transformers/blob/833635e25997a00117076c50bde5c45f9b883ada/src/transformers/modeling_utils.py#L1490).
- [x] discuss model output classes. It feels a bit weird to talk about "hidden states" for convolutional models, most people use the term "features" or "feature maps". These are of shape (batch_size, num_channels, a certain height, a certain width) at each stage, rather than (batch_size, seq_len, hidden_size) at each layer of a Transformer. For now, they are still called `hidden_states` in this PR.

Comments:
_The documentation is not available anymore as the PR was closed or merged._