noamaanMulla-03/transformers-issues huggingface dataset:
RoBERTa convert-script modified to support mapping of bpe tokens
Author: vesteinn
Pull request: yes

# Reordering of embeddings possible when converting roberta with dict file

When RoBERTa is trained using fairseq the preprocessing pipeline bpe encodes tokens and stores a mapping of token ids to bpe-token ids in a dict.txt file. This is mostly just a re-ordering of the tokens (potentially with a few missing ones if they were not found in the training data). The earlier version of the conversion script maps the fairseq model directly creating a need for downstream processing to recover the original tokens, this removes that need by allowing a shuffling of the embedding tensors using the dict.txt file. This enables e.g. the "fill-mask" pipeline out of the box on a converted model.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests? 


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

@julien-c (since prominent in blame)
 

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
output probabilities of generated sequences in generate function
Author: yuvalkirstain
Pull request: no
Labels: name: wontfix

# 🚀 Feature request

output probabilities of generated sequences in generate function (generation utils)

thank you so much! :)

Comments:
Duplicate of https://github.com/huggingface/transformers/issues/3891

Actually, those issues are different and we should probably provide this functionality!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
how can i run gpt2 model on tf serving ?
Author: yiyele
Pull request: no


Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Same probability from fine-tuning custom pre-trained LM
Author: yuanbit
Pull request: no

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     
     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question 
     that you posted.
     -->

## Details
<!-- Description of your issue -->

I have a dataset with QA sentence pairs. I pre-trained a bert model from scractch using run_language_modeling.py where I concatenated the QA pairs and trained the dataset line_by_line.

I tried to fine-tune the custom pre-trained model, but am getting the same probability output for different inputs. I also tried to reduce the learning rate, but the constant probability problem remains.

What could I be doing wrong?

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**: 

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Transformers for Tabular data extraction - e.g., wikitables
Author: ghost
Pull request: no

Hi Team,
Can you please let us know if Transformers can be used to extract information from tabular data. Example is - [https://demo.allennlp.org/wikitables-parser](https://demo.allennlp.org/wikitables-parser) . WikiTables is the dataset.  

Example questions can be: show me all students who got marks greater than 40%   

Wondering if BERT or any other SOTA transformer technology can be leveraged to solve this NLP problem

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Clarify description of the is_split_into_words argument
Author: kstathou
Pull request: yes

# What does this PR do?
Clarifies the description of the `is_split_into_words` argument which is used in the tokenizers. 

Closes #11333 

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Issue: #11333 
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests? No additional tests were needed, I only clarified docs.


## Who can review?
I initially discussed this with @LysandreJik in #11333. I think @sgugger can review it too!

Thank you for taking the time to review this! 

NB the failed CI test seems unrelated.

Comments:
Thank you for the feedback!
Potential bug: Tokens with punctuation are re-tokenized although I've set `is_split_into_words=True`
Author: kstathou
Pull request: no

## Environment info
- `transformers` version: 4.5.1
- Platform: Linux-4.19.112+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.10
- PyTorch version (GPU?): 1.8.1+cu101 (False)
- Tensorflow version (GPU?): 2.4.1 (False)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
tokenizers: Hi @LysandreJik!

## Information
I am working on a token classification task where my input is in the following format: 

```
texts = [['Foo', 'bar', '.'], ['Hello', 'world', '.']]
tags = [['B-ENT, 'I-ENT', 'O'], ['O', 'O, 'O']]
```

- **Model:** `bert-large-cased`
- **Tokenizer:** `BertTokenizerFast`. I align tokens with their tags as in this [tutorial](https://huggingface.co/transformers/custom_datasets.html#token-classification-with-w-nut-emerging-entities).

## Problem
Although I've set `is_split_into_words=True` in the tokenizer, tokens containing punctuation are tokenized.

## To reproduce
I reproduced the issue in this [Google Colab notebook](https://colab.research.google.com/drive/1mNJ-T6kOaC5_a8C3T_qCXBNFP5WYC5oi?usp=sharing).

## Expected behavior
Since I've set `is_split_into_words=True`, I would expect the tokenizer to keep the tokens as they are and split them into subwords with `##`. For example, if a token is `'foo(bar'`, I would expect it to stay that way, instead of being split into `['foo', '(', 'bar']`.

Thanks a lot for reading the issue! 

Comments:
The `is_split_into_words` should be set to skip pre-tokenization (splitting on whitespace), not tokenization. This flag should be set to `True` if you have split your text into individual words, and you're now looking to have each word split into tokens and converted to IDs.

This seems to be unclear from the documentation, we'll work on improving this.

Makes sense, thank you for the clarification! I'd be happy to work on this if needed.

Yes, we would welcome such a contribution! I guess we would need to find all occurrences of that `is_split_into_words` parameter and clarify that pre-tokenization isn't tokenization as one could expect. 

We would gladly welcome a PR!

Great, I will work on this next week! Thanks again for the help!
[Makefile] make sure to test against the local checkout
Author: stas00
Pull request: yes

Currently some scripts in `Makefile` run against the pre-installed `transformers` rather than the checkout it's supposed to test. This PR fixes that by setting ` PYTHONPATH="src"`.

I had to fix that as I was getting at the end of `make fixup`:
```
python utils/check_repo.py
2021-04-25 21:54:53.850434: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
Checking all models are properly tested.
Traceback (most recent call last):
  File "utils/check_repo.py", line 481, in <module>
    check_repo_quality()
  File "utils/check_repo.py", line 473, in check_repo_quality
    check_all_models_are_tested()
  File "utils/check_repo.py", line 233, in check_all_models_are_tested
    modules = get_model_modules()
  File "utils/check_repo.py", line 147, in get_model_modules
    modeling_module = getattr(model_module, submodule)
  File "/mnt/nvme1/code/huggingface/transformers-master/src/transformers/file_utils.py", line 1666, in __getattr__
    value = self._get_module(name)
  File "/mnt/nvme1/code/huggingface/transformers-master/src/transformers/models/albert/__init__.py", line 120, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
  File "/home/stas/anaconda3/envs/py38-pt18/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/mnt/nvme1/code/huggingface/transformers-master/src/transformers/models/albert/modeling_tf_albert.py", line 43, in <module>
    from ...modeling_tf_utils import (
  File "src/transformers/modeling_tf_utils.py", line 32, in <module>
    from .file_utils import (
ImportError: cannot import name 'PushToHubMixin' from 'transformers.file_utils' (/mnt/nvme1/code/huggingface/transformers-master/src/transformers/file_utils.py)
```

The errors are from the pre-installed `transformers` and not the clone I'm working on.

The CI failures are unrelated - this can be safely merged.

@sgugger 


Comments:
Activating gradient checkpointing
Author: ShivanshuPurohit
Pull request: no

Working on a script which uses `GPTNeoForCausalLM` as the model object. As I understand it, gradient checkpointing requires checkpointing every layer. How do I change `GPTNeoForCausalLM` to incorporate gradient checkpointing, given that it doesn't show the layers explicitly but rather uses
```
self.transformer = GPTNeoModel(config)
self.lm_head = nn.Linear(config.hidden_size, config.vocab_size, bias=False)
```
PS. working on [visua-grounding](https://github.com/EleutherAI/visual-grounding/tree/main) with which @stas00 has already been a huge help, and knows the issues.

Comments:
In some places I think "activation checkpointing" is referred to as "gradient checkpointing". The former sounds more logical as it's activations that aren't being saved.  So it's https://pytorch.org/docs/stable/checkpoint.html.

It should be already there, as you can see the `GPTNeoModel` has it setup:

https://github.com/huggingface/transformers/blob/bc2571e61c985ec82819cf01ad038342771c94d0/src/transformers/models/gpt_neo/modeling_gpt_neo.py#L834

Ah. Thanks for the clarification.
update QuickTour docs to reflect model output object
Author: hamelsmu
Pull request: yes

Currently, the [Quick tour](https://huggingface.co/transformers/quicktour.html#) docs shows model output as tuples when you print them out.  In the current version of 🤗, the user sees an object that inherits from the `ModelOutput` class.  Yes, you can still access this object as a tuple, but this might be confusing for many readers, especially since this is the very first document that many people see when using 🤗 .

This PR does the following things:

1. Changes code examples in the Quick Tour to show the output object, not the tuple.  
2. Minor modification in the `Model Output` doc as _both_ PyTorch and Tensorflow models return an object that is an instance of a subclass of `ModelOutput`.
 
@sgugger

P.S. I am planning to go through all of the documentation very carefully like this, please let me know if there is anything along these lines that I can pay more attention to that is needed. 


Comments:
I might need some help or tips with figuring out why styling check CI is failing.  I tried to debug but it is not clear to me what is wrong

Thanks, @sgugger that worked

Thank *you* for the fixes :-)
Finish Making Quick Tour respect the model object
Author: hamelsmu
Pull request: yes

This PR makes the following changes

1.  As a follow up to #11462, finish correcting places where the tuple is mentioned instead of the model object.
2. You must import `AutoModel` as well as `TFAutoModel` for the tutorial to run correctly. 
3.  Cleaned up some language for readability.

@sgugger 

Comments:
Load a T5ForConditionalGeneration's encoder into a T5Model
Author: Palipoor
Pull request: no

Hi,
I know that T5ForConditionalGeneration is a T5Model with decoding. I've got a T5ForConditionalGeneration and that I've fine-tuned on a seq2seq task and now I want to use its T5 encoder to initialize the parameters in a T5Model( to further train it on some other task.) I read the code and I didn't understand what should I do. Can you please help me?

Comments:
Hi @Palipoor, This might do the trick,

```python
first_model = T5ForConditionalGeneration.from_pretrained('your_finetuned_model')
second_model = T5Model.from_pretrained('t5-small') # assuming 'your_finetuned_model' is t5-small

# get first model's encoder weights
first_model_encoder_state_dict = first_model.encoder.state_dict()

# load first model's encoder weights into second_model's encoder
second_model.encoder.load_state_dict(first_model_encoder_state_dict)
```

@patrickvonplaten  can you take a look ?

@patil-suraj has the correct idea I think. You can even make it easier by just doing

```python
t5_model_no_lm_head = T5.from_pretrained("<path_to_t5_for_cond_generation>")  # this will load all weighs that are present in both models. So It will just skip the lm head weighs
```

You can verify that this works by doing the following:

```python 
t5_model_with_lm_head = T5ForConditionalGeneration.from_pretrained('t5-small')
t5_model_with_lm_head.save_pretrained("./")
t5_model_no_lm_head = T5Model.from_pretrained("./")
```

@patrickvonplaten @patil-suraj Thank you both for your resposnes! 

Hi All,
So I have been trying to load a pre-tuned T5 model and then predict a sentence. I have been following this tutorial :
https://www.geeksforgeeks.org/text-to-text-transfer-transformer-in-data-augmentation/

And I have been able to pretune and save the model in a folder. Now I cant seem to know how to load it back and predict a given sentence. I have done the steps suggested by @patil-suraj but then how to use this configuration to do a prediction.

Grateful if you could help. Thanks :)
The performance of the huggingface QA model depend on the order in which it loads
Author: kaka-42
Pull request: no

- `transformers` version: 4.4.2
- Python version: 3.7

I am implementing a paper that I read based on the Question Answering code "run_qa.py" on huggingface.
I added a few layer in the ELECTRA, and I trained and saved only the parameters for the added layer.

when I evaluate, I load that parameters and the rest were initialized by parameters of the pre-trained ELECTRA model.

```
def load_cda_qa_model(args, phase, checkpoint=None):
    # assert phase == 'train' or phase == 'eval'
    config = CONFIG_CLASSES[args.model_type].from_pretrained(args.model_name_or_path)

    model = MODEL_FOR_QUESTION_ANSWERING[args.model_type].from_pretrained(checkpoint)
    tmp_electra = MODEL_FOR_QUESTION_ANSWERING['electra'].from_pretrained(args.model_name_or_path, config=config)

    electra_state_dict = tmp_electra.state_dict()
    model_state_dict = model.state_dict()

    for electra_key, electra_value in electra_state_dict.items():
        model_state_dict[electra_key] = electra_value

    model.load_state_dict(model_state_dict)

    return model
```
the results of two cases are:
## case 1
![1](https://user-images.githubusercontent.com/63441709/116337637-66f45400-a815-11eb-9635-33c78a42f9bd.JPG)

## case 2
![2](https://user-images.githubusercontent.com/63441709/116337644-678cea80-a815-11eb-8167-0dad391d82c4.JPG)

What I want to ask here is why the results change when the order of writing in the red and yellow parts seems to be no difference in code flow.




Comments:
Hi there,

Please use the [forum](https://discuss.huggingface.co/) to ask these types of questions, use issues to report bugs or feature requests.

Thanks!

> Hi there,
> 
> Please use the [forum](https://discuss.huggingface.co/) to ask these types of questions, use issues to report bugs or feature requests.
> 
> Thanks!

Ok I will. Thank you!
Add GPT-Neo
Author: aolko
Pull request: no
Labels: name: New model

# 🌟 New model addition

Please add GPT-Neo

## Model description

> GPT-Neo is the code name for a series of transformer-based language models loosely styled around the GPT architecture that Eleuther AI plans to train and open source. Eleuther AI's primary goal is to replicate a GPT-3 sized model and open source it to the public, for free.

<!-- Important information -->

## Open source status

* [x] the model implementation is available: [Repo](https://github.com/EleutherAI/gpt-neo)
* [x] the model weights are available: [Download](https://the-eye.eu/eleuther_staging/gptneo-release/) (1.3B & 2.7B)
* [x] who are the authors: @sdtblck, @leogao2, @lucidrains, @ConnorJL, @StellaAthena & [others](https://github.com/EleutherAI)

Somewhat related to #4658, #4679, especially [this](https://github.com/huggingface/transformers/issues/4658#issuecomment-754247106)

Comments:
removing runs folders 
Author: rabeehkarimimahabadi
Pull request: no
Labels: name: wontfix

Hi
Do you know where in the codes when training finetune_trainer.py a folder "runs" created? could it be moved to output_dir? so this is always created in the directory running codes than output_dir
thanks

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Hi,

You can move the path of the "**_runs/_**" folder by adding `logging_dir` to your `TrainingArguments` such like this:

```python
training_args = TrainingArguments(
    output_dir=OUTPUT_PATH,
    overwrite_output_dir=True,
    logging_dir=OUTPUT_PATH + "/logs",
   ...
)
```
Update min versions in README and add Flax
Author: sgugger
Pull request: yes

# What does this PR do?

This PR adapts the minimum versions of each backend (PyTorch was still at 1.0 and TensorFlow at 2.0), removes mention of TensorFlow 2.0 to just say TensorFlow (I think it's safe now!) and adds Jax as an official backend since we have worked the API a bit more.

Fixes #11422

Comments:
Update `PreTrainedTokenizerBase` to check/handle batch length for `text_pair` parameter
Author: hamelsmu
Pull request: yes

Consider the following example:

```py
from transformers import AutoTokenizer, AutoModelForQuestionAnswering
import torch
tokenizer = AutoTokenizer.from_pretrained("bert-large-uncased-whole-word-masking-finetuned-squad")
text = r"""
🤗 Transformers (formerly known as pytorch-transformers and pytorch-pretrained-bert) provides general-purpose
architectures (BERT, GPT-2, RoBERTa, XLM, DistilBert, XLNet…) for Natural Language Understanding (NLU) and Natural
Language Generation (NLG) with over 32+ pretrained models in 100+ languages and deep interoperability between
TensorFlow 2.0 and PyTorch.
"""
questions = [
    "How many pretrained models are available in 🤗 Transformers?",
    "What does 🤗 Transformers provide?",
    "🤗 Transformers provides interoperability between which frameworks?"
]

inp = tokenizer(text=questions, 
                text_pair=text, 
                add_special_tokens=True, 
                padding=True, 
                truncation=True,
                return_tensors="pt")

print(inp.input_ids.shape)
```

**The error in the above example is that the parameter `text_pair` is a string, but is supposed to be a `List[str]` to match the batch size of `text`.**

Currently, this silently fails because when  `text_pair` is a string it is treated as an iterable causing `zip(text, text_pair)` to erroneously build the wrong inputs to the model.  This PR adds the following:

1.  If `text_pair` is a string but the user passes in a batch of `text`, we convert the input for them automatically (For example when you want to ask multiple questions of the same passage). 
2.  Adds error checking to see if the batch length of `text` matches the batch length of `text_pair` ONLY when a batch of inputs is used.


@LysandreJik @sgugger








Comments:
Thanks! Failure in the tests is unrelated (some connection problem), so merging.