Repository: huggingface/transformers
Pull request #16507 (2022-03-31 03:20:30)

[WIP] Add Doc Test GPT-J
# What does this PR do?

Fixes the broken doc tests for GPT-J
Apart of the documentation sprint work.

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github [issue] (https://github.com/huggingface/transformers/issues/16292)? Please add a link
      to it if that's the case.
  
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?

## Who can review?
@ydshieh 
@sgugger
 -->


HuggingFaceDocBuilderDev: The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_16507). All of your documentation changes will be reflected on that endpoint.
ydshieh: @ArEnSc 

Thank you for working on GPT-J.
Maybe you can just put any expected values as empty string. Once other parts are done,
please ping me and I will try to get and put the expected values :)
ArEnSc: > @ArEnSc
> 
> Thank you for working on GPT-J. Maybe you can just put any expected value as empty string. Once other parts are done, please ping me and I will try to get and put the expected values :)

yep please do and let me know and we can close this =)
ydshieh: Hi, @ArEnSc 

After discussing with the team, we found that there is no real checkpoints for finetuned GPT-J model on downstream tasks. (There is one for text seq. classification, but it is for Korean language).

If you still want to work on this GPT-J doctest, the best we could do for now is to use a tiny model (that is created for testing purpose)

https://huggingface.co/hf-internal-testing/tiny-random-gptj

With this one, there won't be any OOM issue. Let me know if you want to continue, and if so, don't hesitate if you have any issue using this tiny model checkpoint.

Thanks!
ArEnSc: @ydshieh 
I gave it a shot, it seems to be non deterministic on the outputs for QA and Sequence clasification so how would I disable those test for this model, other than removing the decorator? is there a specified way? Thanks!
```
platform linux -- Python 3.8.10, pytest-7.1.1, pluggy-1.0.0 -- /home/tensor/mlvenv/bin/python3
cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/tensor/transformers/.hypothesis/examples')
rootdir: /home/tensor/transformers, configfile: setup.cfg
plugins: forked-1.4.0, requests-mock-1.9.3, timeout-2.1.0, xdist-2.5.0, dash-2.3.0, hydra-core-1.1.1, hypothesis-6.39.5, datadir-1.3.1, regressions-2.3.1
collected 4 items                                                                                                                                            

src/transformers/models/gptj/modeling_gptj.py::transformers.models.gptj.modeling_gptj.GPTJForCausalLM.forward Some weights of the model checkpoint at hf-internal-testing/tiny-random-gptj were not used when initializing GPTJForCausalLM: ['h.1.mlp.fc_in.bias', 'h.1.attn.out_proj.weight', 'h.3.attn.k_proj.weight', 'h.2.ln_1.weight', 'h.1.attn.masked_bias', 'h.4.attn.out_proj.weight', 'h.1.mlp.fc_in.weight', 'h.0.mlp.fc_out.weight', 'h.4.attn.q_proj.weight', 'h.0.attn.v_proj.weight', 'h.0.mlp.fc_in.bias', 'h.4.attn.bias', 'h.0.mlp.fc_out.bias', 'h.3.attn.bias', 'h.0.attn.k_proj.weight', 'h.1.ln_1.weight', 'h.1.attn.v_proj.weight', 'h.2.attn.out_proj.weight', 'h.2.attn.bias', 'h.1.ln_1.bias', 'h.0.mlp.fc_in.weight', 'h.2.mlp.fc_out.bias', 'h.3.attn.out_proj.weight', 'h.4.attn.k_proj.weight', 'h.3.attn.masked_bias', 'h.0.attn.out_proj.weight', 'h.2.ln_1.bias', 'h.3.mlp.fc_in.bias', 'h.2.mlp.fc_out.weight', 'h.3.mlp.fc_out.bias', 'h.2.attn.k_proj.weight', 'h.3.attn.q_proj.weight', 'wte.weight', 'h.2.mlp.fc_in.bias', 'h.3.ln_1.weight', 'h.4.attn.v_proj.weight', 'h.0.attn.masked_bias', 'h.1.attn.q_proj.weight', 'h.3.attn.v_proj.weight', 'h.4.mlp.fc_out.weight', 'h.4.mlp.fc_out.bias', 'score.weight', 'ln_f.bias', 'h.1.attn.bias', 'h.0.attn.q_proj.weight', 'h.1.mlp.fc_out.bias', 'h.2.attn.masked_bias', 'h.1.mlp.fc_out.weight', 'h.3.mlp.fc_in.weight', 'h.2.attn.v_proj.weight', 'h.4.mlp.fc_in.bias', 'h.4.attn.masked_bias', 'h.4.ln_1.bias', 'h.0.attn.bias', 'h.1.attn.k_proj.weight', 'ln_f.weight', 'h.3.ln_1.bias', 'h.0.ln_1.weight', 'h.2.mlp.fc_in.weight', 'h.3.mlp.fc_out.weight', 'h.0.ln_1.bias', 'h.4.ln_1.weight', 'h.2.attn.q_proj.weight', 'h.4.mlp.fc_in.weight']
- This IS expected if you are initializing GPTJForCausalLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing GPTJForCausalLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
PASSED
src/transformers/models/gptj/modeling_gptj.py::transformers.models.gptj.modeling_gptj.GPTJForQuestionAnswering.forward Some weights of the model checkpoint at hf-internal-testing/tiny-random-gptj were not used when initializing GPTJForQuestionAnswering: ['h.1.mlp.fc_in.bias', 'h.1.attn.out_proj.weight', 'h.3.attn.k_proj.weight', 'h.2.ln_1.weight', 'h.1.attn.masked_bias', 'h.4.attn.out_proj.weight', 'h.1.mlp.fc_in.weight', 'h.0.mlp.fc_out.weight', 'h.4.attn.q_proj.weight', 'h.0.attn.v_proj.weight', 'h.0.mlp.fc_in.bias', 'h.4.attn.bias', 'h.0.mlp.fc_out.bias', 'h.3.attn.bias', 'h.0.attn.k_proj.weight', 'h.1.ln_1.weight', 'h.1.attn.v_proj.weight', 'h.2.attn.out_proj.weight', 'h.2.attn.bias', 'h.1.ln_1.bias', 'h.0.mlp.fc_in.weight', 'h.2.mlp.fc_out.bias', 'h.3.attn.out_proj.weight', 'lm_head.bias', 'h.4.attn.k_proj.weight', 'h.3.attn.masked_bias', 'h.0.attn.out_proj.weight', 'h.2.ln_1.bias', 'h.3.mlp.fc_in.bias', 'h.2.mlp.fc_out.weight', 'h.3.mlp.fc_out.bias', 'h.2.attn.k_proj.weight', 'h.3.attn.q_proj.weight', 'wte.weight', 'h.2.mlp.fc_in.bias', 'h.3.ln_1.weight', 'h.4.attn.v_proj.weight', 'h.0.attn.masked_bias', 'h.1.attn.q_proj.weight', 'h.3.attn.v_proj.weight', 'h.4.mlp.fc_out.weight', 'h.4.mlp.fc_out.bias', 'score.weight', 'ln_f.bias', 'lm_head.weight', 'h.1.attn.bias', 'h.0.attn.q_proj.weight', 'h.1.mlp.fc_out.bias', 'h.2.attn.masked_bias', 'h.1.mlp.fc_out.weight', 'h.3.mlp.fc_in.weight', 'h.2.attn.v_proj.weight', 'h.4.mlp.fc_in.bias', 'h.4.attn.masked_bias', 'h.4.ln_1.bias', 'h.0.attn.bias', 'h.1.attn.k_proj.weight', 'ln_f.weight', 'h.3.ln_1.bias', 'h.0.ln_1.weight', 'h.2.mlp.fc_in.weight', 'h.3.mlp.fc_out.weight', 'h.0.ln_1.bias', 'h.4.ln_1.weight', 'h.2.attn.q_proj.weight', 'h.4.mlp.fc_in.weight']
- This IS expected if you are initializing GPTJForQuestionAnswering from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing GPTJForQuestionAnswering from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of GPTJForQuestionAnswering were not initialized from the model checkpoint at hf-internal-testing/tiny-random-gptj and are newly initialized: ['qa_outputs.bias', 'qa_outputs.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
FAILED
src/transformers/models/gptj/modeling_gptj.py::transformers.models.gptj.modeling_gptj.GPTJForSequenceClassification.forward FAILED
src/transformers/models/gptj/modeling_gptj.py::transformers.models.gptj.modeling_gptj.GPTJModel.forward Some weights of the model checkpoint at hf-internal-testing/tiny-random-gptj were not used when initializing GPTJModel: ['h.1.mlp.fc_in.bias', 'h.1.attn.out_proj.weight', 'h.3.attn.k_proj.weight', 'h.2.ln_1.weight', 'h.1.attn.masked_bias', 'h.4.attn.out_proj.weight', 'h.1.mlp.fc_in.weight', 'h.0.mlp.fc_out.weight', 'h.4.attn.q_proj.weight', 'h.0.attn.v_proj.weight', 'h.0.mlp.fc_in.bias', 'h.4.attn.bias', 'h.0.mlp.fc_out.bias', 'h.3.attn.bias', 'h.0.attn.k_proj.weight', 'h.1.ln_1.weight', 'h.1.attn.v_proj.weight', 'h.2.attn.out_proj.weight', 'h.2.attn.bias', 'h.1.ln_1.bias', 'h.0.mlp.fc_in.weight', 'h.2.mlp.fc_out.bias', 'h.3.attn.out_proj.weight', 'lm_head.bias', 'h.4.attn.k_proj.weight', 'h.3.attn.masked_bias', 'h.0.attn.out_proj.weight', 'h.2.ln_1.bias', 'h.3.mlp.fc_in.bias', 'h.2.mlp.fc_out.weight', 'h.3.mlp.fc_out.bias', 'h.2.attn.k_proj.weight', 'h.3.attn.q_proj.weight', 'wte.weight', 'h.2.mlp.fc_in.bias', 'h.3.ln_1.weight', 'h.4.attn.v_proj.weight', 'h.0.attn.masked_bias', 'h.1.attn.q_proj.weight', 'h.3.attn.v_proj.weight', 'h.4.mlp.fc_out.weight', 'h.4.mlp.fc_out.bias', 'score.weight', 'ln_f.bias', 'lm_head.weight', 'h.1.attn.bias', 'h.0.attn.q_proj.weight', 'h.1.mlp.fc_out.bias', 'h.2.attn.masked_bias', 'h.1.mlp.fc_out.weight', 'h.3.mlp.fc_in.weight', 'h.2.attn.v_proj.weight', 'h.4.mlp.fc_in.bias', 'h.4.attn.masked_bias', 'h.4.ln_1.bias', 'h.0.attn.bias', 'h.1.attn.k_proj.weight', 'ln_f.weight', 'h.3.ln_1.bias', 'h.0.ln_1.weight', 'h.2.mlp.fc_in.weight', 'h.3.mlp.fc_out.weight', 'h.0.ln_1.bias', 'h.4.ln_1.weight', 'h.2.attn.q_proj.weight', 'h.4.mlp.fc_in.weight']
- This IS expected if you are initializing GPTJModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing GPTJModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
PASSED

========================================================================== FAILURES ==========================================================================
_____________________________________ [doctest] transformers.models.gptj.modeling_gptj.GPTJForQuestionAnswering.forward ______________________________________
1129 
1130     >>> inputs = tokenizer(question, text, return_tensors="pt")
1131     >>> with torch.no_grad():
1132     ...     outputs = model(**inputs)
1133 
1134     >>> answer_start_index = outputs.start_logits.argmax()
1135     >>> answer_end_index = outputs.end_logits.argmax()
1136 
1137     >>> predict_answer_tokens = inputs.input_ids[0, answer_start_index : answer_end_index + 1]
1138     >>> tokenizer.decode(predict_answer_tokens)
Expected:
    im Henson?Jim Henson was a nice pupp
Got:
    ' nice'

/home/tensor/transformers/src/transformers/models/gptj/modeling_gptj.py:1138: DocTestFailure
1140 
1141     ```
1142 
1143     ```python
1144     >>> # target is "nice puppet"
1145     >>> target_start_index, target_end_index = torch.tensor([14]), torch.tensor([15])
1146 
1147     >>> outputs = model(**inputs, start_positions=target_start_index, end_positions=target_end_index)
1148     >>> loss = outputs.loss
1149     >>> round(loss.item(), 2)
Expected:
    3.11
Got:
    3.16

/home/tensor/transformers/src/transformers/models/gptj/modeling_gptj.py:1149: DocTestFailure
___________________________________ [doctest] transformers.models.gptj.modeling_gptj.GPTJForSequenceClassification.forward ___________________________________
990           heads.
991   
992     Example of single-label classification:
993 
994     ```python
995     >>> import torch
996     >>> from transformers import GPT2Tokenizer, GPTJForSequenceClassification
997 
998     >>> tokenizer = GPT2Tokenizer.from_pretrained("hf-internal-testing/tiny-random-gptj")
999     >>> model = GPTJForSequenceClassification.from_pretrained("hf-internal-testing/tiny-random-gptj")
UNEXPECTED EXCEPTION: RuntimeError('Error(s) in loading state_dict for GPTJForSequenceClassification:\n\tsize mismatch for score.weight: copying a param with shape torch.Size([2, 512]) from checkpoint, the shape in current model is torch.Size([2, 32]).')
Traceback (most recent call last):
  File "/usr/lib/python3.8/doctest.py", line 1336, in __run
    exec(compile(example.source, filename, "single",
  File "<doctest transformers.models.gptj.modeling_gptj.GPTJForSequenceClassification.forward[3]>", line 1, in <module>
  File "/home/tensor/transformers/src/transformers/modeling_utils.py", line 1882, in from_pretrained
    model, missing_keys, unexpected_keys, mismatched_keys, error_msgs = cls._load_pretrained_model(
  File "/home/tensor/transformers/src/transformers/modeling_utils.py", line 2045, in _load_pretrained_model
    raise RuntimeError(f"Error(s) in loading state_dict for {model.__class__.__name__}:\n\t{error_msg}")
RuntimeError: Error(s) in loading state_dict for GPTJForSequenceClassification:
        size mismatch for score.weight: copying a param with shape torch.Size([2, 512]) from checkpoint, the shape in current model is torch.Size([2, 32]).
/home/tensor/transformers/src/transformers/models/gptj/modeling_gptj.py:999: UnexpectedException
994     ```python
995     >>> import torch
996     >>> from transformers import GPT2Tokenizer, GPTJForSequenceClassification
997 
998     >>> tokenizer = GPT2Tokenizer.from_pretrained("hf-internal-testing/tiny-random-gptj")
999     >>> model = GPTJForSequenceClassification.from_pretrained("hf-internal-testing/tiny-random-gptj")
1000 
1001     >>> inputs = tokenizer("Hello, my dog is cute", return_tensors="pt")
1002 
1003     >>> with torch.no_grad():
UNEXPECTED EXCEPTION: NameError("name 'model' is not defined")
Traceback (most recent call last):
  File "/usr/lib/python3.8/doctest.py", line 1336, in __run
    exec(compile(example.source, filename, "single",
  File "<doctest transformers.models.gptj.modeling_gptj.GPTJForSequenceClassification.forward[5]>", line 2, in <module>
NameError: name 'model' is not defined
/home/tensor/transformers/src/transformers/models/gptj/modeling_gptj.py:1003: UnexpectedException
997 
998     >>> tokenizer = GPT2Tokenizer.from_pretrained("hf-internal-testing/tiny-random-gptj")
999     >>> model = GPTJForSequenceClassification.from_pretrained("hf-internal-testing/tiny-random-gptj")
1000 
1001     >>> inputs = tokenizer("Hello, my dog is cute", return_tensors="pt")
1002 
1003     >>> with torch.no_grad():
1004     ...     logits = model(**inputs).logits
1005 
1006     >>> predicted_class_id = logits.argmax().item()
UNEXPECTED EXCEPTION: NameError("name 'logits' is not defined")
Traceback (most recent call last):
  File "/usr/lib/python3.8/doctest.py", line 1336, in __run
    exec(compile(example.source, filename, "single",
  File "<doctest transformers.models.gptj.modeling_gptj.GPTJForSequenceClassification.forward[6]>", line 1, in <module>
NameError: name 'logits' is not defined
/home/tensor/transformers/src/transformers/models/gptj/modeling_gptj.py:1006: UnexpectedException
998     >>> tokenizer = GPT2Tokenizer.from_pretrained("hf-internal-testing/tiny-random-gptj")
999     >>> model = GPTJForSequenceClassification.from_pretrained("hf-internal-testing/tiny-random-gptj")
1000 
1001     >>> inputs = tokenizer("Hello, my dog is cute", return_tensors="pt")
1002 
1003     >>> with torch.no_grad():
1004     ...     logits = model(**inputs).logits
1005 
1006     >>> predicted_class_id = logits.argmax().item()
1007     >>> model.config.id2label[predicted_class_id]
UNEXPECTED EXCEPTION: NameError("name 'model' is not defined")
Traceback (most recent call last):
  File "/usr/lib/python3.8/doctest.py", line 1336, in __run
    exec(compile(example.source, filename, "single",
  File "<doctest transformers.models.gptj.modeling_gptj.GPTJForSequenceClassification.forward[7]>", line 1, in <module>
NameError: name 'model' is not defined
/home/tensor/transformers/src/transformers/models/gptj/modeling_gptj.py:1007: UnexpectedException
1005 
1006     >>> predicted_class_id = logits.argmax().item()
1007     >>> model.config.id2label[predicted_class_id]
1008     im Henson?Jim Henson was a nice pupp
1009 
1010     ```
1011 
1012     ```python
1013     >>> # To train a model on `num_labels` classes, you can pass `num_labels=num_labels` to `.from_pretrained(...)`
1014     >>> num_labels = len(model.config.id2label)
UNEXPECTED EXCEPTION: NameError("name 'model' is not defined")
Traceback (most recent call last):
  File "/usr/lib/python3.8/doctest.py", line 1336, in __run
    exec(compile(example.source, filename, "single",
  File "<doctest transformers.models.gptj.modeling_gptj.GPTJForSequenceClassification.forward[8]>", line 1, in <module>
NameError: name 'model' is not defined
/home/tensor/transformers/src/transformers/models/gptj/modeling_gptj.py:1014: UnexpectedException
1006     >>> predicted_class_id = logits.argmax().item()
1007     >>> model.config.id2label[predicted_class_id]
1008     im Henson?Jim Henson was a nice pupp
1009 
1010     ```
1011 
1012     ```python
1013     >>> # To train a model on `num_labels` classes, you can pass `num_labels=num_labels` to `.from_pretrained(...)`
1014     >>> num_labels = len(model.config.id2label)
1015     >>> model = GPTJForSequenceClassification.from_pretrained("hf-internal-testing/tiny-random-gptj", num_labels=num_labels)
UNEXPECTED EXCEPTION: NameError("name 'num_labels' is not defined")
Traceback (most recent call last):
  File "/usr/lib/python3.8/doctest.py", line 1336, in __run
    exec(compile(example.source, filename, "single",
  File "<doctest transformers.models.gptj.modeling_gptj.GPTJForSequenceClassification.forward[9]>", line 1, in <module>
NameError: name 'num_labels' is not defined
/home/tensor/transformers/src/transformers/models/gptj/modeling_gptj.py:1015: UnexpectedException
1009 
1010     ```
1011 
1012     ```python
1013     >>> # To train a model on `num_labels` classes, you can pass `num_labels=num_labels` to `.from_pretrained(...)`
1014     >>> num_labels = len(model.config.id2label)
1015     >>> model = GPTJForSequenceClassification.from_pretrained("hf-internal-testing/tiny-random-gptj", num_labels=num_labels)
1016 
1017     >>> labels = torch.tensor(1)
1018     >>> loss = model(**inputs, labels=labels).loss
UNEXPECTED EXCEPTION: NameError("name 'model' is not defined")
Traceback (most recent call last):
  File "/usr/lib/python3.8/doctest.py", line 1336, in __run
    exec(compile(example.source, filename, "single",
  File "<doctest transformers.models.gptj.modeling_gptj.GPTJForSequenceClassification.forward[11]>", line 1, in <module>
NameError: name 'model' is not defined
/home/tensor/transformers/src/transformers/models/gptj/modeling_gptj.py:1018: UnexpectedException
1010     ```
1011 
1012     ```python
1013     >>> # To train a model on `num_labels` classes, you can pass `num_labels=num_labels` to `.from_pretrained(...)`
1014     >>> num_labels = len(model.config.id2label)
1015     >>> model = GPTJForSequenceClassification.from_pretrained("hf-internal-testing/tiny-random-gptj", num_labels=num_labels)
1016 
1017     >>> labels = torch.tensor(1)
1018     >>> loss = model(**inputs, labels=labels).loss
1019     >>> round(loss.item(), 2)
UNEXPECTED EXCEPTION: NameError("name 'loss' is not defined")
Traceback (most recent call last):
  File "/usr/lib/python3.8/doctest.py", line 1336, in __run
    exec(compile(example.source, filename, "single",
  File "<doctest transformers.models.gptj.modeling_gptj.GPTJForSequenceClassification.forward[12]>", line 1, in <module>
NameError: name 'loss' is not defined
/home/tensor/transformers/src/transformers/models/gptj/modeling_gptj.py:1019: UnexpectedException
1022     ```
1023 
1024     Example of multi-label classification:
1025 
1026     ```python
1027     >>> import torch
1028     >>> from transformers import GPT2Tokenizer, GPTJForSequenceClassification
1029 
1030     >>> tokenizer = GPT2Tokenizer.from_pretrained("hf-internal-testing/tiny-random-gptj")
1031     >>> model = GPTJForSequenceClassification.from_pretrained("hf-internal-testing/tiny-random-gptj", problem_type="multi_label_classification")
UNEXPECTED EXCEPTION: RuntimeError('Error(s) in loading state_dict for GPTJForSequenceClassification:\n\tsize mismatch for score.weight: copying a param with shape torch.Size([2, 512]) from checkpoint, the shape in current model is torch.Size([2, 32]).')
Traceback (most recent call last):
  File "/usr/lib/python3.8/doctest.py", line 1336, in __run
    exec(compile(example.source, filename, "single",
  File "<doctest transformers.models.gptj.modeling_gptj.GPTJForSequenceClassification.forward[16]>", line 1, in <module>
  File "/home/tensor/transformers/src/transformers/modeling_utils.py", line 1882, in from_pretrained
    model, missing_keys, unexpected_keys, mismatched_keys, error_msgs = cls._load_pretrained_model(
  File "/home/tensor/transformers/src/transformers/modeling_utils.py", line 2045, in _load_pretrained_model
    raise RuntimeError(f"Error(s) in loading state_dict for {model.__class__.__name__}:\n\t{error_msg}")
RuntimeError: Error(s) in loading state_dict for GPTJForSequenceClassification:
        size mismatch for score.weight: copying a param with shape torch.Size([2, 512]) from checkpoint, the shape in current model is torch.Size([2, 32]).
/home/tensor/transformers/src/transformers/models/gptj/modeling_gptj.py:1031: UnexpectedException
1026     ```python
1027     >>> import torch
1028     >>> from transformers import GPT2Tokenizer, GPTJForSequenceClassification
1029 
1030     >>> tokenizer = GPT2Tokenizer.from_pretrained("hf-internal-testing/tiny-random-gptj")
1031     >>> model = GPTJForSequenceClassification.from_pretrained("hf-internal-testing/tiny-random-gptj", problem_type="multi_label_classification")
1032 
1033     >>> inputs = tokenizer("Hello, my dog is cute", return_tensors="pt")
1034 
1035     >>> with torch.no_grad():
UNEXPECTED EXCEPTION: NameError("name 'model' is not defined")
Traceback (most recent call last):
  File "/usr/lib/python3.8/doctest.py", line 1336, in __run
    exec(compile(example.source, filename, "single",
  File "<doctest transformers.models.gptj.modeling_gptj.GPTJForSequenceClassification.forward[18]>", line 2, in <module>
NameError: name 'model' is not defined
/home/tensor/transformers/src/transformers/models/gptj/modeling_gptj.py:1035: UnexpectedException
1029 
1030     >>> tokenizer = GPT2Tokenizer.from_pretrained("hf-internal-testing/tiny-random-gptj")
1031     >>> model = GPTJForSequenceClassification.from_pretrained("hf-internal-testing/tiny-random-gptj", problem_type="multi_label_classification")
1032 
1033     >>> inputs = tokenizer("Hello, my dog is cute", return_tensors="pt")
1034 
1035     >>> with torch.no_grad():
1036     ...     logits = model(**inputs).logits
1037 
1038     >>> predicted_class_id = logits.argmax().item()
UNEXPECTED EXCEPTION: NameError("name 'logits' is not defined")
Traceback (most recent call last):
  File "/usr/lib/python3.8/doctest.py", line 1336, in __run
    exec(compile(example.source, filename, "single",
  File "<doctest transformers.models.gptj.modeling_gptj.GPTJForSequenceClassification.forward[19]>", line 1, in <module>
NameError: name 'logits' is not defined
/home/tensor/transformers/src/transformers/models/gptj/modeling_gptj.py:1038: UnexpectedException
1030     >>> tokenizer = GPT2Tokenizer.from_pretrained("hf-internal-testing/tiny-random-gptj")
1031     >>> model = GPTJForSequenceClassification.from_pretrained("hf-internal-testing/tiny-random-gptj", problem_type="multi_label_classification")
1032 
1033     >>> inputs = tokenizer("Hello, my dog is cute", return_tensors="pt")
1034 
1035     >>> with torch.no_grad():
1036     ...     logits = model(**inputs).logits
1037 
1038     >>> predicted_class_id = logits.argmax().item()
1039     >>> model.config.id2label[predicted_class_id]
UNEXPECTED EXCEPTION: NameError("name 'model' is not defined")
Traceback (most recent call last):
  File "/usr/lib/python3.8/doctest.py", line 1336, in __run
    exec(compile(example.source, filename, "single",
  File "<doctest transformers.models.gptj.modeling_gptj.GPTJForSequenceClassification.forward[20]>", line 1, in <module>
NameError: name 'model' is not defined
/home/tensor/transformers/src/transformers/models/gptj/modeling_gptj.py:1039: UnexpectedException
1037 
1038     >>> predicted_class_id = logits.argmax().item()
1039     >>> model.config.id2label[predicted_class_id]
1040     im Henson?Jim Henson was a nice pupp
1041 
1042     ```
1043 
1044     ```python
1045     >>> # To train a model on `num_labels` classes, you can pass `num_labels=num_labels` to `.from_pretrained(...)`
1046     >>> num_labels = len(model.config.id2label)
UNEXPECTED EXCEPTION: NameError("name 'model' is not defined")
Traceback (most recent call last):
  File "/usr/lib/python3.8/doctest.py", line 1336, in __run
    exec(compile(example.source, filename, "single",
  File "<doctest transformers.models.gptj.modeling_gptj.GPTJForSequenceClassification.forward[21]>", line 1, in <module>
NameError: name 'model' is not defined
/home/tensor/transformers/src/transformers/models/gptj/modeling_gptj.py:1046: UnexpectedException
1038     >>> predicted_class_id = logits.argmax().item()
1039     >>> model.config.id2label[predicted_class_id]
1040     im Henson?Jim Henson was a nice pupp
1041 
1042     ```
1043 
1044     ```python
1045     >>> # To train a model on `num_labels` classes, you can pass `num_labels=num_labels` to `.from_pretrained(...)`
1046     >>> num_labels = len(model.config.id2label)
1047     >>> model = GPTJForSequenceClassification.from_pretrained("hf-internal-testing/tiny-random-gptj", num_labels=num_labels)
UNEXPECTED EXCEPTION: NameError("name 'num_labels' is not defined")
Traceback (most recent call last):
  File "/usr/lib/python3.8/doctest.py", line 1336, in __run
    exec(compile(example.source, filename, "single",
  File "<doctest transformers.models.gptj.modeling_gptj.GPTJForSequenceClassification.forward[22]>", line 1, in <module>
NameError: name 'num_labels' is not defined
/home/tensor/transformers/src/transformers/models/gptj/modeling_gptj.py:1047: UnexpectedException
1040     im Henson?Jim Henson was a nice pupp
1041 
1042     ```
1043 
1044     ```python
1045     >>> # To train a model on `num_labels` classes, you can pass `num_labels=num_labels` to `.from_pretrained(...)`
1046     >>> num_labels = len(model.config.id2label)
1047     >>> model = GPTJForSequenceClassification.from_pretrained("hf-internal-testing/tiny-random-gptj", num_labels=num_labels)
1048 
1049     >>> num_labels = len(model.config.id2label)
UNEXPECTED EXCEPTION: NameError("name 'model' is not defined")
Traceback (most recent call last):
  File "/usr/lib/python3.8/doctest.py", line 1336, in __run
    exec(compile(example.source, filename, "single",
  File "<doctest transformers.models.gptj.modeling_gptj.GPTJForSequenceClassification.forward[23]>", line 1, in <module>
NameError: name 'model' is not defined
/home/tensor/transformers/src/transformers/models/gptj/modeling_gptj.py:1049: UnexpectedException
1041 
1042     ```
1043 
1044     ```python
1045     >>> # To train a model on `num_labels` classes, you can pass `num_labels=num_labels` to `.from_pretrained(...)`
1046     >>> num_labels = len(model.config.id2label)
1047     >>> model = GPTJForSequenceClassification.from_pretrained("hf-internal-testing/tiny-random-gptj", num_labels=num_labels)
1048 
1049     >>> num_labels = len(model.config.id2label)
1050     >>> labels = torch.nn.functional.one_hot(torch.tensor([predicted_class_id]), num_classes=num_labels).to(
UNEXPECTED EXCEPTION: NameError("name 'predicted_class_id' is not defined")
Traceback (most recent call last):
  File "/usr/lib/python3.8/doctest.py", line 1336, in __run
    exec(compile(example.source, filename, "single",
  File "<doctest transformers.models.gptj.modeling_gptj.GPTJForSequenceClassification.forward[24]>", line 1, in <module>
NameError: name 'predicted_class_id' is not defined
/home/tensor/transformers/src/transformers/models/gptj/modeling_gptj.py:1050: UnexpectedException
1044     ```python
1045     >>> # To train a model on `num_labels` classes, you can pass `num_labels=num_labels` to `.from_pretrained(...)`
1046     >>> num_labels = len(model.config.id2label)
1047     >>> model = GPTJForSequenceClassification.from_pretrained("hf-internal-testing/tiny-random-gptj", num_labels=num_labels)
1048 
1049     >>> num_labels = len(model.config.id2label)
1050     >>> labels = torch.nn.functional.one_hot(torch.tensor([predicted_class_id]), num_classes=num_labels).to(
1051     ...     torch.float
1052     ... )
1053     >>> loss = model(**inputs, labels=labels).loss
UNEXPECTED EXCEPTION: NameError("name 'model' is not defined")
Traceback (most recent call last):
  File "/usr/lib/python3.8/doctest.py", line 1336, in __run
    exec(compile(example.source, filename, "single",
  File "<doctest transformers.models.gptj.modeling_gptj.GPTJForSequenceClassification.forward[25]>", line 1, in <module>
NameError: name 'model' is not defined
/home/tensor/transformers/src/transformers/models/gptj/modeling_gptj.py:1053: UnexpectedException
1045     >>> # To train a model on `num_labels` classes, you can pass `num_labels=num_labels` to `.from_pretrained(...)`
1046     >>> num_labels = len(model.config.id2label)
1047     >>> model = GPTJForSequenceClassification.from_pretrained("hf-internal-testing/tiny-random-gptj", num_labels=num_labels)
1048 
1049     >>> num_labels = len(model.config.id2label)
1050     >>> labels = torch.nn.functional.one_hot(torch.tensor([predicted_class_id]), num_classes=num_labels).to(
1051     ...     torch.float
1052     ... )
1053     >>> loss = model(**inputs, labels=labels).loss
1054     >>> loss.backward()  # doctest: +IGNORE_RESULT
UNEXPECTED EXCEPTION: NameError("name 'loss' is not defined")
Traceback (most recent call last):
  File "/usr/lib/python3.8/doctest.py", line 1336, in __run
    exec(compile(example.source, filename, "single",
  File "<doctest transformers.models.gptj.modeling_gptj.GPTJForSequenceClassification.forward[26]>", line 1, in <module>
NameError: name 'loss' is not defined
/home/tensor/transformers/src/transformers/models/gptj/modeling_gptj.py:1054: UnexpectedException
====================================================================== warnings summary ======================================================================
../mlvenv/lib/python3.8/site-packages/flatbuffers/compat.py:19
  /home/tensor/mlvenv/lib/python3.8/site-packages/flatbuffers/compat.py:19: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
    import imp

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================================================== short test summary info ===================================================================
FAILED src/transformers/models/gptj/modeling_gptj.py::transformers.models.gptj.modeling_gptj.GPTJForQuestionAnswering.forward
FAILED src/transformers/models/gptj/modeling_gptj.py::transformers.models.gptj.modeling_gptj.GPTJForSequenceClassification.forward
=========================================================== 2 failed, 2 passed, 1 warning in 6.75s ==========================================================
```
ydshieh: Hi, @ArEnSc  

Thank you for the effort.

I need to investigate first why it is non deterministic, and see if there is a way to fix this.
We strongly prefer not to disable doctests for some parts in a model.
(Otherwise, our team need to discuss to justify it and make the decision).

I will take a look in this soon!
ydshieh: Hi, @ArEnSc 

I uploaded the checkpoints

```
"ydshieh/tiny-random-gptj-for-sequence-classification"
"ydshieh/tiny-random-gptj-for-question-answering"
```

I tested with `tiny-random-gptj-for-question-answering` and the results are now deterministic. Could you use them please, and let me know if they work well :-)
ArEnSc: @ydshieh there appears to be a CI issue, otherwise the tests pass on my local machine
```

============================= test session starts ==============================
platform linux -- Python 3.7.12, pytest-7.1.1, pluggy-1.0.0 -- /usr/local/bin/python
cachedir: .pytest_cache
rootdir: /home/circleci/transformers, configfile: setup.cfg
plugins: timeout-2.1.0, forked-1.4.0, xdist-2.5.0
collecting ... collected 10130 items / 10120 deselected / 10 selected

tests/test_modeling_common.py::ModelPushToHubTester::test_push_to_hub Cloning https://moon-staging.huggingface.co/__DUMMY_TRANSFORMERS_USER__/test-model into local empty directory.
To https://moon-staging.huggingface.co/__DUMMY_TRANSFORMERS_USER__/test-model
   0e27ac8..89c7f53  main -> main

Downloading: 100% 580/580 [00:00<00:00, 627kB/s]
Downloading: 100% 244k/244k [00:00<00:00, 38.5MB/s]
PASSED
tests/test_modeling_common.py::ModelPushToHubTester::test_push_to_hub_dynamic_model Cloning https://moon-staging.huggingface.co/__DUMMY_TRANSFORMERS_USER__/test-dynamic-model into local empty directory.
To https://moon-staging.huggingface.co/__DUMMY_TRANSFORMERS_USER__/test-dynamic-model
   ddcd1d1..1bcd2d2  main -> main

Downloading: 100% 304/304 [00:00<00:00, 220kB/s]
Explicitly passing a `revision` is encouraged when loading a configuration with custom code to ensure no malicious code has been contributed in a newer revision.
Downloading: 100% 380/380 [00:00<00:00, 406kB/s]
Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision.
Downloading: 100% 772/772 [00:00<00:00, 526kB/s]
Downloading: 100% 5.04k/5.04k [00:00<00:00, 3.92MB/s]
Explicitly passing a `revision` is encouraged when loading a configuration with custom code to ensure no malicious code has been contributed in a newer revision.
Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision.
PASSED
tests/test_modeling_common.py::ModelPushToHubTester::test_push_to_hub_in_organization Cloning https://moon-staging.huggingface.co/valid_org/test-model-org into local empty directory.
To https://moon-staging.huggingface.co/valid_org/test-model-org
   a4cd7e5..c4905e1  main -> main

Downloading: 100% 580/580 [00:00<00:00, 554kB/s]
Downloading: 100% 244k/244k [00:00<00:00, 50.5MB/s]
PASSED
tests/test_modeling_tf_common.py::TFModelPushToHubTester::test_push_to_hub SKIPPED
tests/test_modeling_tf_common.py::TFModelPushToHubTester::test_push_to_hub_in_organization SKIPPED
tests/test_modeling_tf_common.py::TFModelPushToHubTester::test_push_to_hub_with_model_card SKIPPED
tests/trainer/test_trainer.py::TrainerIntegrationWithHubTester::test_push_to_hub Cloning https://moon-staging.huggingface.co/__DUMMY_TRANSFORMERS_USER__/test-trainer into local empty directory.
Saving model checkpoint to /tmp/tmp4ye8tfs3/test-trainer
Configuration saved in /tmp/tmp4ye8tfs3/test-trainer/config.json
Model weights saved in /tmp/tmp4ye8tfs3/test-trainer/pytorch_model.bin
remote: Repository not found.
fatal: repository 'https://moon-staging.huggingface.co/__DUMMY_TRANSFORMERS_USER__/test-trainer/' not found

FAILED
tests/trainer/test_trainer.py::TrainerIntegrationWithHubTester::test_push_to_hub_in_organization PyTorch: setting up devices
The default value for the training argument `--report_to` will change in v5 (from all installed integrations to none). In v5, you will need to use `--report_to all` to get the same behavior as now. You should start updating your code and make this info disappear :-).
Saving model checkpoint to /tmp/tmpcdm0ggiu
Configuration saved in /tmp/tmpcdm0ggiu/config.json
Model weights saved in /tmp/tmpcdm0ggiu/pytorch_model.bin
PyTorch: setting up devices
The default value for the training argument `--report_to` will change in v5 (from all installed integrations to none). In v5, you will need to use `--report_to all` to get the same behavior as now. You should start updating your code and make this info disappear :-).
Cloning https://moon-staging.huggingface.co/valid_org/test-trainer-org into local empty directory.
Saving model checkpoint to /tmp/tmpcdm0ggiu/test-trainer-org
Configuration saved in /tmp/tmpcdm0ggiu/test-trainer-org/config.json
Model weights saved in /tmp/tmpcdm0ggiu/test-trainer-org/pytorch_model.bin
To https://moon-staging.huggingface.co/valid_org/test-trainer-org
   0e8852b..ebeb238  main -> main

To https://moon-staging.huggingface.co/valid_org/test-trainer-org
   ebeb238..b045802  main -> main

https://moon-staging.huggingface.co/valid_org/test-trainer-org/resolve/main/config.json not found in cache or force_download set to True, downloading to /home/circleci/.cache/huggingface/transformers/tmp6ipnjea8
Downloading: 100% 196/196 [00:00<00:00, 152kB/s]
storing https://moon-staging.huggingface.co/valid_org/test-trainer-org/resolve/main/config.json in cache at /home/circleci/.cache/huggingface/transformers/4e38440c1081859ca3a7aa4cbdb55097a0f85ef5953a2c548d4ae84e5272d5ac.eb08f615fcf788135f3c4543a39165b7da9b0445503c438c3b795304805be7f2
creating metadata file for /home/circleci/.cache/huggingface/transformers/4e38440c1081859ca3a7aa4cbdb55097a0f85ef5953a2c548d4ae84e5272d5ac.eb08f615fcf788135f3c4543a39165b7da9b0445503c438c3b795304805be7f2
loading configuration file https://moon-staging.huggingface.co/valid_org/test-trainer-org/resolve/main/config.json from cache at /home/circleci/.cache/huggingface/transformers/4e38440c1081859ca3a7aa4cbdb55097a0f85ef5953a2c548d4ae84e5272d5ac.eb08f615fcf788135f3c4543a39165b7da9b0445503c438c3b795304805be7f2
Model config RegressionModelConfig {
  "a": 0,
  "architectures": [
    "RegressionPreTrainedModel"
  ],
  "b": 0,
  "double_output": false,
  "hidden_size": 1,
  "torch_dtype": "float32",
  "transformers_version": "4.18.0.dev0"
}

https://moon-staging.huggingface.co/valid_org/test-trainer-org/resolve/main/pytorch_model.bin not found in cache or force_download set to True, downloading to /home/circleci/.cache/huggingface/transformers/tmp20kl5_7m
Downloading: 100% 999/999 [00:00<00:00, 1.03MB/s]
storing https://moon-staging.huggingface.co/valid_org/test-trainer-org/resolve/main/pytorch_model.bin in cache at /home/circleci/.cache/huggingface/transformers/5249fb8d0037a847367e2eef5976497b7f1d8c82da5708c00f0c12339aaa735f.467756c5d7039c90664b3b42ab72e12b75089b5ff223d2dc9d978bd43251e08b
creating metadata file for /home/circleci/.cache/huggingface/transformers/5249fb8d0037a847367e2eef5976497b7f1d8c82da5708c00f0c12339aaa735f.467756c5d7039c90664b3b42ab72e12b75089b5ff223d2dc9d978bd43251e08b
loading weights file https://moon-staging.huggingface.co/valid_org/test-trainer-org/resolve/main/pytorch_model.bin from cache at /home/circleci/.cache/huggingface/transformers/5249fb8d0037a847367e2eef5976497b7f1d8c82da5708c00f0c12339aaa735f.467756c5d7039c90664b3b42ab72e12b75089b5ff223d2dc9d978bd43251e08b
All model checkpoint weights were used when initializing RegressionPreTrainedModel.

All the weights of RegressionPreTrainedModel were initialized from the model checkpoint at valid_org/test-trainer-org.
If your task is similar to the task the model of the checkpoint was trained on, you can already use RegressionPreTrainedModel for predictions without further training.
PASSED
tests/trainer/test_trainer.py::TrainerIntegrationWithHubTester::test_push_to_hub_with_saves_each_epoch PyTorch: setting up devices
The default value for the training argument `--report_to` will change in v5 (from all installed integrations to none). In v5, you will need to use `--report_to all` to get the same behavior as now. You should start updating your code and make this info disappear :-).
Cloning https://moon-staging.huggingface.co/__DUMMY_TRANSFORMERS_USER__/test-trainer-epoch into local empty directory.
***** Running training *****
  Num examples = 64
  Num Epochs = 3
  Instantaneous batch size per device = 8
  Total train batch size (w. parallel, distributed & accumulation) = 8
  Gradient Accumulation steps = 1
  Total optimization steps = 24
  0% 0/24 [00:00<?, ?it/s]Could not estimate the number of tokens of the input, floating-point operations will not be computed
Could not estimate the number of tokens of the input, floating-point operations will not be computed
Could not estimate the number of tokens of the input, floating-point operations will not be computed
Could not estimate the number of tokens of the input, floating-point operations will not be computed
Could not estimate the number of tokens of the input, floating-point operations will not be computed
Could not estimate the number of tokens of the input, floating-point operations will not be computed
Could not estimate the number of tokens of the input, floating-point operations will not be computed
Could not estimate the number of tokens of the input, floating-point operations will not be computed
Saving model checkpoint to /tmp/tmpek8xqng0/test-trainer-epoch/checkpoint-8
Configuration saved in /tmp/tmpek8xqng0/test-trainer-epoch/checkpoint-8/config.json
Model weights saved in /tmp/tmpek8xqng0/test-trainer-epoch/checkpoint-8/pytorch_model.bin
Could not estimate the number of tokens of the input, floating-point operations will not be computed
 38% 9/24 [00:02<00:03,  4.14it/s]Could not estimate the number of tokens of the input, floating-point operations will not be computed
Could not estimate the number of tokens of the input, floating-point operations will not be computed
Could not estimate the number of tokens of the input, floating-point operations will not be computed
Could not estimate the number of tokens of the input, floating-point operations will not be computed
Could not estimate the number of tokens of the input, floating-point operations will not be computed
Could not estimate the number of tokens of the input, floating-point operations will not be computed
Could not estimate the number of tokens of the input, floating-point operations will not be computed
Saving model checkpoint to /tmp/tmpek8xqng0/test-trainer-epoch/checkpoint-16
Configuration saved in /tmp/tmpek8xqng0/test-trainer-epoch/checkpoint-16/config.json
Model weights saved in /tmp/tmpek8xqng0/test-trainer-epoch/checkpoint-16/pytorch_model.bin
Could not estimate the number of tokens of the input, floating-point operations will not be computed
 71% 17/24 [00:06<00:02,  2.48it/s]Could not estimate the number of tokens of the input, floating-point operations will not be computed
Could not estimate the number of tokens of the input, floating-point operations will not be computed
Could not estimate the number of tokens of the input, floating-point operations will not be computed
Could not estimate the number of tokens of the input, floating-point operations will not be computed
Could not estimate the number of tokens of the input, floating-point operations will not be computed
Could not estimate the number of tokens of the input, floating-point operations will not be computed
Could not estimate the number of tokens of the input, floating-point operations will not be computed
Saving model checkpoint to /tmp/tmpek8xqng0/test-trainer-epoch/checkpoint-24
Configuration saved in /tmp/tmpek8xqng0/test-trainer-epoch/checkpoint-24/config.json
Model weights saved in /tmp/tmpek8xqng0/test-trainer-epoch/checkpoint-24/pytorch_model.bin


Training completed. Do not forget to share your model on huggingface.co/models =)


100% 24/24 [00:10<00:00,  2.26it/s]
Cloning https://moon-staging.huggingface.co/__DUMMY_TRANSFORMERS_USER__/test-trainer-epoch into local empty directory.
{'train_runtime': 10.6404, 'train_samples_per_second': 18.044, 'train_steps_per_second': 2.256, 'train_loss': 10.069999694824219, 'epoch': 3.0}
PASSED
tests/trainer/test_trainer.py::TrainerIntegrationWithHubTester::test_push_to_hub_with_saves_each_n_steps PyTorch: setting up devices
The default value for the training argument `--report_to` will change in v5 (from all installed integrations to none). In v5, you will need to use `--report_to all` to get the same behavior as now. You should start updating your code and make this info disappear :-).
Cloning https://moon-staging.huggingface.co/__DUMMY_TRANSFORMERS_USER__/test-trainer-step into local empty directory.
***** Running training *****
  Num examples = 64
  Num Epochs = 3
  Instantaneous batch size per device = 8
  Total train batch size (w. parallel, distributed & accumulation) = 8
  Gradient Accumulation steps = 1
  Total optimization steps = 24
  0% 0/24 [00:00<?, ?it/s]Could not estimate the number of tokens of the input, floating-point operations will not be computed
Could not estimate the number of tokens of the input, floating-point operations will not be computed
Could not estimate the number of tokens of the input, floating-point operations will not be computed
Could not estimate the number of tokens of the input, floating-point operations will not be computed
Could not estimate the number of tokens of the input, floating-point operations will not be computed
Saving model checkpoint to /tmp/tmpxh_v0zlu/test-trainer-step/checkpoint-5
Configuration saved in /tmp/tmpxh_v0zlu/test-trainer-step/checkpoint-5/config.json
Model weights saved in /tmp/tmpxh_v0zlu/test-trainer-step/checkpoint-5/pytorch_model.bin
Could not estimate the number of tokens of the input, floating-point operations will not be computed
 25% 6/24 [00:02<00:06,  2.75it/s]Could not estimate the number of tokens of the input, floating-point operations will not be computed
Could not estimate the number of tokens of the input, floating-point operations will not be computed
Could not estimate the number of tokens of the input, floating-point operations will not be computed
Could not estimate the number of tokens of the input, floating-point operations will not be computed
Saving model checkpoint to /tmp/tmpxh_v0zlu/test-trainer-step/checkpoint-10
Configuration saved in /tmp/tmpxh_v0zlu/test-trainer-step/checkpoint-10/config.json
Model weights saved in /tmp/tmpxh_v0zlu/test-trainer-step/checkpoint-10/pytorch_model.bin
Could not estimate the number of tokens of the input, floating-point operations will not be computed
 46% 11/24 [00:06<00:08,  1.60it/s]Could not estimate the number of tokens of the input, floating-point operations will not be computed
Could not estimate the number of tokens of the input, floating-point operations will not be computed
Could not estimate the number of tokens of the input, floating-point operations will not be computed
Could not estimate the number of tokens of the input, floating-point operations will not be computed
Saving model checkpoint to /tmp/tmpxh_v0zlu/test-trainer-step/checkpoint-15
Configuration saved in /tmp/tmpxh_v0zlu/test-trainer-step/checkpoint-15/config.json
Model weights saved in /tmp/tmpxh_v0zlu/test-trainer-step/checkpoint-15/pytorch_model.bin
Could not estimate the number of tokens of the input, floating-point operations will not be computed
 67% 16/24 [00:10<00:05,  1.38it/s]Could not estimate the number of tokens of the input, floating-point operations will not be computed
Could not estimate the number of tokens of the input, floating-point operations will not be computed
Could not estimate the number of tokens of the input, floating-point operations will not be computed
Could not estimate the number of tokens of the input, floating-point operations will not be computed
Saving model checkpoint to /tmp/tmpxh_v0zlu/test-trainer-step/checkpoint-20
Configuration saved in /tmp/tmpxh_v0zlu/test-trainer-step/checkpoint-20/config.json
Model weights saved in /tmp/tmpxh_v0zlu/test-trainer-step/checkpoint-20/pytorch_model.bin
Could not estimate the number of tokens of the input, floating-point operations will not be computed
 88% 21/24 [00:14<00:02,  1.30it/s]Could not estimate the number of tokens of the input, floating-point operations will not be computed
Could not estimate the number of tokens of the input, floating-point operations will not be computed
Could not estimate the number of tokens of the input, floating-point operations will not be computed


Training completed. Do not forget to share your model on huggingface.co/models =)


100% 24/24 [00:14<00:00,  1.61it/s]
Cloning https://moon-staging.huggingface.co/__DUMMY_TRANSFORMERS_USER__/test-trainer-step into local empty directory.
{'train_runtime': 14.8771, 'train_samples_per_second': 12.906, 'train_steps_per_second': 1.613, 'train_loss': 10.069999694824219, 'epoch': 3.0}
PASSED

=================================== FAILURES ===================================
_______________ TrainerIntegrationWithHubTester.test_push_to_hub _______________

self = <huggingface_hub.repository.Repository object at 0x7f22617b9c50>
upstream = 'origin main', blocking = True, auto_lfs_prune = True

    def git_push(
        self,
        upstream: Optional[str] = None,
        blocking: Optional[bool] = True,
        auto_lfs_prune: Optional[bool] = False,
    ) -> Union[str, Tuple[str, CommandInProgress]]:
        """
        git push
    
        If used without setting `blocking`, will return url to commit on remote
        repo. If used with `blocking=True`, will return a tuple containing the
        url to commit and the command object to follow for information about the
        process.
    
        Args:
            upstream (`str`, *optional*):
                Upstream to which this should push. If not specified, will push
                to the lastly defined upstream or to the default one (`origin
                main`).
            blocking (`bool`, *optional*, defaults to `True`):
                Whether the function should return only when the push has
                finished. Setting this to `False` will return an
                `CommandInProgress` object which has an `is_done` property. This
                property will be set to `True` when the push is finished.
            auto_lfs_prune (`bool`, *optional*, defaults to `False`):
                Whether to automatically prune files once they have been pushed
                to the remote.
        """
        command = "git push"
    
        if upstream:
            command += f" --set-upstream {upstream}"
    
        number_of_commits = commits_to_push(self.local_dir, upstream)
    
        if number_of_commits > 1:
            logger.warning(
                f"Several commits ({number_of_commits}) will be pushed upstream."
            )
            if blocking:
                logger.warning("The progress bars may be unreliable.")
    
        try:
            with _lfs_log_progress():
                process = subprocess.Popen(
                    command.split(),
                    stderr=subprocess.PIPE,
                    stdout=subprocess.PIPE,
                    encoding="utf-8",
                    cwd=self.local_dir,
                )
    
                if blocking:
                    stdout, stderr = process.communicate()
                    return_code = process.poll()
                    process.kill()
    
                    if len(stderr):
                        logger.warning(stderr)
    
                    if return_code:
                        raise subprocess.CalledProcessError(
>                           return_code, process.args, output=stdout, stderr=stderr
                        )
E                       subprocess.CalledProcessError: Command '['git', 'push', '--set-upstream', 'origin', 'main']' returned non-zero exit status 128.

../.local/lib/python3.7/site-packages/huggingface_hub/repository.py:1202: CalledProcessError

During handling of the above exception, another exception occurred:

self = <tests.trainer.test_trainer.TrainerIntegrationWithHubTester testMethod=test_push_to_hub>

    def test_push_to_hub(self):
        with tempfile.TemporaryDirectory() as tmp_dir:
            trainer = get_regression_trainer(
                output_dir=os.path.join(tmp_dir, "test-trainer"),
                push_to_hub=True,
                hub_token=self._token,
            )
>           url = trainer.push_to_hub()

tests/trainer/test_trainer.py:1552: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
src/transformers/trainer.py:2869: in push_to_hub
    commit_message=commit_message, blocking=blocking, auto_lfs_prune=True
../.local/lib/python3.7/site-packages/huggingface_hub/repository.py:1478: in push_to_hub
    auto_lfs_prune=auto_lfs_prune,
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <huggingface_hub.repository.Repository object at 0x7f22617b9c50>
upstream = 'origin main', blocking = True, auto_lfs_prune = True

    def git_push(
        self,
        upstream: Optional[str] = None,
        blocking: Optional[bool] = True,
        auto_lfs_prune: Optional[bool] = False,
    ) -> Union[str, Tuple[str, CommandInProgress]]:
        """
        git push
    
        If used without setting `blocking`, will return url to commit on remote
        repo. If used with `blocking=True`, will return a tuple containing the
        url to commit and the command object to follow for information about the
        process.
    
        Args:
            upstream (`str`, *optional*):
                Upstream to which this should push. If not specified, will push
                to the lastly defined upstream or to the default one (`origin
                main`).
            blocking (`bool`, *optional*, defaults to `True`):
                Whether the function should return only when the push has
                finished. Setting this to `False` will return an
                `CommandInProgress` object which has an `is_done` property. This
                property will be set to `True` when the push is finished.
            auto_lfs_prune (`bool`, *optional*, defaults to `False`):
                Whether to automatically prune files once they have been pushed
                to the remote.
        """
        command = "git push"
    
        if upstream:
            command += f" --set-upstream {upstream}"
    
        number_of_commits = commits_to_push(self.local_dir, upstream)
    
        if number_of_commits > 1:
            logger.warning(
                f"Several commits ({number_of_commits}) will be pushed upstream."
            )
            if blocking:
                logger.warning("The progress bars may be unreliable.")
    
        try:
            with _lfs_log_progress():
                process = subprocess.Popen(
                    command.split(),
                    stderr=subprocess.PIPE,
                    stdout=subprocess.PIPE,
                    encoding="utf-8",
                    cwd=self.local_dir,
                )
    
                if blocking:
                    stdout, stderr = process.communicate()
                    return_code = process.poll()
                    process.kill()
    
                    if len(stderr):
                        logger.warning(stderr)
    
                    if return_code:
                        raise subprocess.CalledProcessError(
                            return_code, process.args, output=stdout, stderr=stderr
                        )
    
        except subprocess.CalledProcessError as exc:
>           raise EnvironmentError(exc.stderr)
E           OSError: remote: Repository not found.
E           fatal: repository 'https://moon-staging.huggingface.co/__DUMMY_TRANSFORMERS_USER__/test-trainer/' not found

../.local/lib/python3.7/site-packages/huggingface_hub/repository.py:1206: OSError
------------------------------ Captured log call -------------------------------
WARNING  huggingface_hub.repository:repository.py:694 Cloning https://moon-staging.huggingface.co/__DUMMY_TRANSFORMERS_USER__/test-trainer into local empty directory.
WARNING  huggingface_hub.repository:repository.py:1198 remote: Repository not found.
fatal: repository 'https://moon-staging.huggingface.co/__DUMMY_TRANSFORMERS_USER__/test-trainer/' not found
=============================== warnings summary ===============================
src/transformers/dynamic_module_utils.py:81
  /home/circleci/transformers/src/transformers/dynamic_module_utils.py:81: DeprecationWarning: invalid escape sequence \s
    relative_imports = re.findall("^\s*import\s+\.(\S+)\s*$", content, flags=re.MULTILINE)

src/transformers/dynamic_module_utils.py:83
  /home/circleci/transformers/src/transformers/dynamic_module_utils.py:83: DeprecationWarning: invalid escape sequence \s
    relative_imports += re.findall("^\s*from\s+\.(\S+)\s+import", content, flags=re.MULTILINE)

src/transformers/dynamic_module_utils.py:125
  /home/circleci/transformers/src/transformers/dynamic_module_utils.py:125: DeprecationWarning: invalid escape sequence \s
    imports = re.findall("^\s*import\s+(\S+)\s*$", content, flags=re.MULTILINE)

src/transformers/dynamic_module_utils.py:127
  /home/circleci/transformers/src/transformers/dynamic_module_utils.py:127: DeprecationWarning: invalid escape sequence \s
    imports += re.findall("^\s*from\s+(\S+)\s+import", content, flags=re.MULTILINE)

src/transformers/modeling_utils.py:199
  /home/circleci/transformers/src/transformers/modeling_utils.py:199: DeprecationWarning: invalid escape sequence \d
    bit_search = re.search("[^\d](\d+)$", str(dtype))

src/transformers/models/glpn/modeling_glpn.py:637
  /home/circleci/transformers/src/transformers/models/glpn/modeling_glpn.py:637: DeprecationWarning: invalid escape sequence \s
    """

tests/led/test_modeling_led.py:510
  /home/circleci/transformers/tests/led/test_modeling_led.py:510: DeprecationWarning: invalid escape sequence \e
    ARTICLE_LEP = """the lep experiments at the resonance of @xmath1-boson have tested the standard model ( sm ) at quantum level , measuring the @xmath1-decay into fermion pairs with an accuracy of one part in ten thousands . the good agreement of the lep data with the sm predictions have severely constrained the behavior of new physics at the @xmath1-pole . taking these achievements into account one can imagine that the physics of @xmath1-boson will again play the central role in the frontier of particle physics if the next generation @xmath1 factory comes true with the generated @xmath1 events several orders of magnitude higher than that of the lep . this factory can be realized in the gigaz option of the international linear collider ( ilc)@xcite . the ilc is a proposed electron - positron collider with tunable energy ranging from @xmath12 to @xmath13 and polarized beams in its first phase , and the gigaz option corresponds to its operation on top of the resonance of @xmath1 boson by adding a bypass to its main beam line . given the high luminosity , @xmath14 , and the cross section at the resonance of @xmath1 boson , @xmath15 , about @xmath16 @xmath1 events can be generated in an operational year of @xmath17 of gigaz , which implies that the expected sensitivity to the branching ratio of @xmath1-decay can be improved from @xmath18 at the lep to @xmath19 at the gigaz@xcite . in light of this , the @xmath1-boson properties , especially its exotic or rare decays which are widely believed to be sensitive to new physics , should be investigated comprehensively to evaluate their potential in probing new physics .    among the rare @xmath1-decays , the flavor changing ( fc ) processes were most extensively studied to explore the flavor texture in new physics @xcite , and it was found that , although these processes are severely suppressed in the sm , their branching ratios in new physics models can be greatly enhanced to @xmath19 for lepton flavor violation decays @xcite and @xmath20 for quark flavor violation decays @xcite . besides the fc processes , the @xmath1-decay into light higgs boson(s ) is another type of rare process that was widely studied , e.g. the decay @xmath21 ( @xmath22 ) with the particle @xmath0 denoting a light higgs boson was studied in @xcite , the decay @xmath23 was studied in the two higgs doublet model ( 2hdm)@xcite and the minimal supersymmetric standard model ( mssm)@xcite , and the decay @xmath4 was studied in a model independent way @xcite , in 2hdm@xcite and also in mssm@xcite . these studies indicate that , in contrast with the kinematic forbidden of these decays in the sm , the rates of these decays can be as large as @xmath18 in new physics models , which lie within the expected sensitivity of the gigaz . in this work , we extend the previous studies of these decays to some new models and investigate these decays altogether . we are motivated by some recent studies on the singlet extension of the mssm , such as the next - to - minimal supersymmetric standard model ( nmssm ) @xcite and the nearly minimal supersymmetric standard model ( nmssm ) @xcite , where a light cp - odd higgs boson @xmath0 with singlet - dominant component may naturally arise from the spontaneous breaking of some approximate global symmetry like @xmath24 or peccei - quuin symmetry @xcite . these non - minimal supersymmetric models can not only avoid the @xmath25-problem , but also alleviate the little hierarchy by having such a light higgs boson @xmath0 @xcite . we are also motivated by that , with the latest experiments , the properties of the light higgs boson are more stringently constrained than before . so it is worth updating the previous studies . so far there is no model - independent lower bound on the lightest higgs boson mass . in the sm , it must be heavier than @xmath26 gev , obtained from the null observation of the higgs boson at lep experiments . however , due to the more complex structure of the higgs sector in the extensions of the sm , this lower bound can be significantly relaxed according to recent studies , e.g. , for the cp - odd higgs boson @xmath0 we have @xmath27 gev in the nmssm @xcite , @xmath28 gev in the nmssm @xcite , and @xmath29 gev in the lepton - specific 2hdm ( l2hdm ) @xcite . with such a light cp - odd higgs boson , the z - decay into one or more @xmath0 is open up . noting that the decay @xmath30 is forbidden due to bose symmetry , we in this work study the rare @xmath1-decays @xmath6 ( @xmath22 ) , @xmath31 and @xmath4 in a comparative way for four models , namely the type - ii 2hdm@xcite , the l2hdm @xcite , the nmssm and the nmssm . in our study , we examine carefully the constraints on the light @xmath0 from many latest experimental results . this work is organized as follows . in sec . ii we briefly describe the four new physics models . in sec . iii we present the calculations of the rare @xmath1-decays . in sec . iv we list the constraints on the four new physics models . in sec . v we show the numerical results for the branching ratios of the rare @xmath1-decays in various models . finally , the conclusion is given in sec . as the most economical way , the sm utilizes one higgs doublet to break the electroweak symmetry . as a result , the sm predicts only one physical higgs boson with its properties totally determined by two free parameters . in new physics models , the higgs sector is usually extended by adding higgs doublets and/or singlets , and consequently , more physical higgs bosons are predicted along with more free parameters involved in . the general 2hdm contains two @xmath32 doublet higgs fields @xmath33 and @xmath34 , and with the assumption of cp - conserving , its scalar potential can be parameterized as@xcite : @xmath35,\end{aligned}\ ] ] where @xmath36 ( @xmath37 ) are free dimensionless parameters , and @xmath38 ( @xmath39 ) are the parameters with mass dimension . after the electroweak symmetry breaking , the spectrum of this higgs sector includes three massless goldstone modes , which become the longitudinal modes of @xmath40 and @xmath1 bosons , and five massive physical states : two cp - even higgs bosons @xmath41 and @xmath42 , one neutral cp - odd higgs particle @xmath0 and a pair of charged higgs bosons @xmath43 . noting the constraint @xmath44 with @xmath45 and @xmath46 denoting the vacuum expectation values ( vev ) of @xmath33 and @xmath34 respectively , we choose @xmath47 as the input parameters with @xmath48 , and @xmath49 being the mixing angle that diagonalizes the mass matrix of the cp - even higgs fields . the difference between the type - ii 2hdm and the l2hdm comes from the yukawa coupling of the higgs bosons to quark / lepton . in the type - ii 2hdm , one higgs doublet @xmath34 generates the masses of up - type quarks and the other doublet @xmath33 generates the masses of down - type quarks and charged leptons ; while in the l2hdm one higgs doublet @xmath33 couples only to leptons and the other doublet @xmath34 couples only to quarks . so the yukawa interactions of @xmath0 to fermions in these two models are given by @xcite @xmath50 with @xmath51 denoting generation index . obviously , in the type - ii 2hdm the @xmath52 coupling and the @xmath53 coupling can be simultaneously enhanced by @xmath54 , while in the l2hdm only the @xmath53 coupling is enhanced by @xmath55 . the structures of the nmssm and the nmssm are described by their superpotentials and corresponding soft - breaking terms , which are given by @xcite @xmath56 where @xmath57 is the superpotential of the mssm without the @xmath25 term , @xmath58 and @xmath59 are higgs doublet and singlet superfields with @xmath60 and @xmath61 being their scalar component respectively , @xmath62 , @xmath63 , @xmath64 , @xmath65 , @xmath66 and @xmath67 are soft breaking parameters , and @xmath68 and @xmath69 are coefficients of the higgs self interactions .    with the superpotentials and the soft - breaking terms , one can get the higgs potentials of the nmssm and the nmssm respectively . like the 2hdm , the higgs bosons with same cp property will mix and the mass eigenstates are obtained by diagonalizing the corresponding mass matrices : @xmath70 where the fields on the right hands of the equations are component fields of @xmath71 , @xmath72 and @xmath61 defined by @xmath73 @xmath74 and @xmath75 are respectively the cp - even and cp - odd neutral higgs bosons , @xmath76 and @xmath77 are goldstone bosons eaten by @xmath1 and @xmath78 , and @xmath79 is the charged higgs boson . so both the nmssm and nmssm predict three cp - even higgs bosons , two cp - odd higgs bosons and one pair of charged higgs bosons . in general , the lighter cp - odd higgs @xmath0 in these model is the mixture of the singlet field @xmath80 and the doublet field combination , @xmath81 , i.e. @xmath82 and its couplings to down - type quarks are then proportional to @xmath83 . so for singlet dominated @xmath0 , @xmath84 is small and the couplings are suppressed . as a comparison , the interactions of @xmath0 with the squarks are given by@xcite @xmath85 i.e. the interaction does not vanish when @xmath86 approaches zero . just like the 2hdm where we use the vevs of the higgs fields as fundamental parameters , we choose @xmath68 , @xmath69 , @xmath87 , @xmath88 , @xmath66 and @xmath89 as input parameters for the nmssm@xcite and @xmath68 , @xmath54 , @xmath88 , @xmath65 , @xmath90 and @xmath91 as input parameters for the nmssm@xcite . about the nmssm and the nmssm , three points should be noted . the first is for the two models , there is no explicit @xmath92term , and the effective @xmath25 parameter ( @xmath93 ) is generated when the scalar component of @xmath59 develops a vev . the second is , the nmssm is actually same as the nmssm with @xmath94@xcite , because the tadpole terms @xmath95 and its soft breaking term @xmath96 in the nmssm do not induce any interactions , except for the tree - level higgs boson masses and the minimization conditions . and the last is despite of the similarities , the nmssm has its own peculiarity , which comes from its neutralino sector . in the basis @xmath97 , its neutralino mass matrix is given by @xcite @xmath98 where @xmath99 and @xmath100 are @xmath101 and @xmath102 gaugino masses respectively , @xmath103 , @xmath104 , @xmath105 and @xmath106 . after diagonalizing this matrix one can get the mass eigenstate of the lightest neutralino @xmath107 with mass taking the following form @xcite @xmath108 this expression implies that @xmath107 must be lighter than about @xmath109 gev for @xmath110 ( from lower bound on chargnio mass ) and @xmath111 ( perturbativity bound ) . like the other supersymmetric models , @xmath107 as the lightest sparticle acts as the dark matter in the universe , but due to its singlino - dominated nature , it is difficult to annihilate sufficiently to get the correct density in the current universe . so the relic density of @xmath107 plays a crucial way in selecting the model parameters . for example , as shown in @xcite , for @xmath112 , there is no way to get the correct relic density , and for the other cases , @xmath107 mainly annihilates by exchanging @xmath1 boson for @xmath113 , or by exchanging a light cp - odd higgs boson @xmath0 with mass satisfying the relation @xmath114 for @xmath115 . for the annihilation , @xmath54 and @xmath25 are required to be less than 10 and @xmath116 respectively because through eq.([mass - exp ] ) a large @xmath87 or @xmath25 will suppress @xmath117 to make the annihilation more difficult . the properties of the lightest cp - odd higgs boson @xmath0 , such as its mass and couplings , are also limited tightly since @xmath0 plays an important role in @xmath107 annihilation . the phenomenology of the nmssm is also rather special , and this was discussed in detail in @xcite . in the type - ii 2hdm , l2hdm , nmssm and nmssm , the rare @xmath1-decays @xmath118 ( @xmath22 ) , @xmath3 and @xmath4 may proceed by the feynman diagrams shown in fig.[fig1 ] , fig.[fig2 ] and fig.[fig3 ] respectively . for these diagrams , the intermediate state @xmath119 represents all possible cp - even higgs bosons in the corresponding model , i.e. @xmath41 and @xmath42 in type - ii 2hdm and l2hdm and @xmath41 , @xmath42 and @xmath120 in nmssm and nmssm . in order to take into account the possible resonance effects of @xmath119 in fig.[fig1](c ) for @xmath2 and fig.[fig3 ] ( a ) for @xmath11 , we have calculated all the decay modes of @xmath119 and properly included the width effect in its propagator . as to the decay @xmath121 , two points should be noted . one is , unlike the decays @xmath6 and @xmath11 , this process proceeds only through loops mediated by quarks / leptons in the type - ii 2hdm and l2hdm , and additionally by sparticles in the nmssm and nmssm . so in most cases its rate should be much smaller than the other two . the other is due to cp - invariance , loops mediated by squarks / sleptons give no contribution to the decay@xcite . in actual calculation , this is reflected by the fact that the coupling coefficient of @xmath122 differs from that of @xmath123 by a minus sign ( see eq.([asqsq ] ) ) , and as a result , the squark - mediated contributions to @xmath121 are completely canceled out .    with regard to the rare decay @xmath11 , we have more explanations . in the lowest order , this decay proceeds by the diagram shown in fig.[fig3 ] ( a ) , and hence one may think that , as a rough estimate , it is enough to only consider the contributions from fig.[fig3](a ) . however , we note that in some cases of the type - ii 2hdm and l2hdm , due to the cancelation of the contributions from different @xmath119 in fig.[fig3 ] ( a ) and also due to the potentially largeness of @xmath124 couplings ( i.e. larger than the electroweak scale @xmath125 ) , the radiative correction from the higgs - mediated loops may dominate over the tree level contribution even when the tree level prediction of the rate , @xmath126 , exceeds @xmath20 . on the other hand , we find the contribution from quark / lepton - mediated loops can be safely neglected if @xmath127 in the type - ii 2hdm and the l2hdm . in the nmssm and the nmssm , besides the corrections from the higgs- and quark / lepton - mediated loops , loops involving sparticles such as squarks , charginos and neutralinos can also contribute to the decay . we numerically checked that the contributions from squarks and charginos can be safely neglected if @xmath127 . we also calculated part of potentially large neutralino correction ( note that there are totally about @xmath128 diagrams for such correction ! ) and found they can be neglected too . since considering all the radiative corrections will make our numerical calculation rather slow , we only include the most important correction , namely that from higgs - mediated loops , in presenting our results for the four models . one can intuitively understand the relative smallness of the sparticle contribution to @xmath11 as follows . first consider the squark contribution which is induced by the @xmath129 interaction ( @xmath130 denotes the squark in chirality state ) and the @xmath131 interaction through box diagrams . because the @xmath132 interaction conserves the chirality of the squarks while the @xmath133 interaction violates the chirality , to get non - zero contribution to @xmath11 from the squark loops , at least four chiral flippings are needed , with three of them provided by @xmath131 interaction and the rest provided by the left - right squark mixing . this means that , if one calculates the amplitude in the chirality basis with the mass insertion method , the amplitude is suppressed by the mixing factor @xmath134 with @xmath135 being the off diagonal element in squark mass matrix . next consider the chargino / neutralino contributions . since for a light @xmath0 , its doublet component , parameterized by @xmath84 in eq.([mixing ] ) , is usually small , the couplings of @xmath0 with the sparticles will never be tremendously large@xcite . so the chargino / neutralino contributions are not important too . in our calculation of the decays , we work in the mass eigenstates of sparticles instead of in the chirality basis . for the type - ii 2hdm and the l2hdm , we consider the following constraints @xcite :    * theoretical constraints on @xmath136 from perturbativity , unitarity and requirements that the scalar potential is finit at large field values and contains no flat directions @xcite , which imply that @xmath137 * the constraints from the lep search for neutral higgs bosons . we compute the signals from the higgs - strahlung production @xmath138 ( @xmath139 ) with @xmath140 @xcite and from the associated production @xmath141 with @xmath142 @xcite , and compare them with the corresponding lep data which have been inputted into our code . we also consider the constraints from @xmath138 by looking for a peak of @xmath143 recoil mass distribution of @xmath1-boson @xcite and the constraint of @xmath144 mev when @xmath145 @xcite . + these constraints limit the quantities such as @xmath146 \times br ( h_i \to \bar{b } b ) $ ] on the @xmath147 plane with the the subscript @xmath148 denoting the coupling coefficient of the @xmath149 interaction . they also impose a model - dependent lower bound on @xmath150 , e.g. , @xmath151 for the type - ii 2hdm ( from our scan results ) , @xmath152 for the l2hdm@xcite , and @xmath153 for the nmssm @xcite . these bounds are significantly lower than that of the sm , i.e. @xmath154 , partially because in new physics models , unconventional decay modes of @xmath155 such as @xmath156 are open up . as to the nmssm , another specific reason for allowing a significantly lighter cp - even higgs boson is that the boson may be singlet - dominated in this model . + with regard to the lightest cp - odd higgs boson @xmath0 , we checked that there is no lower bound on its mass so long as the @xmath157 interaction is weak or @xmath155 is sufficiently heavy . * the constraints from the lep search for a light higgs boson via the yukawa process @xmath158 with @xmath22 and @xmath61 denoting a scalar @xcite . these constraints can limit the @xmath159 coupling versus @xmath160 in new physics models . * the constraints from the cleo - iii limit on @xmath161 and the latest babar limits on @xmath162 . these constraints will put very tight constraints on the @xmath163 coupling for @xmath164 . in our analysis , we use the results of fig.8 in the second paper of @xcite to excluded the unfavored points . * the constraints from @xmath165 couplings . since the higgs sector can give sizable higher order corrections to @xmath165 couplings , we calculate them to one loop level and require the corrected @xmath165 couplings to lie within the @xmath166 range of their fitted value . the sm predictions for the couplings at @xmath1-pole are given by @xmath167 and @xmath168 @xcite , and the fitted values are given by @xmath169 and @xmath170 , respectively@xcite . we adopt the formula in @xcite to the 2hdm in our calculation . * the constraints from @xmath171 leptonic decay . we require the new physics correction to the branching ratio @xmath172 to be in the range of @xmath173 @xcite . we use the formula in @xcite in our calculation . + about the constraints ( 5 ) and ( 6 ) , two points should be noted . one is all higgs bosons are involved in the constraints by entering the self energy of @xmath171 lepton , the @xmath174 vertex correction or the @xmath175 vertex correction , and also the box diagrams for @xmath176@xcite . since the yukawa couplings of the higgs bosons to @xmath171 lepton get enhanced by @xmath54 and so do the corrections , @xmath54 must be upper bounded for given spectrum of the higgs sector . generally speaking , the lighter @xmath0 is , the more tightly @xmath54 is limited@xcite . the other point is in the type - ii 2hdm , @xmath177 , b - physics observables as well as @xmath178 decays discussed above can constraint the model in a tighter way than the constraints ( 5 ) and ( 6 ) since the yukawa couplings of @xmath171 lepton and @xmath179 quark are simultaneously enhanced by @xmath54 . but for the l2hdm , because only the yukawa couplings of @xmath171 lepton get enhanced ( see eq.[yukawa ] ) , the constraints ( 5 ) and ( 6 ) are more important in limiting @xmath54 . * indirect constraints from the precision electroweak observables such as @xmath180 , @xmath181 and @xmath182 , or their combinations @xmath183 @xcite . we require @xmath184 to be compatible with the lep / sld data at @xmath185 confidence level@xcite . we also require new physics prediction of @xmath186 is within the @xmath187 range of its experimental value . the latest results for @xmath188 are @xmath189 ( measured value ) and @xmath190 ( sm prediction ) for @xmath191 gev @xcite . in our code , we adopt the formula for these observables presented in @xcite to the type - ii 2hdm and the l2hdm respectively . + in calculating @xmath180 , @xmath181 and @xmath182 , we note that these observables get dominant contributions from the self energies of the gauge bosons @xmath1 , @xmath192 and @xmath193 . since there is no @xmath194 coupling or @xmath195 coupling , @xmath0 must be associated with the other higgs bosons to contribute to the self energies . so by the uv convergence of these quantities , one can infer that , for the case of a light @xmath0 and @xmath196 , these quantities depend on the spectrum of the higgs sector in a way like @xmath197 at leading order , which implies that a light @xmath0 can still survive the constraints from the precision electroweak observables given the splitting between @xmath150 and @xmath198 is moderate@xcite . * the constraints from b physics observables such as the branching ratios for @xmath199 , @xmath200 and @xmath201 , and the mass differences @xmath202 and @xmath203 . we require their theoretical predications to agree with the corresponding experimental values at @xmath187 level . + in the type - ii 2hdm and the l2hdm , only the charged higgs boson contributes to these observables by loops , so one can expect that @xmath198 versus @xmath54 is to be limited . combined analysis of the limits in the type - ii 2hdm has been done by the ckmfitter group , and the lower bound of @xmath204 as a function of @xmath87 was given in fig.11 of @xcite . this analysis indicates that @xmath198 must be heavier than @xmath205 at @xmath185 c.l . regardless the value of @xmath54 . in this work , we use the results of fig.11 in @xcite to exclude the unfavored points . as for the l2hdm , b physics actually can not put any constraints@xcite because in this model the couplings of the charged higgs boson to quarks are proportional to @xmath206 and in the case of large @xmath54 which we are interested in , they are suppressed . in our analysis of the l2hdm , we impose the lep bound on @xmath198 , i.e. @xmath207@xcite . * the constraints from the muon anomalous magnetic moment @xmath208 . now both the theoretical prediction and the experimental measured value of @xmath208 have reached a remarkable precision , but a significant deviation still exists : @xmath209 @xcite . in the 2hdm , @xmath208 gets additional contributions from the one - loop diagrams induced by the higgs bosons and also from the two - loop barr - zee diagrams mediated by @xmath0 and @xmath155@xcite . if the higgs bosons are much heavier than @xmath25 lepton mass , the contributions from the barr - zee diagrams are more important , and to efficiently alleviate the discrepancy of @xmath208 , one needs a light @xmath0 along with its enhanced couplings to @xmath25 lepton and also to heavy fermions such as bottom quark and @xmath171 lepton to push up the effects of the barr - zee diagram@xcite . the cp - even higgs bosons are usually preferred to be heavy since their contributions to @xmath208 are negative . + in the type - ii 2hdm , because @xmath54 is tightly constrained by the process @xmath210 at the lep@xcite and the @xmath178 decay@xcite , the barr - zee diagram contribution is insufficient to enhance @xmath208 to @xmath187 range around its measured value@xcite . so in our analysis , we require the type - ii 2hdm to explain @xmath208 at @xmath211 level . while for the l2hdm , @xmath54 is less constrained compared with the type - ii 2hdm , and the barr - zee diagram involving the @xmath171-loop is capable to push up greatly the theoretical prediction of @xmath208@xcite . therefore , we require the l2hdm to explain the discrepancy at @xmath187 level . + unlike the other constraints discussed above , the @xmath208 constraint will put a two - sided bound on @xmath54 since on the one hand , it needs a large @xmath54 to enhance the barr - zee contribution , but on the other hand , too large @xmath54 will result in an unacceptable large @xmath208 . * since this paper concentrates on a light @xmath0 , the decay @xmath212 is open up with a possible large decay width . we require the width of any higgs boson to be smaller than its mass to avoid a too fat higgs boson@xcite . we checked that for the scenario characterized by @xmath213 , the coefficient of @xmath214 interaction is usually larger than the electroweak scale @xmath125 , and consequently a large decay width is resulted . for the nmssm and nmssm , the above constraints become more complicated because in these models , not only more higgs bosons are involved in , but also sparticles enter the constraints . so it is not easy to understand some of the constraints intuitively . take the process @xmath199 as an example . in the supersymmetric models , besides the charged higgs contribution , chargino loops , gluino loops as well as neutralino loops also contribute to the process@xcite , and depending on the susy parameters , any of these contributions may become dominated over or be canceled by other contributions . as a result , although the charged higgs affects the process in the same way as that in the type - ii 2hdm , charged higgs as light as @xmath215 is still allowed even for @xmath216@xcite .    since among the constraints , @xmath208 is rather peculiar in that it needs new physics to explain the discrepancy between @xmath217 and @xmath218 , we discuss more about its dependence on susy parameters . in the nmssm and the nmssm , @xmath208 receives contributions from higgs loops and neutralino / chargino loops . for the higgs contribution , it is quite similar to that of the type - ii 2hdm except that more higgs bosons are involved in@xcite . for the neutralino / chargino contribution , in the light bino limit ( i.e. @xmath219 ) , it can be approximated by@xcite @xmath220 for @xmath221 with @xmath222 being smuon mass . so combining the two contributions together , one can learn that a light @xmath0 along with large @xmath54 and/or light smuon with moderate @xmath87 are favored to dilute the discrepancy .    because more parameters are involved in the constraints on the supersymmetric models , we consider following additional constraints to further limit their parameters :    * direct bounds on sparticle masses from the lep1 , the lep2 and the tevatron experiments @xcite . * the lep1 bound on invisible z decay @xmath223 ; the lep2 bound on neutralino production @xmath224 and @xmath225@xcite . * dark matter constraints from the wmap relic density 0.0975 @xmath226 0.1213 @xcite . note that among the above constraints , the constraint ( 2 ) on higgs sector and the constraint ( c ) on neutralino sector are very important . this is because in the supersymmetric models , the sm - like higgs is upper bounded by about @xmath227 at tree level and by about @xmath228 at loop level , and that the relic density restricts the lsp annihilation cross section in a certain narrow range .    in our analysis of the nmssm , we calculate the constraints ( 3 ) and ( 5 - 7 ) by ourselves and utilize the code nmssmtools @xcite to implement the rest constraints . we also extend nmssmtools to the nmssm to implement the constraints . for the extension , the most difficult thing we faced is how to adapt the code micromegas@xcite to the nmssm case . we solve this problem by noting the following facts :    * as we mentioned before , the nmssm is actually same as the nmssm with the trilinear singlet term setting to zero . so we can utilize the model file of the nmssm as the input of the micromegas and set @xmath229 . * since in the nmssm , the lsp is too light to annihilate into higgs pairs , there is no need to reconstruct the effective higgs potential to calculate precisely the annihilation channel @xmath230 with @xmath61 denoting any of higgs bosons@xcite . we thank the authors of the nmssmtools for helpful discussion on this issue when we finish such extension@xcite . with the above constraints , we perform four independent random scans over the parameter space of the type - ii 2hdm , the l2hdm , the nmssm and the nmssm respectively . we vary the parameters in following ranges : @xmath231 for the type - ii 2hdm , @xmath232 for the l2hdm , @xmath233 for the nmssm , and @xmath234 for the nmssm .    in performing the scans , we note that for the nmssm and the nmssm , some constraints also rely on the gaugino masses and the soft breaking parameters in the squark sector and the slepton sector . since these parameters affect little on the properties of @xmath0 , we fix them to reduce the number of free parameters in our scan . for the squark sector , we adopt the @xmath235 scenario which assumes that the soft mass parameters for the third generation squarks are degenerate : @xmath236 800 gev , and that the trilinear couplings of the third generation squarks are also degenerate , @xmath237 with @xmath238 . for the slepton sector , we assume all the soft - breaking masses and trilinear parameters to be 100 gev . this setting is necessary for the nmssm since this model is difficult to explain the muon anomalous moment at @xmath239 level for heavy sleptons@xcite . finally , we assume the grand unification relation @xmath240 for the gaugino masses with @xmath241 being fine structure constants of the different gauge group .    with large number of random points in the scans , we finally get about @xmath242 , @xmath243 , @xmath244 and @xmath242 samples for the type - ii 2hdm , the l2hdm , the nmssm and the nmssm respectively which survive the constraints and satisfy @xmath245 . analyzing the properties of the @xmath0 indicates that for most of the surviving points in the nmssm and the nmssm , its dominant component is the singlet field ( numerically speaking , @xmath246 ) so that its couplings to the sm fermions are suppressed@xcite . our analysis also indicates that the main decay products of @xmath0 are @xmath247 for the l2hdm@xcite , @xmath248 ( dominant ) and @xmath247 ( subdominant ) for the type - ii 2hdm , the nmssm and the nmssm , and in some rare cases , neutralino pairs in the nmssm@xcite .    in fig.[fig4 ] , we project the surviving samples on the @xmath249 plane . this figure shows that the allowed range of @xmath54 is from @xmath250 to @xmath251 in the type - ii 2hdm , and from @xmath252 to @xmath253 in the l2hdm . just as we introduced before , the lower bounds of @xmath254 come from the fact that we require the models to explain the muon anomalous moment , while the upper bound is due to we have imposed the constraint from the lep process @xmath255 , which have limited the upper reach of the @xmath256 coupling for light @xmath61 @xcite(for the dependence of @xmath256 coupling on @xmath54 , see sec . this figure also indicates that for the nmssm and the nmssm , @xmath54 is upper bounded by @xmath257 . for the nmssm , this is because large @xmath87 can suppress the dark matter mass to make its annihilation difficult ( see @xcite and also sec . ii ) , but for the nmssm , this is because we choose a light slepton mass so that large @xmath54 can enhance @xmath208 too significantly to be experimentally unacceptable . we checked that for the slepton mass as heavy as @xmath258 , @xmath259 is still allowed for the nmssm .    in fig.[fig5 ] and fig.[fig6 ] , we show the branching ratios of @xmath260 and @xmath261 respectively . fig.[fig5 ] indicates , among the four models , the type - ii 2hdm predicts the largest ratio for @xmath260 with its value varying from @xmath262 to @xmath263 . the underlying reason is in the type - ii 2hdm , the @xmath264 coupling is enhanced by @xmath54 ( see fig.[fig4 ] ) , while in the other three model , the coupling is suppressed either by @xmath265 or by the singlet component of the @xmath0 . fig.[fig6 ] shows that the l2hdm predicts the largest rate for @xmath266 with its value reaching @xmath5 in optimum case , and for the other three models , the ratio of @xmath261 is at least about one order smaller than that of @xmath267 . this feature can be easily understood from the @xmath268 coupling introduced in sect . we emphasize that , if the nature prefers a light @xmath0 , @xmath260 and/or @xmath269 in the type - ii 2hdm and the l2hdm will be observable at the gigaz . then by the rates of the two decays , one can determine whether the type - ii 2hdm or the l2hdm is the right theory . on the other hand , if both decays are observed with small rates or fail to be observed , the singlet extensions of the mssm are favored .    in fig.[fig7 ] , we show the rate of @xmath3 as the function of @xmath270 . this figure indicates that the branching ratio of @xmath121 can reach @xmath271 , @xmath272 , @xmath273 and @xmath274 for the optimal cases of the type - ii 2hdm , the l2hdm , the nmssm and the nmssm respectively , which implies that the decay @xmath121 will never be observable at the gigaz if the studied model is chosen by nature . the reason for the smallness is , as we pointed out before , that the decay @xmath121 proceeds only at loop level . comparing the optimum cases of the type - ii 2hdm , the nmssm and the nmssm shown in fig.5 - 7 , one may find that the relation @xmath275 holds for any of the decays . this is because the decays are all induced by the yukawa couplings with similar structure for the models . in the supersymmetric models , the large singlet component of the light @xmath0 is to suppress the yukawa couplings , and the @xmath0 in the nmssm has more singlet component than that in the nmssm . next we consider the decay @xmath11 , which , unlike the above decays , depends on the higgs self interactions . in fig.[fig8 ] we plot its rate as a function of @xmath270 and this figure indicates that the @xmath276 may be the largest among the ratios of the exotic @xmath1 decays , reaching @xmath277 in the optimum cases of the type - ii 2hdm , the l2hdm and the nmssm . the underlying reason is , in some cases , the intermediate state @xmath119 in fig.[fig3 ] ( a ) may be on - shell . in fact , we find this is one of the main differences between the nmssm and the nmssm , that is , in the nmssm , @xmath119 in fig.[fig3 ] ( a ) may be on - shell ( corresponds to the points with large @xmath278 ) while in the nmssm , this seems impossible . so we conclude that the decay @xmath11 may serve as an alternative channel to test new physics models , especially it may be used to distinguish the nmssm from the nmssm if the supersymmetry is found at the lhc and the @xmath11 is observed at the gigaz with large rate . before we end our discussion , we note that in the nmssm , the higgs boson @xmath0 may be lighter than @xmath279 without conflicting with low energy data from @xmath178 decays and the other observables ( see fig.[fig4]-[fig8 ] ) . in this case , @xmath0 is axion - like as pointed out in @xcite . we checked that , among the rare @xmath1 decays discussed in this paper , the largest branching ratio comes from @xmath280 which can reach @xmath281 . since in this case , the decay product of @xmath0 is highly collinear muon pair , detecting the decay @xmath280 may need some knowledge about detectors , which is beyond our discussion . in this paper , we studied the rare @xmath1-decays @xmath2 ( @xmath7 ) , @xmath282 and @xmath4 in the type - ii 2hdm , lepton - specific 2hdm , nmssm and nmssm , which predict a light cp - odd higgs boson @xmath0 . in the parameter space allowed by current experiments , the branching ratio can be as large as @xmath5 for @xmath118 , @xmath8 for @xmath3 and @xmath9 for @xmath4 , which implies that the decays @xmath2 and @xmath283 may be accessible at the gigaz option . since different models predict different size of branching ratios , these decays can be used to distinguish different model through the measurement of these rare decays . this work was supported in part by hastit under grant no . 2009hastit004 , by the national natural science foundation of china ( nnsfc ) under grant nos . 10821504 , 10725526 , 10635030 , 10775039 , 11075045 and by the project of knowledge innovation program ( pkip ) of chinese academy of sciences under grant no . .        for some reviews , see , e.g. , m.  a.  perez , g.  tavares - velasco and j.  j.  toscano , int . j.  mod . a * 19 * , 159 ( 2004 ) ; j. m. yang , arxiv:1006.2594 . j.  i.  illana , m.  masip , 67 , 035004 ( 2003 ) ; j. cao , z. xiong , j. m. yang , 32 , 245 ( 2004 ) . d. atwood _ et al_. , 66 , 093005 ( 2002 ) . j. kalinowski , and s. pokorski , 219 , 116 ( 1989 ) ; a. djouadi , p. m. zerwas and j. zunft , 259 , 175 ( 1991 ) ; a. djouadi , j. kalinowski , and p. m. zerwas , z. phys . c * 54 * , 255 ( 1992 ) . m. krawczyk , _ et al . _ , 19 , 463 ( 2001 ) ; 8 , 495 ( 1999 ) . j. f. gunion , g. gamberini and s. f. novaes , 38 , 3481 ( 1988 ) ; thomas j. weiler and tzu - chiang yuan , 318 , 337 ( 1989 ) ; a. djouadi , _ et al . _ , 1 , 163 ( 1998)[hep - ph/9701342 ] . d.  chang and w.  y.  keung , phys . lett .  * 77 * , 3732 ( 1996 ) . e.  keith and e.  ma , 57 , 2017 ( 1998 ) ; m.  a.  perez , g.  tavares - velasco and j.  j. toscano , int . j.  mod.phys . a * 19 * , 159 ( 2004 ) . f.  larios , g.  tavares - velasco and c. p.  yuan , 64 , 055004 ( 2001 ) ; 66 , 075006 ( 2002 ) . a. djouadi , _ et al . _ , 10 , 27 ( 1999 ) [ hep - ph/9903229 ] . for a detailed introduction of the nmssm , see f.  franke and h. fraas , int . j.  mod . a * 12 * ( 1997 ) 479 ; for a recent review of the nmssm , see for example , u. ellwanger , c. hugonie , and a. m. teixeira , arxiv : 0910.1785 . see , e.g. , j.  r.  ellis , j.  f.  gunion , h.  e.  haber , l.  roszkowski and f.  zwirner , phys .  rev . d * 39 * ( 1989 ) 844 ; m.  drees , int . j.  mod . phys .  a * 4 * ( 1989 ) 3635 ; u.  ellwanger , m.  rausch de traubenberg and c.  a.  savoy , phys . b * 315 * ( 1993 ) 331 ; nucl . b * 492 * ( 1997 ) 21 ; d.j . miller , r. nevzorov , p.m. zerwas , 681 , 3 ( 2004 ) .    c.  panagiotakopoulos , k.  tamvakis , 446 , 224 ( 1999 ) ; 469 , 145 ( 1999 ) ; c. panagiotakopoulos , a. pilaftsis , 63 , 055003 ( 2001 ) ; a.  dedes , _ et al . _ , 63 , 055009 ( 2001 ) ; a.  menon , _ et al . _ , 70 , 035005 ( 2004 ) ; v.  barger , _ et al . _ , 630 , 85 ( 2005 ) . c.  balazs , _ et al . _ , 0706 , 066 ( 2007 ) . b. a. dobrescu , k. t. matchev , 0009 , 031 ( 2000 ) ; a. arhrib , k. cheung , t. j. hou , k. w. song , hep - ph/0611211 ; 0703 , 073 ( 2007 ) ; x. g. he , j. tandean , and g. valencia , 98 , 081802 ( 2007 ) ; 0806 , 002 ( 2008 ) ; f. domingo _ et al_. , 0901 , 061 ( 2009 ) ; gudrun hiller , 70 , 034018 ( 2004 ) ; r. dermisek , and john f. gunion , 75 , 075019 ( 2007 ) ; 79 , 055014 ( 2009 ) ; 81 , 055001 ( 2010 ) ; r. dermisek , john f. gunion , and b. mcelrath , 76 , 051105 ( 2007 ) ; z. heng , _ et al_. , 77 , 095012 ( 2008 ) ; a. belyaev _ et al_. , 81 , 075021 ( 2010 ) ; d. das and u.  ellwanger , arxiv:1007.1151 [ hep - ph ] . s.  andreas , o.  lebedev , s.  ramos - sanchez and a.  ringwald , arxiv:1005.3978 [ hep - ph ] . j.  f.  gunion , jhep * 0908 * , 032 ( 2009 ) ; r. dermisek and j.  f.  gunion , phys .  rev . d * 81 * , 075003 ( 2010 ) . r.  dermisek and j.  f. gunion , phys . lett .   * 95 * , 041801 ( 2005 ) ; phys . d * 73 * , 111701 ( 2006 ) . j. cao , h. e. logan , j. m. yang , 79 , 091701 ( 2009 ) . j. cao , p. wan , l. wu , j. m. yang , 80 , 071701 ( 2009 ) . j. f. gunion and h. e. haber , 67 , 075019 ( 2003 ) . r.  m.  barnett , _ et al . _ , phys . b * 136 * , 191 ( 1984 ) ; r.  m.  barnett , g.  senjanovic and d.  wyler , phys . d * 30 * , 1529 ( 1984 ) ; y.  grossman , nucl . b * 426 * , 355 ( 1994 ) . h.  s.  goh , l.  j.  hall and p. kumar , jhep * 0905 * , 097 ( 2009 ) ; a.  g. akeroyd and w.  j.  stirling , nucl . b * 447 * , 3 ( 1995 ) ; a.  g.  akeroyd , phys . b * 377 * , 95 ( 1996 ) ; h.  e.  logan and d.  maclennan , phys .  rev . d * 79 * , 115022 ( 2009 ) ; m. aoki , _ et al . _ , arxiv:0902.4665 [ hep - ph ] . v.  barger , p.  langacker , h.  s.  lee and g. shaughnessy , phys . d * 73 * , 115010 ( 2006 ) . s. hesselbach , _ et . _ , arxiv:0810.0511v2 [ hep - ph ] . de vivie and p.  janot [ aleph collaboration ] , pa13 - 027 contribution to the international conference on high energy physics , warsaw , poland , 2531 july 1996 ; j. kurowska , o.  grajek and p.  zalewski [ delphi collaboration ] , cern - open-99 - 385 . [ aleph collaboration and delphi collaboration and l3 collaboration ] , phys . rept .   * 427 * , 257 ( 2006 ) . j.  cao and j.  m.  yang , jhep * 0812 * , 006 ( 2008 ) . m.  krawczyk and d.  temes , eur . j.   c * 44 * , 435 ( 2005 ) . g.  altarelli and r.  barbieri , 253 , 161 ( 1991 ) ; m. e. peskin , t. takeuchi , 46 , 381 ( 1992 ) . c. amsler , _ et al . _ , ( particle data group ) , 667 , 1 ( 2008 ) . o. deschamps , s.  descotes - genon , s.  monteil , v.  niess , s.  tjampens and v.  tisserand , arxiv:0907.5135 [ hep - ph ] . s.  su and b. thomas , phys . d * 79 * , 095014 ( 2009 ) . g. abbiendi , _ et al . _ , eur .  phys . j.   c * 32 * , 453 ( 2004 ) . m.  davier , _ et al . _ , 66 , 1 ( 2010 ) . k.  cheung , _ et al . _ , phys . d * 64 * , 111301 ( 2001 ) . k.  cheung and o.  c.  w. kong , phys . d * 68 * , 053003 ( 2003 ) . t. besmer , c. greub , t.hurth , 609 , 359 ( 2001 ) ; f. borzumati , _ et al . _ , 62 , 075005(2000 ) . j.  cao , k.  i.  hikasa , w.  wang , j.  m.  yang and l.  x.  yu , phys . d * 82 * , 051701 ( 2010 ) [ arxiv:1006.4811 [ hep - ph ] ] . j.  f.  gunion , _ et . d * 73 * , 015011 ( 2006 ) . martin and j.  d.  wells , phys . d * 64 * , 035003 ( 2001 ) . j.  abdallah _ et al . _ , eur . j.   c * 31 * , 421 ( 2004 ) ; g.  abbiendi _ et al . _ , eur . j. c * 35 * , 1 ( 2004 ) . j.  dunkley _ et al . _ [ wmap collaboration ] , astrophys . j.  suppl . * 180 * , 306 ( 2009 ) [ arxiv:0803.0586 [ astro - ph ] ] . u. ellwanger _ et al . _ , 02 , 066 ( 2005 ) . g.  belanger , f.  boudjema , a.  pukhov and a.  semenov , comput . commun .   * 174 * , 577 ( 2006 ) ; comput . phys .  commun . * 176 * , 367 ( 2007 ) . g.  belanger , f.  boudjema , c. hugonie , a.  pukhov and a.  semenov , jcap * 0509 * , 001 ( 2005 ) ."""

tests/led/test_modeling_led.py:512
  /home/circleci/transformers/tests/led/test_modeling_led.py:512: DeprecationWarning: invalid escape sequence \ 
    ARTICLE_MAGNET = """it is well known that the classical magnetoresistance ( mr ) in metals or semiconductors with a closed free electron fermi surface increases quadratically with increasing magnetic field @xmath2 for @xmath3 and saturates when @xmath4 . here @xmath5 is the zero - magnetic - field mobility . hence , the extraordinarily high and linear mr ( lmr ) , which breaks this familiar rule , has been gaining much attention as soon as its discovery . in the past decade , this unexpected lmr has been reported in silver chalcogenide,@xcite indium antimonide,@xcite silicon,@xcite mnas - gaas composite material,@xcite and graphene.@xcite    kapitza s linear law@xcite indicates that the metal shows a magnetoresistance linear in perpendicular magnetic field when it has an open fermi surface and a mean free path longer than the electronic larmor radius . recently , another two models , irrespective of the open fermi surface , have been constructed to provide possible mechanisms for the lmr phenomenon . abrikosov suggested a quantum - limit origin of lmr for the homogenous system with a gapless linear energy spectrum.@xcite his model requires that landau levels are well formed and the carrier concentration is small that all electrons occupy only the lowest landau band . alternatively , parish and littlewood developed a classical model without involving linear spectrum.@xcite ignoring the concrete microscopic mechanism , they attributed this unusual mr to the mobility fluctuations in a strongly inhomogenous system . topological insulators@xcite ( tis ) are novel materials with a full energy gap in bulk , while there are gapless surface states . due to its unique band structure with only one helical dirac cone and linear energy dispersion,@xcite the surface states of the ti bi@xmath0se@xmath1 become an excellent platform for the study of quantum - limit lmr . the recent experiment in this flat surface system , however , reported that a large positive mr , which becomes very linear above a characteristic field of @xmath6@xmath7@xmath8 t , was observed even in an opposite situation where the carrier sheet density is high that electrons occupy more than one landau levels.@xcite moreover , they found that raising temperature to room temperature almost has no influence on the observed lmr . it is striking that this observation is in conflict with abrikosov s model and also with the classical parish - littlewood model . so far a reliable theoretical scheme capable of explaining this novel experiment has still been lacking .    in this paper , we generalize the balance - equation approach@xcite to a system modeling the surface states of a three - dimensional ti to investigate the two - dimensional magnetotransport in it . we find that a positive , nonsaturating and dominantly linear magnetoresistance can appear within quite wide magnetic - field range in the ti surface state having a positive and finite effective g - factor . this linear magnetoresistance shows up in the system of high carrier concentration and low mobility when electrons are in extended states and spread over many smeared landau levels , and persists up to room temperature , providing a possible mechanism for the recently observed linear magnetoresistance in topological insulator bi@xmath0se@xmath1 nanoribbons.@xcite we consider the surface state of a bi@xmath0se@xmath1-type large bulk gap ti in the @xmath9-@xmath10 plane under the influence of a uniform magnetic field @xmath11 applied along the @xmath12 direction.@xcite following the experimental observation,@xcite we assume that the fermi energy locates in the gap of the bulk band and above the dirac point , i.e. the surface carriers are electrons . further , the separations of the fermi energy from the bottom of bulk band and dirac point are much larger than the highest temperature ( @xmath13 ) considered in this work . hence , the contribution from the bulk band to the magnetotransport is negligible . these electrons , scattered by randomly distributed impurities and by phonons , are driven by a uniform in - plane electric field @xmath14 in the topological surface . the hamiltonian of this many - electron and phonon system consists of an electron part @xmath15 , a phonon part @xmath16 , and electron - impurity and electron - phonon interactions @xmath17 and @xmath18 : @xmath19 here , the electron hamiltonian is taken in the form @xmath20 , \ ] ] in which @xmath21 , @xmath22 , @xmath23 and @xmath24 , stand , respectively , for the canonical momentum , coordinate , momentum and spin operators of the @xmath25th electron having charge @xmath26 , @xmath27 is the vector potential of the perpendicular magnetic field @xmath28 in the landau gauge , @xmath29 is the fermi velocity , @xmath30 is the effective g - factor of the surface electron , and @xmath31 is the bohr magneton with @xmath32 the free electron mass . the sum index @xmath25 in eq.([helectron ] ) goes over all electrons of total number @xmath33 in the surface state of unit area .    in the frame work of balance equation approach,@xcite the two - dimensional center - of - mass ( c.m . ) momentum and coordinate @xmath34 and @xmath35 , and the relative - electron momenta and coordinates @xmath36 and @xmath37 are introduced to write the hamiltonian @xmath15 into the sum of a single - particle c.m . part @xmath38 and a many - particle relative - electron part @xmath39 : @xmath40 , with @xmath41.\end{aligned}\ ] ] in this , @xmath42 is the canonical momentum of the center - of - mass and @xmath43 is the canonical momentum for the @xmath25th relative electron . here we have also introduced c.m . spin operators @xmath44 and @xmath45 . the commutation relations between the c.m . spin operators @xmath46 and @xmath47 and the spin operators @xmath48 , @xmath49 and @xmath50 of the @xmath25th electron are of order of @xmath51 : @xmath52= n^{-1}2\,{\rm i}\,\varepsi lon_{\beta_1\beta_2\beta_3}\sigma_j^{\beta_3}$ ] with @xmath53 . therefore , for a macroscopic large @xmath33 system , the c.m . part @xmath38 actually commutes with the relative - electron part @xmath54 in the hamiltonian , i.e. the c.m . motion and the relative motion of electrons are truly separated from each other . the couplings between the two emerge only through the electron impurity and electron  phonon interactions . furthermore , the electric field @xmath55 shows up only in @xmath38 . and , in view of @xmath56={\rm i}\delta_{\alpha \beta}(\delta_{ij}-1/n)\simeq { \rm i}\delta_{\alpha\beta}\delta_{ij}$ ] , i.e. the relative - electron momenta and coordinates can be treated as canonical conjugate variables , the relative - motion part @xmath54 is just the hamiltonian of @xmath33 electrons in the surface state of ti in the magnetic field without the presence of the electric field .    in terms of the c.m . coordinate @xmath57 and the relative electron density operator @xmath58 , the electron impurity and electron  phonon interactions can be written as@xcite @xmath59 here @xmath60 and @xmath61 are respectively the impurity potential ( an impurity at randomly distributed position @xmath62 ) and electron  phonon coupling matrix element in the plane - wave representation , and @xmath63 with @xmath64 and @xmath65 being the creation and annihilation operators for a phonon of wavevector @xmath66 in branch @xmath67 having frequency @xmath68 . velocity ( operator ) @xmath69 is the time variation of its coordinate : @xmath70= v_{\rm f}(\sigma_{\rm c}^y\ , \hat{i}-\sigma_{\rm c}^x\ , \hat{j})$ ] . to derive a force - balance equation for steady state transport we consider the heisenberg equation for the rate of change of the c.m . canonical momentum @xmath71 : @xmath72= - n e({\bm v}\times { \bm b})- n e{\bm e}+{\bm { f}}_{\rm i}+{\bm { f}}_{\rm p},\ ] ] in which the frictional forces @xmath73 and @xmath74 share the same expressions as given in ref ..    the statistical average of the operator equation can be determined to linear order in the electron  impurity and electron phonon interactions @xmath17 and @xmath18 with the initial density matrix @xmath75 at temperature @xmath76 when the in - plane electric field @xmath77 is not strong . for steady - transport states we have @xmath78 , leading to a force - balance equation of the form @xmath79 here @xmath80 , the statistically averaged velocity of the moving center - of - mass , is identified as the average rate of change of its position , i.e. the drift velocity of the electron system driven by the electric field @xmath77 , and @xmath81 and @xmath82 are frictional forces experienced by the center - of - mass due to impurity and phonon scatterings : @xmath83,\label{fp}\end{aligned}\ ] ] in which @xmath84 is the bose distribution function , @xmath85 , and @xmath86 stands for the imaginary part of the fourier spectrum of the relative - electron density correlation function defined by @xmath87\big\rangle_{0},\ ] ] where @xmath88 and @xmath89 denotes the statistical averaging over the initial density matrix @xmath90.@xcite    the force - balance equation describes the steady - state two - dimensional magnetotransport in the surface state of a ti . note that the frictional forces @xmath81 and @xmath82 are in the opposite direction of the drift velocity @xmath91 and their magnitudes are functions of @xmath92 only . with the drift velocity @xmath93 in the @xmath9 direction , the force - balance equation eq . yields a transverse resistivity @xmath94 , and a longitudinal resistivity @xmath95 . the linear one is in the form @xmath96 for calculating the electron density correlation function @xmath97 we proceed in the landau representation.@xcite the landau levels of the single - particle hamiltonian @xmath98 of the relative - electron system in the absence of electric field are composed of a positive `` @xmath99 '' and a negative `` @xmath100 '' branch@xcite @xmath101 with @xmath102 and @xmath103 , and a zero ( @xmath104 ) level @xmath105 the corresponding landau wave functions are @xmath106 and @xmath107 for @xmath108 ; and @xmath109 for @xmath104 . here @xmath110 is the wavevector of the system along @xmath9 direction ; @xmath111 with @xmath112 ; and @xmath113 is the harmonic oscillator eigenfunction with @xmath114 being the hermite polynomial , @xmath115 , and @xmath116 . each landau level contains @xmath117 electron states for system of unit surface area . the positive branch @xmath118 and the @xmath104 level @xmath119 of the above energy spectra are indeed quite close to those of the surface states in the bulk gap of bi@xmath0se@xmath1-family materials derived from microscopic band calculation.@xcite    the landau levels are broadened due to impurity , phonon and electron - electron scatterings . we model the imaginary part of the retarded green s function , or the density - of - states , of the broadened landau level @xmath120 ( written for `` + ' ' -branch and @xmath104 levels ) , using a gaussian - type form:@xcite @xmath121,\ ] ] with a half - width @xmath122 of the form:@xcite @xmath123^{1/2}$ ] . here @xmath124 is the single - particle lifetime and @xmath125 is the cyclotron frequency of linear - energy - dispersion system with @xmath126 being the zero - temperature fermi level . using a semi - empirical parameter @xmath127 to relate @xmath124 with the transport scattering time @xmath128 , and expressing @xmath129 with the zero - field mobility @xmath5 at finite temperature,@xcite we can write the landau - level broadening as @xmath130^{1/2}.\ ] ]    in the present study we consider the case of @xmath120-doping , i.e. the fermi level is high enough above the energy zero of the dirac cone in the range of `` + ' ' -branch levels and the states of `` @xmath100''-branch levels are completely filled , that they are irrelevant to electron transport . special attention has to be paid to the @xmath104 level , since , depending on the direction of exchange potential the effective g - factor of a ti surface state , @xmath30 , can be positive , zero or negative.@xcite the sign and magnitude of the effective g - factor determines how many states of the zero level should be included in or excluded from the available states for electron occupation in the case of @xmath120-doping at a magnetic field . ( i ) if @xmath131 , the @xmath104 level center is exactly at @xmath132 and the system is electron - hole symmetric . the total number of negative energy states ( including the states of the lower half of the @xmath104 level and states of the @xmath100"-branch levels ) and that of positive energy states ( including the states of the upper half of the @xmath104 level and states of the @xmath99"-branch levels ) do not change when changing magnetic field . therefore , the lower - half negative energy states of this level are always filled and the upper - half positive - energy states of it are available for the occupation of particles which are counted as electrons participating in transport in the case of @xmath120-doping . ( ii ) for a finite positive @xmath133 , the @xmath104 level @xmath134 moves downward to negative energy and its distance to the nearest  @xmath100"-branch level is @xmath135 closer than to the nearest  + " -branch level at finite magnetic field strength @xmath2 . this is equivalent to the opening of an increasingly enlarged ( with increasing @xmath2 ) energy gap between the  + " -branch states and the states of the zero - level and the  @xmath100"-branch levels . the opening of a sufficient energy gap implies that with increasing magnetic field the states in the  + " -branch levels would no longer shrink into the zero - level , and thus the @xmath104 level should be completely excluded from the conduction band , i.e. only particles occupying the  + " -branch states are counted as electrons participating in transport in the case of @xmath120-doping , when the magnetic field @xmath2 gets larger than a certain value ( depending on the magnitude of @xmath30 ) . ( iii ) for a finite negative @xmath136 , the @xmath104 level @xmath134 moves upward to positive energy and an increasingly enlarged energy gap will be opened between the states of the zero - level and the  + " -branch and the states of  @xmath100"-branch levels , and particles occupying the @xmath104 level and  + " -branch states are electrons participating in transport when the magnetic field @xmath2 gets larger than a certain value .    as a result , the experimentally accessible sheet density @xmath33 of electrons participating in transport is related to the fermi energy @xmath137 by the following equation valid at finite @xmath30 for the magnetic field @xmath2 larger than a certain value : @xmath138 in which @xmath139 + 1\}^{-1}$ ] is the fermi distribution function at temperature @xmath76 and the summation index @xmath120 goes over @xmath140 for @xmath133 , or @xmath141 for @xmath136 . in the case of @xmath131 , @xmath142\ ] ] valid for arbitrary magnetic field , in which @xmath143 . the imaginary part of relative - electron density correlation function in the presence of a magnetic field , @xmath86 , can be expressed in the landau representation as@xcite @xmath144 in which the transform factor @xmath145 ^ 2,\end{aligned}\ ] ] with @xmath146 , @xmath147 , @xmath148 , and @xmath149 being associated laguerre polynomials . the landau - representation correlation function @xmath150 in eq.([piqw ] ) can be constructed with the imaginary part of the retarded green s function @xmath151 , or the density - of - states , of the @xmath120th landau level as@xcite @xmath152\nonumber\\ & \hspace{1.2cm}\times{\rm im}g_n(\epsilon+\omega){\rm im}g_{n'}(\epsilon).\end{aligned}\ ] ] the summation indices @xmath120 and @xmath153 in eq.([piqw ] ) are taken over @xmath140 for @xmath133 , or @xmath154 for @xmath136 . in the case of @xmath131 , eq.([piqw ] ) still works and the summation indices @xmath120 and @xmath153 go over @xmath154 but with @xmath155 replaced by @xmath156 in eq.([p2nn ] ) . numerical calculations are performed for the magnetoresistivity @xmath157 of surface state in a uniform ti bi@xmath0se@xmath1 . at zero temperature the elastic scattering contributing to the resistivity is modeled by a coulomb potential due to charged impurities:@xcite @xmath158 with @xmath159 being the impurity density , which is determined by the zero - magnetic - field mobility @xmath5 . at temperatures higher than @xmath160,@xcite phonon scatterings play increasingly important role and the dominant inelastic contribution comes from optical phonons . for this polar material , the scattering by optical phonons via the deformation potential can be neglected . hence , we take account of inelastic scattering from optical phonons via frhlich coupling : @xmath161 . in the numerical calculation we use the following parameters:@xcite fermi velocity @xmath162 , static dielectric constant @xmath163 , optical dielectric constant @xmath164 , and phonon energy @xmath165 . the broadening parameter is taken to be @xmath166 . as a function of the magnetic field @xmath2 having different effective g - factors : @xmath167 and @xmath168 for a ti surface system with electron sheet density @xmath169 in the cases of zero - magnetic - field mobility @xmath170 ( a ) and @xmath171 ( b ) . several integer - number positions of filling factor @xmath172 are marked in ( b).,scaledwidth=40.0% ]    fig.[diffg ] shows the calculated magnetoresistivity @xmath157 versus the magnetic field strength @xmath2 for a ti surface system with electron sheet density @xmath169 but having different effective g - factors : @xmath167 and @xmath168 for two values of zero - magnetic - field mobility @xmath170 and @xmath171 , representing different degree of landau - level broadening . in the case without zeeman splitting ( @xmath131 ) the resistivity @xmath157 exhibits almost no change with changing magnetic field up to 10 t , except the shubnikov - de haas ( sdh ) oscillation showing up in the case of @xmath171 . this kind of magnetoresistance behavior was indeed seen experimentally in the electron - hole symmetrical massless system of single - layer graphene.@xcite in the case of a positive g - factor , @xmath173 , the magnetoresistivity increases linearly with increasing magnetic field ; while for a negative g - factor , @xmath174 , the magnetoresistivity decreases linearly with increasing magnetic field . is shown as a function of the magnetic field @xmath2 for different values of zero - magnetic - field mobility : ( a ) @xmath175 , ( b ) @xmath176 , ( c ) @xmath177 , ( d ) @xmath178 , ( e ) @xmath179 , and ( f ) @xmath180 . the inset of ( a ) illustrates the same for a larger magnetic - field range @xmath181 . the filling factor @xmath182 is plotted versus the magnetic field in ( f ) ; and several integer - number positions of @xmath182 are also marked in ( d ) and ( e ) . here the surface electron density @xmath169 and the lattice temperature @xmath183.,scaledwidth=47.0% ]    in the following we will give more detailed examination on the linearly increasing magnetoresistance in the positive @xmath30 case . fig.[rhob ] shows the calculated resistivity @xmath157 versus the magnetic field strength @xmath2 at lattice temperature @xmath183 for system of carrier sheet density @xmath169 and @xmath173 , having different zero - field mobility @xmath184 and @xmath180 . all resistivity curves for mobility @xmath185 exhibit clear linearity in the magnetic - field range and appear no tendency of saturation at the highest field shown in the figure . especially , for the case @xmath170 , the linear behavior extends even up to the magnetic field of @xmath186 , as illustrated in the inset of fig.[rhob](a ) . this feature contradicts the classical mr which saturates at sufficiently large magnetic field @xmath187 . note that here we only present the calculated @xmath157 for magnetic field @xmath2 larger than @xmath188 t , for which a sufficient energy gap @xmath135 is assumed to open that with further increase of the magnetic field the states in the `` + ' ' -branch levels no longer shrink into the zero level and thus it should be excluded from the conduction band . this is of course not true for very weak magnetic field . when @xmath189 the energy gap @xmath190 , the situation becomes similar to the case of @xmath131 : the whole upper half of the zero - level states are available to electron occupation and we should have a flat resistivity @xmath157 when changing magnetic field . with increasing @xmath2 the portion of the zero - level states available to conduction electrons decreases until the magnetic field reaches @xmath191 . as a result the resistivity @xmath157 should exhibit a crossover from a flat changing at small @xmath2 to positively linear increasing at @xmath192 . this is just the behavior observed in the ti bi@xmath0se@xmath1.@xcite    note that in the case of @xmath170 , the broadened landau - level widths are always larger than the neighboring level interval : @xmath193 , which requires @xmath194 ^ 2 $ ] , even for the lowest landau level @xmath195 , i.e. the whole landau - level spectrum is smeared . with increasing the zero - field mobility the magnitude of resistivity @xmath157 decreases , and when the broadened landau - level width becomes smaller than the neighboring level interval , @xmath196 , a weak sdh oscillation begin to occur around the linearly - dependent average value of @xmath157 at higher portion of the magnetic field range , as seen in fig.[rhob](c ) , ( d ) and ( e ) for @xmath197 and @xmath198 . on the other hand , in the case of large mobility , e.g. @xmath199 , where the broadened landau - level widths @xmath200 are much smaller than the neighboring level interval even for level index @xmath120 as large as @xmath201 , the magnetoresistivity shows pronounced sdh oscillation and the linear - dependent behavior disappears , before the appearance of quantum hall effect,@xcite as shown in fig.[rhob](f ) . abrikosov s model for the lmr requires the applied magnetic field large enough to reach the quantum limit at which all the carriers are within the lowest landau level,@xcite while it is obvious that more than one landau levels are occupied in the experimental samples in the field range in which the linear and non - saturating magnetoresistivity was observed.@xcite for the given electron surface density @xmath202 , the number of occupied landau levels , or the filling factor @xmath172 , at different magnetic fields is shown in fig.[rhob](f ) , as well as in the fig.[rhob](d ) and ( e ) , where the integer - number positions of @xmath203 , i.e. filling up to entire @xmath182 landau levels , coincide with the minima of the density - of - states or the dips of sdh oscillation . this is in contrast with @xmath131 case , where the integer number of @xmath203 , which implies a filling up to the center position of the @xmath182th landau levels , locates at a peak of sdh oscillation , as shown in fig.[diffg]b . the observed sdh oscillations in the bi@xmath0se@xmath1 nanoribbon exhibiting nonsaturating surface lmr in the experiment@xcite favor the former case : a finite positive effective @xmath133 .     is plotted as a function of the surface electron density @xmath33 at magnetic field @xmath204 : ( a ) at different values of zero - field mobility @xmath5 , and ( b ) at different values of zero - field conductivity @xmath205.,scaledwidth=40.0% ]     at various lattice temperatures . here the zero - magnetic - field mobility at zero temperature is @xmath206.,scaledwidth=35.0% ]    next , we examine the density - dependence of the linear magnetoresistivity . to compare with abrikosov s quantum magnetoresistance which suggests a @xmath207 behavior,@xcite we show the calculated @xmath208 for above lmr versus the carrier sheet density @xmath33 in fig.[rhon ] at fixed magnetic field @xmath209 t . the mobility is taken respectively to be @xmath210 and @xmath211m@xmath212/vs to make the resistivity in the lmr regime . a clearly linear dependence of @xmath213 on the surface density @xmath33 is seen in all cases , indicating that this non - saturating linear resistivity is almost inversely proportional to the carrier density . in the figure we also show @xmath208 versus @xmath33 under the condition of different given conductivity @xmath214 and @xmath215 . in this case the half - width @xmath216 is independent of surface density . the linear dependence still holds , indicating that this linear behavior is not sensitive to the modest @xmath33-dependence of landau level broadening @xmath216 as long as the system is in the overlapped landau level regime . from the above discussion , it is obvious that lmr shows up in the system having overlapped landau levels and the separation of landau levels makes the mr departure from the linear increase . at high temperature , the thermal energy would smear the level separation and phonon scatterings further broaden landau levels . hence , it is believed that this lmr will be robust against raising temperature . this is indeed the case as seen in fig.[rhot ] , where we plot the calculated magnetoresistivity @xmath157 for the above system with zero - temperature linear mobility @xmath217m@xmath212/vs versus the magnetic field at different lattice temperatures . we can see that raising temperature to room temperature has little effect on the linearity of mr . due to the decreased mobility at higher temperature from phonon scattering , the weak sdh oscillation on the linear background tends to vanish . these features are in good agreement with the experimental report.@xcite in summary , we have studied the two - dimensional magnetotransport in the flat surface of a three - dimensional ti , which arises from the surface states with a wavevector - linear energy dispersion and a finite , positive zeeman splitting within the bulk energy gap . when the level broadening is comparable to or larger than the landau - level separation and the conduction electrons spread over many landau levels , a positive , dominantly linear and non - saturating magnetoresistance appears within a quite wide range of magnetic field and persists up to room temperature . this remarkable lmr provides a possible mechanism for the recently observed linear magnetoresistance in topological insulator bi@xmath0se@xmath1 nanoribbons.@xcite    in contrast to quantum hall effect which appears in the case of well formed landau levels and to abrikosov s quantum magnetotransport,@xcite which is limited to the extreme quantum limit that all electrons coalesce into the lowest landau level , the discussed lmr is a phenomena of pure classical two - dimensional magnetotransport in a system having linear - energy - dispersion , appearing in the regime of overlapped landau levels , irrespective of its showing up in relatively high magnetic field range . furthermore , the present scheme deals with spatially uniform case without invoking the mobility fluctuation in a strongly inhomogeneous system , which is required in the classical parish and littlewood model to produce a lmr.@xcite    the appearance of this significant positive - increasing linear magnetoresistance depends on the existence of a positive and sizable effective g - factor . if the zeeman energy splitting is quite small the resistivity @xmath157 would exhibit little change with changing magnetic field . in the case of a negative and sizable effective g - factor the magnetoresistivity would decrease linearly with increasing magnetic field . therefore , the behavior of the longitudinal resistivity versus magnetic field may provide a useful way for judging the direction and the size of the effective zeeman energy splitting in ti surface states . this work was supported by the national science foundation of china ( grant no . 11104002 ) , the national basic research program of china ( grant no . 2012cb927403 ) and by the program for science&technology innovation talents in universities of henan province ( grant no . 2012hastit029 ) ."""

src/transformers/models/maskformer/modeling_maskformer.py:2018
  /home/circleci/transformers/src/transformers/models/maskformer/modeling_maskformer.py:2018: DeprecationWarning: invalid escape sequence \e
    """

src/transformers/optimization.py:309
tests/trainer/test_trainer.py::TrainerIntegrationWithHubTester::test_push_to_hub_with_saves_each_epoch
tests/trainer/test_trainer.py::TrainerIntegrationWithHubTester::test_push_to_hub_with_saves_each_n_steps
  /home/circleci/transformers/src/transformers/optimization.py:309: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning
    FutureWarning,

tests/test_modeling_common.py::ModelPushToHubTester::test_push_to_hub
tests/trainer/test_trainer.py::TrainerIntegrationWithHubTester::test_push_to_hub
  /home/circleci/.local/lib/python3.7/site-packages/huggingface_hub/hf_api.py:510: FutureWarning: HfApi.login: This method is deprecated in favor of `set_access_token` and will be removed in v0.7.
    FutureWarning,

tests/test_modeling_common.py::ModelPushToHubTester::test_push_to_hub
  /home/circleci/.local/lib/python3.7/site-packages/huggingface_hub/utils/_deprecation.py:43: FutureWarning: Pass token='test-model' as keyword args. From version 0.7 passing these as positional arguments will result in an error
    FutureWarning,

tests/test_modeling_common.py::ModelPushToHubTester::test_push_to_hub
tests/test_modeling_common.py::ModelPushToHubTester::test_push_to_hub_in_organization
  /home/circleci/.local/lib/python3.7/site-packages/huggingface_hub/hf_api.py:599: FutureWarning: `create_repo` now takes `token` as an optional positional argument. Be sure to adapt your code!
    FutureWarning,

tests/test_modeling_common.py::ModelPushToHubTester::test_push_to_hub_in_organization
  /home/circleci/.local/lib/python3.7/site-packages/huggingface_hub/utils/_deprecation.py:43: FutureWarning: Pass token='test-model-org' as keyword args. From version 0.7 passing these as positional arguments will result in an error
    FutureWarning,

tests/test_modeling_common.py::ModelPushToHubTester::test_push_to_hub_in_organization
tests/test_modeling_common.py::ModelPushToHubTester::test_push_to_hub_in_organization
tests/trainer/test_trainer.py::TrainerIntegrationWithHubTester::test_push_to_hub_with_saves_each_n_steps
  /home/circleci/.local/lib/python3.7/site-packages/huggingface_hub/hf_api.py:82: FutureWarning: `name` and `organization` input arguments are deprecated and will be removed in v0.7. Pass `repo_id` instead.
    FutureWarning,

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED tests/trainer/test_trainer.py::TrainerIntegrationWithHubTester::test_push_to_hub
= 1 failed, 6 passed, 3 skipped, 10120 deselected, 21 warnings in 98.05s (0:01:38) =

Exited with code exit status 1
CircleCI received exit code 1
```
ArEnSc: @ydshieh I think we should be good to go here let me know =)
ydshieh: @ArEnSc Would you mind to remove the comments with (very) long error message - just easier to load this page :-) Thanks!
ArEnSc: @ydshieh looks like it good to go!
ArEnSc: @ydshieh I think this is done after CI passes =)

Patch:
diff --git a/docs/source/en/model_doc/byt5.mdx b/docs/source/en/model_doc/byt5.mdx
index 06ed195226..dc4c5a6caf 100644
--- a/docs/source/en/model_doc/byt5.mdx
+++ b/docs/source/en/model_doc/byt5.mdx
@@ -48,37 +48,98 @@ fine-tuning. If you are doing multi-task fine-tuning, you should use a prefix.
 ByT5 works on raw UTF-8 bytes, so it can be used without a tokenizer:
 
 ```python
-from transformers import T5ForConditionalGeneration
-import torch
+>>> from transformers import T5ForConditionalGeneration
+>>> import torch
 
-model = T5ForConditionalGeneration.from_pretrained("google/byt5-small")
+>>> model = T5ForConditionalGeneration.from_pretrained("google/byt5-small")
 
-input_ids = torch.tensor([list("Life is like a box of chocolates.".encode("utf-8"))]) + 3  # add 3 for special tokens
-labels = (
-    torch.tensor([list("La vie est comme une boîte de chocolat.".encode("utf-8"))]) + 3
-)  # add 3 for special tokens
+>>> num_special_tokens = 3
+>>> # Model has 3 special tokens which take up the input ids 0,1,2 of ByT5.
+>>> # => Need to shift utf-8 character encodings by 3 before passing ids to model.
 
-loss = model(input_ids, labels=labels).loss  # forward pass
+>>> input_ids = torch.tensor([list("Life is like a box of chocolates.".encode("utf-8"))]) + num_special_tokens
+
+>>> labels = torch.tensor([list("La vie est comme une boîte de chocolat.".encode("utf-8"))]) + num_special_tokens
+
+>>> loss = model(input_ids, labels=labels).loss
+>>> loss.item()
+2.66
 ```
 
 For batched inference and training it is however recommended to make use of the tokenizer:
 
 ```python
-from transformers import T5ForConditionalGeneration, AutoTokenizer
-
-model = T5ForConditionalGeneration.from_pretrained("google/byt5-small")
-tokenizer = AutoTokenizer.from_pretrained("google/byt5-small")
+>>> from transformers import T5ForConditionalGeneration, AutoTokenizer
+
+>>> model = T5ForConditionalGeneration.from_pretrained("google/byt5-small")
+>>> tokenizer = AutoTokenizer.from_pretrained("google/byt5-small")
+
+>>> model_inputs = tokenizer(
+...     ["Life is like a box of chocolates.", "Today is Monday."], padding="longest", return_tensors="pt"
+... )
+>>> labels_dict = tokenizer(
+...     ["La vie est comme une boîte de chocolat.", "Aujourd'hui c'est lundi."], padding="longest", return_tensors="pt"
+... )
+>>> labels = labels_dict.input_ids
+
+>>> loss = model(**model_inputs, labels=labels).loss
+>>> loss.item()
+17.9
+```
 
-model_inputs = tokenizer(
-    ["Life is like a box of chocolates.", "Today is Monday."], padding="longest", return_tensors="pt"
-)
-labels = tokenizer(
-    ["La vie est comme une boîte de chocolat.", "Aujourd'hui c'est lundi."], padding="longest", return_tensors="pt"
-).input_ids
+Similar to [T5](t5), ByT5 was trained on the span-mask denoising task. However, 
+since the model works directly on characters, the pretraining task is a bit 
+different. Let's corrupt some characters of the 
+input sentence `"The dog chases a ball in the park."` and ask ByT5 to predict them 
+for us.
 
-loss = model(**model_inputs, labels=labels).loss  # forward pass
+```python
+>>> from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
+>>> import torch
+
+>>> tokenizer = AutoTokenizer.from_pretrained("google/byt5-base")
+>>> model = AutoModelForSeq2SeqLM.from_pretrained("google/byt5-base")
+
+>>> input_ids_prompt = "The dog chases a ball in the park."
+>>> input_ids = tokenizer(input_ids_prompt).input_ids
+
+>>> # Note that we cannot add "{extra_id_...}" to the string directly
+>>> # as the Byte tokenizer would incorrectly merge the tokens
+>>> # For ByT5, we need to work directly on the character level
+>>> # Contrary to T5, ByT5 does not use sentinel tokens for masking, but instead
+>>> # uses final utf character ids.
+>>> # UTF-8 is represented by 8 bits and ByT5 has 3 special tokens.
+>>> # => There are 2**8+2 = 259 input ids and mask tokens count down from index 258.
+>>> # => mask to "The dog [258]a ball [257]park."
+
+>>> input_ids = torch.tensor([input_ids[:8] + [258] + input_ids[14:21] + [257] + input_ids[28:]])
+>>> input_ids
+tensor([[ 87, 107, 104,  35, 103, 114, 106,  35, 258,  35, 100,  35, 101, 100, 111, 111, 257,  35, 115, 100, 117, 110,  49,   1]])
+
+>>> # ByT5 produces only one char at a time so we need to produce many more output characters here -> set `max_length=100`.
+>>> output_ids = model.generate(input_ids, max_length=100)[0].tolist()
+>>> output_ids
+[0, 258, 108, 118,  35, 119, 107, 104,  35, 114, 113, 104,  35, 122, 107, 114,  35, 103, 114, 104, 118, 257,  35, 108, 113,  35, 119, 107, 104,  35, 103, 108, 118, 102, 114, 256, 108, 113,  35, 119, 107, 104, 35, 115, 100, 117, 110,  49,  35,  87, 107, 104,  35, 103, 114, 106, 35, 108, 118,  35, 119, 107, 104,  35, 114, 113, 104,  35, 122, 107, 114,  35, 103, 114, 104, 118,  35, 100,  35, 101, 100, 111, 111,  35, 108, 113, 255,  35, 108, 113,  35, 119, 107, 104,  35, 115, 100, 117, 110,  49]
+
+>>> # ^- Note how 258 descends to 257, 256, 255
+
+>>> # Now we need to split on the sentinel tokens, let's write a short loop for this
+>>> output_ids_list = []
+>>> start_token = 0
+>>> sentinel_token = 258
+>>> while sentinel_token in output_ids:
+...     split_idx = output_ids.index(sentinel_token)
+...     output_ids_list.append(output_ids[start_token:split_idx])
+...     start_token = split_idx
+...     sentinel_token -= 1
+
+>>> output_ids_list.append(output_ids[start_token:])
+>>> output_string = tokenizer.batch_decode(output_ids_list)
+>>> output_string
+['<pad>', 'is the one who does', ' in the disco', 'in the park. The dog is the one who does a ball in', ' in the park.']
 ```
 
+
 ## ByT5Tokenizer
 
 [[autodoc]] ByT5Tokenizer
diff --git a/docs/source/en/model_doc/t5.mdx b/docs/source/en/model_doc/t5.mdx
index ef605a3523..c312b3df81 100644
--- a/docs/source/en/model_doc/t5.mdx
+++ b/docs/source/en/model_doc/t5.mdx
@@ -32,9 +32,9 @@ NLP, we release our dataset, pre-trained models, and code.*
 Tips:
 
 - T5 is an encoder-decoder model pre-trained on a multi-task mixture of unsupervised and supervised tasks and for which
-  each task is converted into a text-to-text format. T5 works well on a variety of tasks out-of-the-box by prepending a
-  different prefix to the input corresponding to each task, e.g., for translation: *translate English to German: ...*,
-  for summarization: *summarize: ...*.
+each task is converted into a text-to-text format. T5 works well on a variety of tasks out-of-the-box by prepending a
+different prefix to the input corresponding to each task, e.g., for translation: *translate English to German: ...*,
+for summarization: *summarize: ...*.
 
 - T5 uses relative scalar embeddings. Encoder input padding can be done on the left and on the right.
 
@@ -83,130 +83,140 @@ language modeling head on top of the decoder.
 
 - Unsupervised denoising training
 
-  In this setup, spans of the input sequence are masked by so-called sentinel tokens (*a.k.a* unique mask tokens) and
-  the output sequence is formed as a concatenation of the same sentinel tokens and the *real* masked tokens. Each
-  sentinel token represents a unique mask token for this sentence and should start with `<extra_id_0>`,
-  `<extra_id_1>`, ... up to `<extra_id_99>`. As a default, 100 sentinel tokens are available in
-  [`T5Tokenizer`].
+In this setup, spans of the input sequence are masked by so-called sentinel tokens (*a.k.a* unique mask tokens) and
+the output sequence is formed as a concatenation of the same sentinel tokens and the *real* masked tokens. Each
+sentinel token represents a unique mask token for this sentence and should start with `<extra_id_0>`,
+`<extra_id_1>`, ... up to `<extra_id_99>`. As a default, 100 sentinel tokens are available in
+[`T5Tokenizer`].
 
-  For instance, the sentence "The cute dog walks in the park" with the masks put on "cute dog" and "the" should be
-  processed as follows:
+For instance, the sentence "The cute dog walks in the park" with the masks put on "cute dog" and "the" should be
+processed as follows:
 
-  ```python
-  from transformers import T5Tokenizer, T5ForConditionalGeneration
+```python
+>>> from transformers import T5Tokenizer, T5ForConditionalGeneration
+
+>>> tokenizer = T5Tokenizer.from_pretrained("t5-small")
+>>> model = T5ForConditionalGeneration.from_pretrained("t5-small")
 
-  tokenizer = T5Tokenizer.from_pretrained("t5-small")
-  model = T5ForConditionalGeneration.from_pretrained("t5-small")
+>>> input_ids = tokenizer("The <extra_id_0> walks in <extra_id_1> park", return_tensors="pt").input_ids
+>>> labels = tokenizer("<extra_id_0> cute dog <extra_id_1> the <extra_id_2>", return_tensors="pt").input_ids
 
-  input_ids = tokenizer("The <extra_id_0> walks in <extra_id_1> park", return_tensors="pt").input_ids
-  labels = tokenizer("<extra_id_0> cute dog <extra_id_1> the <extra_id_2>", return_tensors="pt").input_ids
-  # the forward function automatically creates the correct decoder_input_ids
-  loss = model(input_ids=input_ids, labels=labels).loss
-  ```
+>>> # the forward function automatically creates the correct decoder_input_ids
+>>> loss = model(input_ids=input_ids, labels=labels).loss
+>>> loss.item()
+3.7837
+```
 
-  If you're interested in pre-training T5 on a new corpus, check out the [run_t5_mlm_flax.py](https://github.com/huggingface/transformers/tree/main/examples/flax/language-modeling) script in the Examples
-  directory.
+If you're interested in pre-training T5 on a new corpus, check out the [run_t5_mlm_flax.py](https://github.com/huggingface/transformers/tree/main/examples/flax/language-modeling) script in the Examples
+directory.
 
 - Supervised training
 
-  In this setup, the input sequence and output sequence are a standard sequence-to-sequence input-output mapping.
-  Suppose that we want to fine-tune the model for translation for example, and we have a training example: the input
-  sequence "The house is wonderful." and output sequence "Das Haus ist wunderbar.", then they should be prepared for
-  the model as follows:
-
-  ```python
-  from transformers import T5Tokenizer, T5ForConditionalGeneration
-
-  tokenizer = T5Tokenizer.from_pretrained("t5-small")
-  model = T5ForConditionalGeneration.from_pretrained("t5-small")
-
-  input_ids = tokenizer("translate English to German: The house is wonderful.", return_tensors="pt").input_ids
-  labels = tokenizer("Das Haus ist wunderbar.", return_tensors="pt").input_ids
-  # the forward function automatically creates the correct decoder_input_ids
-  loss = model(input_ids=input_ids, labels=labels).loss
-  ```
-
-  As you can see, only 2 inputs are required for the model in order to compute a loss: `input_ids` (which are the
-  `input_ids` of the encoded input sequence) and `labels` (which are the `input_ids` of the encoded
-  target sequence). The model will automatically create the `decoder_input_ids` based on the `labels`, by
-  shifting them one position to the right and prepending the `config.decoder_start_token_id`, which for T5 is
-  equal to 0 (i.e. the id of the pad token). Also note the task prefix: we prepend the input sequence with 'translate
-  English to German: ' before encoding it. This will help in improving the performance, as this task prefix was used
-  during T5's pre-training.
-
-  However, the example above only shows a single training example. In practice, one trains deep learning models in
-  batches. This entails that we must pad/truncate examples to the same length. For encoder-decoder models, one
-  typically defines a `max_source_length` and `max_target_length`, which determine the maximum length of the
-  input and output sequences respectively (otherwise they are truncated). These should be carefully set depending on
-  the task.
-
-  In addition, we must make sure that padding token id's of the `labels` are not taken into account by the loss
-  function. In PyTorch and Tensorflow, this can be done by replacing them with -100, which is the `ignore_index`
-  of the `CrossEntropyLoss`. In Flax, one can use the `decoder_attention_mask` to ignore padded tokens from
-  the loss (see the [Flax summarization script](https://github.com/huggingface/transformers/tree/main/examples/flax/summarization) for details). We also pass
-  `attention_mask` as additional input to the model, which makes sure that padding tokens of the inputs are
-  ignored. The code example below illustrates all of this.
-
-  ```python
-  from transformers import T5Tokenizer, T5ForConditionalGeneration
-  import torch
-
-  tokenizer = T5Tokenizer.from_pretrained("t5-small")
-  model = T5ForConditionalGeneration.from_pretrained("t5-small")
-
-  # the following 2 hyperparameters are task-specific
-  max_source_length = 512
-  max_target_length = 128
-
-  # Suppose we have the following 2 training examples:
-  input_sequence_1 = "Welcome to NYC"
-  output_sequence_1 = "Bienvenue à NYC"
-
-  input_sequence_2 = "HuggingFace is a company"
-  output_sequence_2 = "HuggingFace est une entreprise"
-
-  # encode the inputs
-  task_prefix = "translate English to French: "
-  input_sequences = [input_sequence_1, input_sequence_2]
-  encoding = tokenizer(
-      [task_prefix + sequence for sequence in input_sequences],
-      padding="longest",
-      max_length=max_source_length,
-      truncation=True,
-      return_tensors="pt",
-  )
-  input_ids, attention_mask = encoding.input_ids, encoding.attention_mask
-
-  # encode the targets
-  target_encoding = tokenizer(
-      [output_sequence_1, output_sequence_2], padding="longest", max_length=max_target_length, truncation=True
-  )
-  labels = target_encoding.input_ids
-
-  # replace padding token id's of the labels by -100
-  labels = torch.tensor(labels)
-  labels[labels == tokenizer.pad_token_id] = -100
-
-  # forward pass
-  loss = model(input_ids=input_ids, attention_mask=attention_mask, labels=labels).loss
-  ```
+In this setup, the input sequence and output sequence are a standard sequence-to-sequence input-output mapping.
+Suppose that we want to fine-tune the model for translation for example, and we have a training example: the input
+sequence "The house is wonderful." and output sequence "Das Haus ist wunderbar.", then they should be prepared for
+the model as follows:
+
+```python
+>>> from transformers import T5Tokenizer, T5ForConditionalGeneration
+
+>>> tokenizer = T5Tokenizer.from_pretrained("t5-small")
+>>> model = T5ForConditionalGeneration.from_pretrained("t5-small")
+
+>>> input_ids = tokenizer("translate English to German: The house is wonderful.", return_tensors="pt").input_ids
+>>> labels = tokenizer("Das Haus ist wunderbar.", return_tensors="pt").input_ids
+
+>>> # the forward function automatically creates the correct decoder_input_ids
+>>> loss = model(input_ids=input_ids, labels=labels).loss
+>>> loss.item()
+0.2542
+```
+
+As you can see, only 2 inputs are required for the model in order to compute a loss: `input_ids` (which are the
+`input_ids` of the encoded input sequence) and `labels` (which are the `input_ids` of the encoded
+target sequence). The model will automatically create the `decoder_input_ids` based on the `labels`, by
+shifting them one position to the right and prepending the `config.decoder_start_token_id`, which for T5 is
+equal to 0 (i.e. the id of the pad token). Also note the task prefix: we prepend the input sequence with 'translate
+English to German: ' before encoding it. This will help in improving the performance, as this task prefix was used
+during T5's pre-training.
+
+However, the example above only shows a single training example. In practice, one trains deep learning models in
+batches. This entails that we must pad/truncate examples to the same length. For encoder-decoder models, one
+typically defines a `max_source_length` and `max_target_length`, which determine the maximum length of the
+input and output sequences respectively (otherwise they are truncated). These should be carefully set depending on
+the task.
+
+In addition, we must make sure that padding token id's of the `labels` are not taken into account by the loss
+function. In PyTorch and Tensorflow, this can be done by replacing them with -100, which is the `ignore_index`
+of the `CrossEntropyLoss`. In Flax, one can use the `decoder_attention_mask` to ignore padded tokens from
+the loss (see the [Flax summarization script](https://github.com/huggingface/transformers/tree/main/examples/flax/summarization) for details). We also pass
+`attention_mask` as additional input to the model, which makes sure that padding tokens of the inputs are
+ignored. The code example below illustrates all of this.
+
+```python
+>>> from transformers import T5Tokenizer, T5ForConditionalGeneration
+>>> import torch
+
+>>> tokenizer = T5Tokenizer.from_pretrained("t5-small")
+>>> model = T5ForConditionalGeneration.from_pretrained("t5-small")
+
+>>> # the following 2 hyperparameters are task-specific
+>>> max_source_length = 512
+>>> max_target_length = 128
+
+>>> # Suppose we have the following 2 training examples:
+>>> input_sequence_1 = "Welcome to NYC"
+>>> output_sequence_1 = "Bienvenue à NYC"
+
+>>> input_sequence_2 = "HuggingFace is a company"
+>>> output_sequence_2 = "HuggingFace est une entreprise"
+
+>>> # encode the inputs
+>>> task_prefix = "translate English to French: "
+>>> input_sequences = [input_sequence_1, input_sequence_2]
+
+>>> encoding = tokenizer(
+...     [task_prefix + sequence for sequence in input_sequences],
+...     padding="longest",
+...     max_length=max_source_length,
+...     truncation=True,
+...     return_tensors="pt",
+... )
+
+>>> input_ids, attention_mask = encoding.input_ids, encoding.attention_mask
+
+>>> # encode the targets
+>>> target_encoding = tokenizer(
+...     [output_sequence_1, output_sequence_2], padding="longest", max_length=max_target_length, truncation=True
+... )
+>>> labels = target_encoding.input_ids
+
+>>> # replace padding token id's of the labels by -100 so it's ignored by the loss
+>>> labels = torch.tensor(labels)
+>>> labels[labels == tokenizer.pad_token_id] = -100
+
+>>> # forward pass
+>>> loss = model(input_ids=input_ids, attention_mask=attention_mask, labels=labels).loss
+>>> loss.item()
+0.188
+```
 
 Additional training tips:
 
 - T5 models need a slightly higher learning rate than the default one set in the `Trainer` when using the AdamW
-  optimizer. Typically, 1e-4 and 3e-4 work well for most problems (classification, summarization, translation, question
-  answering, question generation). Note that T5 was pre-trained using the AdaFactor optimizer.
+optimizer. Typically, 1e-4 and 3e-4 work well for most problems (classification, summarization, translation, question
+answering, question generation). Note that T5 was pre-trained using the AdaFactor optimizer.
 
-- According to [this forum post](https://discuss.huggingface.co/t/t5-finetuning-tips/684), task prefixes matter when
-  (1) doing multi-task training (2) your task is similar or related to one of the supervised tasks used in T5's
-  pre-training mixture (see Appendix D of the [paper](https://arxiv.org/pdf/1910.10683.pdf) for the task prefixes
-  used).
+According to [this forum post](https://discuss.huggingface.co/t/t5-finetuning-tips/684), task prefixes matter when
+(1) doing multi-task training (2) your task is similar or related to one of the supervised tasks used in T5's
+pre-training mixture (see Appendix D of the [paper](https://arxiv.org/pdf/1910.10683.pdf) for the task prefixes
+used).
 
-- If training on TPU, it is recommended to pad all examples of the dataset to the same length or make use of
-  *pad_to_multiple_of* to have a small number of predefined bucket sizes to fit all examples in. Dynamically padding
-  batches to the longest example is not recommended on TPU as it triggers a recompilation for every batch shape that is
-  encountered during training thus significantly slowing down the training. only padding up to the longest example in a
-  batch) leads to very slow training on TPU.
+If training on TPU, it is recommended to pad all examples of the dataset to the same length or make use of
+*pad_to_multiple_of* to have a small number of predefined bucket sizes to fit all examples in. Dynamically padding
+batches to the longest example is not recommended on TPU as it triggers a recompilation for every batch shape that is
+encountered during training thus significantly slowing down the training. only padding up to the longest example in a
+batch) leads to very slow training on TPU.
 
 <a id='inference'></a>
 
@@ -219,15 +229,15 @@ There's also [this blog post](https://huggingface.co/blog/encoder-decoder#encode
 generation works in general in encoder-decoder models.
 
 ```python
-from transformers import T5Tokenizer, T5ForConditionalGeneration
+>>> from transformers import T5Tokenizer, T5ForConditionalGeneration
 
-tokenizer = T5Tokenizer.from_pretrained("t5-small")
-model = T5ForConditionalGeneration.from_pretrained("t5-small")
+>>> tokenizer = T5Tokenizer.from_pretrained("t5-small")
+>>> model = T5ForConditionalGeneration.from_pretrained("t5-small")
 
-input_ids = tokenizer("translate English to German: The house is wonderful.", return_tensors="pt").input_ids
-outputs = model.generate(input_ids)
-print(tokenizer.decode(outputs[0], skip_special_tokens=True))
-# Das Haus ist wunderbar.
+>>> input_ids = tokenizer("translate English to German: The house is wonderful.", return_tensors="pt").input_ids
+>>> outputs = model.generate(input_ids)
+>>> print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+Das Haus ist wunderbar.
 ```
 
 Note that T5 uses the `pad_token_id` as the `decoder_start_token_id`, so when doing generation without using
@@ -236,31 +246,47 @@ Note that T5 uses the `pad_token_id` as the `decoder_start_token_id`, so when do
 The example above only shows a single example. You can also do batched inference, like so:
 
 ```python
-from transformers import T5Tokenizer, T5ForConditionalGeneration
-
-tokenizer = T5Tokenizer.from_pretrained("t5-small")
-model = T5ForConditionalGeneration.from_pretrained("t5-small")
+>>> from transformers import T5Tokenizer, T5ForConditionalGeneration
+
+>>> tokenizer = T5Tokenizer.from_pretrained("t5-small")
+>>> model = T5ForConditionalGeneration.from_pretrained("t5-small")
+
+>>> task_prefix = "translate English to German: "
+>>> sentences = [
+...     "The house is wonderful.",
+...     "I like to work in NYC.",
+>>> ]  # use different length sentences to test batching
+>>> inputs = tokenizer([task_prefix + sentence for sentence in sentences], return_tensors="pt", padding=True)
+
+>>> output_sequences = model.generate(
+...     input_ids=inputs["input_ids"],
+...     attention_mask=inputs["attention_mask"],
+...     do_sample=False,  # disable sampling to test if batching affects output
+... )
+
+>>> print(tokenizer.batch_decode(output_sequences, skip_special_tokens=True))
+['Das Haus ist wunderbar.', 'Ich arbeite gerne in NYC.']
+```
 
-# when generating, we will use the logits of right-most token to predict the next token
-# so the padding should be on the left
-tokenizer.padding_side = "left"
-tokenizer.pad_token = tokenizer.eos_token  # to avoid an error
+Because T5 has been trained with the span-mask denoising objective,
+it can be used to predict the sentinel (masked-out) tokens during inference.
+The predicted tokens will then be placed between the sentinel tokens.
 
-task_prefix = "translate English to German: "
-sentences = ["The house is wonderful.", "I like to work in NYC."]  # use different length sentences to test batching
-inputs = tokenizer([task_prefix + sentence for sentence in sentences], return_tensors="pt", padding=True)
+```python
+>>> from transformers import T5Tokenizer, T5ForConditionalGeneration
 
-output_sequences = model.generate(
-    input_ids=inputs["input_ids"],
-    attention_mask=inputs["attention_mask"],
-    do_sample=False,  # disable sampling to test if batching affects output
-)
+>>> tokenizer = T5Tokenizer.from_pretrained("t5-small")
+>>> model = T5ForConditionalGeneration.from_pretrained("t5-small")
 
-print(tokenizer.batch_decode(output_sequences, skip_special_tokens=True))
+>>> input_ids = tokenizer("The <extra_id_0> walks in <extra_id_1> park", return_tensors="pt").input_ids
 
-# ['Das Haus ist wunderbar.', 'Ich arbeite gerne in NYC.']
+>>> sequence_ids = model.generate(input_ids)
+>>> sequences = tokenizer.batch_decode(sequence_ids)
+>>> sequences
+['<pad> <extra_id_0> park offers<extra_id_1> the<extra_id_2> park.
']
 ```
 
+
 <a id='scripts'></a>
 
 ## Performance
diff --git a/docs/source/en/model_doc/t5v1.1.mdx b/docs/source/en/model_doc/t5v1.1.mdx
index 512c5c59ce..b15188961d 100644
--- a/docs/source/en/model_doc/t5v1.1.mdx
+++ b/docs/source/en/model_doc/t5v1.1.mdx
@@ -20,9 +20,9 @@ repository by Colin Raffel et al. It's an improved version of the original T5 mo
 One can directly plug in the weights of T5v1.1 into a T5 model, like so:
 
 ```python
-from transformers import T5ForConditionalGeneration
+>>> from transformers import T5ForConditionalGeneration
 
-model = T5ForConditionalGeneration.from_pretrained("google/t5-v1_1-base")
+>>> model = T5ForConditionalGeneration.from_pretrained("google/t5-v1_1-base")
 ```
 
 T5 Version 1.1 includes the following improvements compared to the original T5 model:
diff --git a/src/transformers/generation_logits_process.py b/src/transformers/generation_logits_process.py
index 1b0762079d..7aa4004913 100644
--- a/src/transformers/generation_logits_process.py
+++ b/src/transformers/generation_logits_process.py
@@ -679,3 +679,16 @@ class ExponentialDecayLengthPenalty(LogitsProcessor):
                 self.regulation_factor, cur_len - self.regulation_start
             )
         return scores
+
+
+class LogitNormalization(LogitsProcessor, LogitsWarper):
+    r"""
+    [`LogitsWarper`] and [`LogitsProcessor`] for normalizing the scores using log-softmax. It's important to normalize
+    the scores during beam search, after applying the logits processors or warpers, since the search algorithm used in
+    this library doesn't do it (it only does it before, but they may need re-normalization) but it still supposes that
+    the scores are normalized when comparing the hypotheses.
+    """
+
+    def __call__(self, input_ids: torch.Tensor, scores: torch.Tensor) -> torch.Tensor:
+        scores = scores.log_softmax(dim=-1)
+        return scores
diff --git a/src/transformers/generation_tf_utils.py b/src/transformers/generation_tf_utils.py
index 83ba8a84cd..918f9169ab 100644
--- a/src/transformers/generation_tf_utils.py
+++ b/src/transformers/generation_tf_utils.py
@@ -2514,6 +2514,7 @@ class TFGenerationMixin:
 
         # 3. init tensors to use for "xla-compileable" generate function
         batch_size, num_beams, cur_len = input_ids.shape
+        input_ids_length = cur_len
 
         # per batch, beam-item holding current token in loop, pre-populated with `pad_token_id`
         sequences = tf.TensorArray(
@@ -2568,7 +2569,14 @@ class TFGenerationMixin:
         # 4. define "xla-compile-able" stop-condition and auto-regressive function
         # define stop-condition and auto-regressive function
         def beam_search_cond_fn(
-            cur_len, running_sequences, running_scores, sequences, scores, is_sent_finished, model_kwargs
+            cur_len,
+            running_sequences,
+            running_scores,
+            sequences,
+            scores,
+            is_sent_finished,
+            model_kwargs,
+            input_ids_length,
         ):
             """
             Beam Search termination condition function -- halts the generation loop if any of these conditions becomes
@@ -2597,7 +2605,7 @@ class TFGenerationMixin:
             scores,
             is_sent_finished,
             model_kwargs,
-            input_ids_length=1,
+            input_ids_length,
             intermediary_running_sequences=None,
         ):
             """
@@ -2754,9 +2762,11 @@ class TFGenerationMixin:
 
                 # if we don't cache past key values we need the whole input
                 if model_kwargs.get("past", None) is None:
-                    input_ids_length = cur_len + 1
+                    next_input_ids_length = cur_len + 1
                     # let's throw out `past` since we don't want `None` tensors
                     model_kwargs.pop("past", None)
+                else:
+                    next_input_ids_length = 1
 
             # 9. Prepare the `tf.TensorArray` for the next iteration
             next_sequences = sequences.unstack(tf.transpose(next_sequences_seq_last, perm=[2, 0, 1]))
@@ -2772,6 +2782,7 @@ class TFGenerationMixin:
                 next_scores,
                 next_is_sent_finished,
                 next_model_kwargs,
+                next_input_ids_length,
             )
 
         # 5. run generation
@@ -2780,8 +2791,7 @@ class TFGenerationMixin:
             beam_search_body_fn, intermediary_running_sequences=intermediary_running_sequences
         )
 
-        # 1st generation step has to be run before to initialize `past`
-        beam_search_body_fn_first_iter = partial(beam_search_body_fn, input_ids_length=cur_len)
+        # 1st generation step has to be run before to initialize `past` (if active)
         (
             cur_len,
             running_sequences,
@@ -2790,20 +2800,44 @@ class TFGenerationMixin:
             scores,
             is_sent_finished,
             model_kwargs,
-        ) = beam_search_body_fn_first_iter(
-            cur_len, running_sequences, running_scores, sequences, scores, is_sent_finished, model_kwargs
+            input_ids_length,
+        ) = beam_search_body_fn(
+            cur_len,
+            running_sequences,
+            running_scores,
+            sequences,
+            scores,
+            is_sent_finished,
+            model_kwargs,
+            input_ids_length,
         )
 
         # 2-to-n generation steps can then be run in autoregressive fashion (only in case 1st generation step does
         # NOT yield EOS token though)
         if beam_search_cond_fn(
-            cur_len, running_sequences, running_scores, sequences, scores, is_sent_finished, model_kwargs
+            cur_len,
+            running_sequences,
+            running_scores,
+            sequences,
+            scores,
+            is_sent_finished,
+            model_kwargs,
+            input_ids_length,
         ):
             maximum_iterations = max_length - cur_len
-            cur_len, running_sequences, running_scores, sequences, scores, is_sent_finished, _ = tf.while_loop(
+            cur_len, running_sequences, running_scores, sequences, scores, is_sent_finished, _, _ = tf.while_loop(
                 beam_search_cond_fn,
                 beam_search_body_fn,
-                (cur_len, running_sequences, running_scores, sequences, scores, is_sent_finished, model_kwargs),
+                (
+                    cur_len,
+                    running_sequences,
+                    running_scores,
+                    sequences,
+                    scores,
+                    is_sent_finished,
+                    model_kwargs,
+                    input_ids_length,
+                ),
                 maximum_iterations=maximum_iterations,
             )
 
diff --git a/src/transformers/generation_utils.py b/src/transformers/generation_utils.py
index b37e11af03..1bdcd06f0d 100644
--- a/src/transformers/generation_utils.py
+++ b/src/transformers/generation_utils.py
@@ -32,6 +32,7 @@ from .generation_logits_process import (
     ForcedEOSTokenLogitsProcessor,
     HammingDiversityLogitsProcessor,
     InfNanRemoveLogitsProcessor,
+    LogitNormalization,
     LogitsProcessorList,
     MinLengthLogitsProcessor,
     NoBadWordsLogitsProcessor,
@@ -636,6 +637,7 @@ class GenerationMixin:
         typical_p: Optional[float] = None,
         temperature: Optional[float] = None,
         num_beams: Optional[int] = None,
+        renormalize_logits: Optional[bool] = None,
     ) -> LogitsProcessorList:
         """
         This class returns a [`LogitsProcessorList`] list object that contains all relevant [`LogitsWarper`] instances
@@ -660,6 +662,9 @@ class GenerationMixin:
             warpers.append(TopPLogitsWarper(top_p=top_p, min_tokens_to_keep=(2 if num_beams > 1 else 1)))
         if typical_p is not None and typical_p < 1.0:
             warpers.append(TypicalLogitsWarper(mass=typical_p, min_tokens_to_keep=(2 if num_beams > 1 else 1)))
+        # `LogitNormalization` should always be the last logit processor, when present
+        if renormalize_logits is True:
+            warpers.append(LogitNormalization())
         return warpers
 
     def _get_logits_processor(
@@ -682,6 +687,7 @@ class GenerationMixin:
         remove_invalid_values: bool,
         exponential_decay_length_penalty: Tuple,
         logits_processor: Optional[LogitsProcessorList],
+        renormalize_logits: Optional[bool],
     ) -> LogitsProcessorList:
         """
         This class returns a [`LogitsProcessorList`] list object that contains all relevant [`LogitsProcessor`]
@@ -754,6 +760,9 @@ class GenerationMixin:
                 ExponentialDecayLengthPenalty(exponential_decay_length_penalty, eos_token_id, input_ids_seq_length)
             )
         processors = self._merge_criteria_processor_list(processors, logits_processor)
+        # `LogitNormalization` should always be the last logit processor, when present
+        if renormalize_logits is True:
+            processors.append(LogitNormalization())
         return processors
 
     def _get_stopping_criteria(
@@ -858,6 +867,7 @@ class GenerationMixin:
         diversity_penalty: Optional[float] = None,
         prefix_allowed_tokens_fn: Optional[Callable[[int, torch.Tensor], List[int]]] = None,
         logits_processor: Optional[LogitsProcessorList] = LogitsProcessorList(),
+        renormalize_logits: Optional[bool] = None,
         stopping_criteria: Optional[StoppingCriteriaList] = StoppingCriteriaList(),
         constraints: Optional[List[Constraint]] = None,
         output_attentions: Optional[bool] = None,
@@ -986,6 +996,10 @@ class GenerationMixin:
                  Custom logits processors that complement the default logits processors built from arguments and a
                  model's config. If a logit processor is passed that is already created with the arguments or a model's
                  config an error is thrown. This feature is intended for advanced users.
+            renormalize_logits: (`bool`, *optional*, defaults to `False`):
+                Whether to renormalize the logits after applying all the logits processors or warpers (including the
+                custom ones). It's highly recommended to set this flag to `True` as the search algorithms suppose the
+                score logits are normalized but some logit processors or warpers break the normalization.
             stopping_criteria (`StoppingCriteriaList`, *optional*):
                  Custom stopping criteria that complement the default stopping criteria built from arguments and a
                  model's config. If a stopping criteria is passed that is already created with the arguments or a
@@ -1241,6 +1255,7 @@ class GenerationMixin:
             remove_invalid_values=remove_invalid_values,
             exponential_decay_length_penalty=exponential_decay_length_penalty,
             logits_processor=logits_processor,
+            renormalize_logits=renormalize_logits,
         )
 
         # 8. prepare stopping criteria
@@ -1271,7 +1286,12 @@ class GenerationMixin:
         elif is_sample_gen_mode:
             # 10. prepare logits warper
             logits_warper = self._get_logits_warper(
-                top_k=top_k, top_p=top_p, typical_p=typical_p, temperature=temperature, num_beams=num_beams
+                top_k=top_k,
+                top_p=top_p,
+                typical_p=typical_p,
+                temperature=temperature,
+                num_beams=num_beams,
+                renormalize_logits=renormalize_logits,
             )
 
             # 11. expand input_ids with `num_return_sequences` additional sequences per batch
@@ -1333,7 +1353,12 @@ class GenerationMixin:
         elif is_beam_sample_gen_mode:
             # 10. prepare logits warper
             logits_warper = self._get_logits_warper(
-                top_k=top_k, top_p=top_p, typical_p=typical_p, temperature=temperature, num_beams=num_beams
+                top_k=top_k,
+                top_p=top_p,
+                typical_p=typical_p,
+                temperature=temperature,
+                num_beams=num_beams,
+                renormalize_logits=renormalize_logits,
             )
 
             if stopping_criteria.max_length is None:
diff --git a/src/transformers/modeling_utils.py b/src/transformers/modeling_utils.py
index 011134de7a..fb2ad9e952 100644
--- a/src/transformers/modeling_utils.py
+++ b/src/transformers/modeling_utils.py
@@ -1966,8 +1966,14 @@ class PreTrainedModel(nn.Module, ModuleUtilsMixin, GenerationMixin, PushToHubMix
                     "properly saved?"
                 )
 
-        if state_dict is not None:
-            # Whole checkpoint
+        def _find_mismatched_keys(
+            state_dict,
+            model_state_dict,
+            loaded_keys,
+            add_prefix_to_model,
+            remove_prefix_from_model,
+            ignore_mismatched_sizes,
+        ):
             mismatched_keys = []
             if ignore_mismatched_sizes:
                 for checkpoint_key in loaded_keys:
@@ -1988,6 +1994,18 @@ class PreTrainedModel(nn.Module, ModuleUtilsMixin, GenerationMixin, PushToHubMix
                         )
                         del state_dict[checkpoint_key]
 
+            return mismatched_keys
+
+        if state_dict is not None:
+            # Whole checkpoint
+            mismatched_keys = _find_mismatched_keys(
+                state_dict,
+                model_state_dict,
+                loaded_keys,
+                add_prefix_to_model,
+                remove_prefix_from_model,
+                ignore_mismatched_sizes,
+            )
             error_msgs = _load_state_dict_into_model(model_to_load, state_dict, start_prefix)
         else:
             # Sharded checkpoint
@@ -1996,30 +2014,19 @@ class PreTrainedModel(nn.Module, ModuleUtilsMixin, GenerationMixin, PushToHubMix
                 resolved_archive_file = [resolved_archive_file]
 
             error_msgs = []
+            mismatched_keys = []
             for shard_file in resolved_archive_file:
                 state_dict = load_state_dict(shard_file)
                 # Mistmatched keys contains tuples key/shape1/shape2 of weights in the checkpoint that have a shape not
                 # matching the weights in the model.
-                mismatched_keys = []
-                if ignore_mismatched_sizes:
-                    for checkpoint_key in loaded_keys:
-                        model_key = checkpoint_key
-                        if remove_prefix_from_model:
-                            # The model key starts with `prefix` but `checkpoint_key` doesn't so we add it.
-                            model_key = f"{prefix}.{checkpoint_key}"
-                        elif add_prefix_to_model:
-                            # The model key doesn't start with `prefix` but `checkpoint_key` does so we remove it.
-                            model_key = ".".join(checkpoint_key.split(".")[1:])
-
-                        if (
-                            model_key in model_state_dict
-                            and state_dict[checkpoint_key].shape != model_state_dict[model_key].shape
-                        ):
-                            mismatched_keys.append(
-                                (checkpoint_key, state_dict[checkpoint_key].shape, model_state_dict[model_key].shape)
-                            )
-                            del state_dict[checkpoint_key]
-
+                mismatched_keys += _find_mismatched_keys(
+                    state_dict,
+                    model_state_dict,
+                    loaded_keys,
+                    add_prefix_to_model,
+                    remove_prefix_from_model,
+                    ignore_mismatched_sizes,
+                )
                 error_msgs += _load_state_dict_into_model(model_to_load, state_dict, start_prefix)
 
         if len(error_msgs) > 0:
diff --git a/src/transformers/models/gptj/modeling_gptj.py b/src/transformers/models/gptj/modeling_gptj.py
index 18e3aa6303..d045603038 100755
--- a/src/transformers/models/gptj/modeling_gptj.py
+++ b/src/transformers/models/gptj/modeling_gptj.py
@@ -36,10 +36,19 @@ from .configuration_gptj import GPTJConfig
 
 logger = logging.get_logger(__name__)
 
-_CHECKPOINT_FOR_DOC = "EleutherAI/gpt-j-6B"
+_CHECKPOINT_FOR_DOC = "hf-internal-testing/tiny-random-gptj"
 _CONFIG_FOR_DOC = "GPTJConfig"
 _TOKENIZER_FOR_DOC = "GPT2Tokenizer"
 
+_CHECKPOINT_FOR_QA = "ydshieh/tiny-random-gptj-for-question-answering"
+_QA_EXPECTED_OUTPUT = "' was Jim Henson?Jim Henson was a n'"
+_QA_EXPECTED_LOSS = 3.13
+
+_CHECKPOINT_FOR_SEQUENCE_CLASSIFICATION = "ydshieh/tiny-random-gptj-for-sequence-classification"
+_SEQ_CLASS_EXPECTED_OUTPUT = "'LABEL_0'"
+_SEQ_CLASS_EXPECTED_LOSS = 0.76
+
+
 GPTJ_PRETRAINED_MODEL_ARCHIVE_LIST = [
     "EleutherAI/gpt-j-6B",
     # See all GPT-J models at https://huggingface.co/models?filter=gptj
@@ -892,9 +901,11 @@ class GPTJForSequenceClassification(GPTJPreTrainedModel):
     @add_start_docstrings_to_model_forward(GPTJ_INPUTS_DOCSTRING.format("batch_size, sequence_length"))
     @add_code_sample_docstrings(
         processor_class=_TOKENIZER_FOR_DOC,
-        checkpoint=_CHECKPOINT_FOR_DOC,
+        checkpoint=_CHECKPOINT_FOR_SEQUENCE_CLASSIFICATION,
         output_type=SequenceClassifierOutputWithPast,
         config_class=_CONFIG_FOR_DOC,
+        expected_output=_SEQ_CLASS_EXPECTED_OUTPUT,
+        expected_loss=_SEQ_CLASS_EXPECTED_LOSS,
     )
     def forward(
         self,
@@ -1017,9 +1028,11 @@ class GPTJForQuestionAnswering(GPTJPreTrainedModel):
     @add_start_docstrings_to_model_forward(GPTJ_INPUTS_DOCSTRING.format("batch_size, sequence_length"))
     @add_code_sample_docstrings(
         processor_class=_TOKENIZER_FOR_DOC,
-        checkpoint=_CHECKPOINT_FOR_DOC,
+        checkpoint=_CHECKPOINT_FOR_QA,
         output_type=QuestionAnsweringModelOutput,
         config_class=_CONFIG_FOR_DOC,
+        expected_output=_QA_EXPECTED_OUTPUT,
+        expected_loss=_QA_EXPECTED_LOSS,
     )
     def forward(
         self,
diff --git a/src/transformers/models/rag/modeling_rag.py b/src/transformers/models/rag/modeling_rag.py
index 593efa694e..642b13c580 100644
--- a/src/transformers/models/rag/modeling_rag.py
+++ b/src/transformers/models/rag/modeling_rag.py
@@ -1400,6 +1400,7 @@ class RagTokenForGeneration(RagPreTrainedModel):
         n_docs: Optional[int] = None,
         prefix_allowed_tokens_fn: Callable[[int, torch.Tensor], List[int]] = None,
         logits_processor: Optional[LogitsProcessorList] = LogitsProcessorList(),
+        renormalize_logits: Optional[bool] = None,
         stopping_criteria: Optional[StoppingCriteriaList] = StoppingCriteriaList(),
         forced_bos_token_id: Optional[int] = None,
         forced_eos_token_id: Optional[int] = None,
@@ -1624,6 +1625,7 @@ class RagTokenForGeneration(RagPreTrainedModel):
             remove_invalid_values=remove_invalid_values,
             exponential_decay_length_penalty=exponential_decay_length_penalty,
             logits_processor=logits_processor,
+            renormalize_logits=renormalize_logits,
         )
 
         if num_beams == 1:
diff --git a/src/transformers/tokenization_utils_base.py b/src/transformers/tokenization_utils_base.py
index c76197f544..899c5d3a02 100644
--- a/src/transformers/tokenization_utils_base.py
+++ b/src/transformers/tokenization_utils_base.py
@@ -1150,35 +1150,35 @@ class SpecialTokensMixin:
 
     @bos_token_id.setter
     def bos_token_id(self, value):
-        self._bos_token = self.convert_tokens_to_ids(value)
+        self._bos_token = self.convert_ids_to_tokens(value) if value is not None else None
 
     @eos_token_id.setter
     def eos_token_id(self, value):
-        self._eos_token = self.convert_tokens_to_ids(value)
+        self._eos_token = self.convert_ids_to_tokens(value) if value is not None else None
 
     @unk_token_id.setter
     def unk_token_id(self, value):
-        self._unk_token = self.convert_tokens_to_ids(value)
+        self._unk_token = self.convert_ids_to_tokens(value) if value is not None else None
 
     @sep_token_id.setter
     def sep_token_id(self, value):
-        self._sep_token = self.convert_tokens_to_ids(value)
+        self._sep_token = self.convert_ids_to_tokens(value) if value is not None else None
 
     @pad_token_id.setter
     def pad_token_id(self, value):
-        self._pad_token = self.convert_tokens_to_ids(value)
+        self._pad_token = self.convert_ids_to_tokens(value) if value is not None else None
 
     @cls_token_id.setter
     def cls_token_id(self, value):
-        self._cls_token = self.convert_tokens_to_ids(value)
+        self._cls_token = self.convert_ids_to_tokens(value) if value is not None else None
 
     @mask_token_id.setter
     def mask_token_id(self, value):
-        self._mask_token = self.convert_tokens_to_ids(value)
+        self._mask_token = self.convert_ids_to_tokens(value) if value is not None else None
 
     @additional_special_tokens_ids.setter
     def additional_special_tokens_ids(self, values):
-        self._additional_special_tokens = [self.convert_tokens_to_ids(value) for value in values]
+        self._additional_special_tokens = [self.convert_ids_to_tokens(value) for value in values]
 
     @property
     def special_tokens_map(self) -> Dict[str, Union[str, List[str]]]:


Test patch:
diff --git a/tests/byt5/test_tokenization_byt5.py b/tests/byt5/test_tokenization_byt5.py
index eb210530f0..7e4f97d374 100644
--- a/tests/byt5/test_tokenization_byt5.py
+++ b/tests/byt5/test_tokenization_byt5.py
@@ -332,3 +332,41 @@ class ByT5TokenizationTest(TokenizerTesterMixin, unittest.TestCase):
                 string = tokenizer.convert_tokens_to_string(tokens)
 
                 self.assertIsInstance(string, str)
+
+    # We need a different implementation of the test of the same name defined in TokenizerTesterMixin because this tokenizer
+    # doesn't have a vocab
+    def test_tokenizers_common_ids_setters(self):
+        tokenizers = self.get_tokenizers()
+        for tokenizer in tokenizers:
+            with self.subTest(f"{tokenizer.__class__.__name__}"):
+                attributes_list = [
+                    "bos_token",
+                    "eos_token",
+                    "unk_token",
+                    "sep_token",
+                    "pad_token",
+                    "cls_token",
+                    "mask_token",
+                ]
+
+                token_id_to_test_setters = 0
+                token_to_test_setters = tokenizer.convert_ids_to_tokens(
+                    token_id_to_test_setters, skip_special_tokens=False
+                )
+
+                for attr in attributes_list:
+                    setattr(tokenizer, attr + "_id", None)
+                    self.assertEqual(getattr(tokenizer, attr), None)
+                    self.assertEqual(getattr(tokenizer, attr + "_id"), None)
+
+                    setattr(tokenizer, attr + "_id", token_id_to_test_setters)
+                    self.assertEqual(getattr(tokenizer, attr), token_to_test_setters)
+                    self.assertEqual(getattr(tokenizer, attr + "_id"), token_id_to_test_setters)
+
+                setattr(tokenizer, "additional_special_tokens_ids", [])
+                self.assertListEqual(getattr(tokenizer, "additional_special_tokens"), [])
+                self.assertListEqual(getattr(tokenizer, "additional_special_tokens_ids"), [])
+
+                setattr(tokenizer, "additional_special_tokens_ids", [token_id_to_test_setters])
+                self.assertListEqual(getattr(tokenizer, "additional_special_tokens"), [token_to_test_setters])
+                self.assertListEqual(getattr(tokenizer, "additional_special_tokens_ids"), [token_id_to_test_setters])
diff --git a/tests/canine/test_tokenization_canine.py b/tests/canine/test_tokenization_canine.py
index d894237ff5..0a949e6d78 100644
--- a/tests/canine/test_tokenization_canine.py
+++ b/tests/canine/test_tokenization_canine.py
@@ -271,6 +271,43 @@ class CanineTokenizationTest(TokenizerTesterMixin, unittest.TestCase):
                 decoded = tokenizer.decode(encoded, spaces_between_special_tokens=self.space_between_special_tokens)
                 self.assertIn(decoded, [output, output.lower()])
 
+    # cannot use default `test_tokenizers_common_ids_setters` method because tokenizer has no vocab
+    def test_tokenizers_common_ids_setters(self):
+        tokenizers = self.get_tokenizers()
+        for tokenizer in tokenizers:
+            with self.subTest(f"{tokenizer.__class__.__name__}"):
+                attributes_list = [
+                    "bos_token",
+                    "eos_token",
+                    "unk_token",
+                    "sep_token",
+                    "pad_token",
+                    "cls_token",
+                    "mask_token",
+                ]
+
+        token_to_test_setters = "a"
+        token_id_to_test_setters = ord(token_to_test_setters)
+
+        for attr in attributes_list:
+            setattr(tokenizer, attr + "_id", None)
+            self.assertEqual(getattr(tokenizer, attr), None)
+            self.assertEqual(getattr(tokenizer, attr + "_id"), None)
+
+            setattr(tokenizer, attr + "_id", token_id_to_test_setters)
+            self.assertEqual(getattr(tokenizer, attr), token_to_test_setters)
+            self.assertEqual(getattr(tokenizer, attr + "_id"), token_id_to_test_setters)
+
+        setattr(tokenizer, "additional_special_tokens_ids", [])
+        self.assertListEqual(getattr(tokenizer, "additional_special_tokens"), [])
+        self.assertListEqual(getattr(tokenizer, "additional_special_tokens_ids"), [])
+
+        additional_special_token_id = 0xE006
+        additional_special_token = chr(additional_special_token_id)
+        setattr(tokenizer, "additional_special_tokens_ids", [additional_special_token_id])
+        self.assertListEqual(getattr(tokenizer, "additional_special_tokens"), [additional_special_token])
+        self.assertListEqual(getattr(tokenizer, "additional_special_tokens_ids"), [additional_special_token_id])
+
     # tokenizer has a fixed vocab_size (namely all possible unicode code points)
     def test_add_tokens_tokenizer(self):
         pass
diff --git a/tests/generation/test_generation_logits_process.py b/tests/generation/test_generation_logits_process.py
index 8489efd44d..7a515d3e92 100644
--- a/tests/generation/test_generation_logits_process.py
+++ b/tests/generation/test_generation_logits_process.py
@@ -33,6 +33,7 @@ if is_torch_available():
         ForcedEOSTokenLogitsProcessor,
         HammingDiversityLogitsProcessor,
         InfNanRemoveLogitsProcessor,
+        LogitNormalization,
         LogitsProcessorList,
         MinLengthLogitsProcessor,
         NoBadWordsLogitsProcessor,
@@ -537,3 +538,18 @@ class LogitsProcessorTest(unittest.TestCase):
                 scores_after_start[penalty_start + 1 :, eos_token_id], scores[penalty_start + 1 :, eos_token_id]
             ).all()
         )
+
+    def test_normalization(self):
+        input_ids = None
+
+        scores = torch.tensor(
+            [[-23.18, -29.96, -43.54, 47.77], [-33.58, -26.87, -32.96, 22.51]], device=torch_device, dtype=torch.float
+        )
+
+        logit_normalization = LogitNormalization()
+        normalized_scores = logit_normalization(input_ids, scores).exp()
+
+        ones = torch.ones(scores.shape[0], device=torch_device, dtype=torch.float)
+        self.assertTrue(normalized_scores.sum(dim=-1).allclose(ones))
+
+        self.assertTrue(normalized_scores.allclose(scores.softmax(dim=-1)))
diff --git a/tests/test_tokenization_common.py b/tests/test_tokenization_common.py
index 2d26d76b9a..fe16e5e1cd 100644
--- a/tests/test_tokenization_common.py
+++ b/tests/test_tokenization_common.py
@@ -540,6 +540,43 @@ class TokenizerTesterMixin:
                 for attr in attributes_list:
                     self.assertTrue(hasattr(tokenizer, attr))
 
+    def test_tokenizers_common_ids_setters(self):
+        tokenizers = self.get_tokenizers()
+        for tokenizer in tokenizers:
+            with self.subTest(f"{tokenizer.__class__.__name__}"):
+                attributes_list = [
+                    "bos_token",
+                    "eos_token",
+                    "unk_token",
+                    "sep_token",
+                    "pad_token",
+                    "cls_token",
+                    "mask_token",
+                ]
+
+                vocab = tokenizer.get_vocab()
+                token_id_to_test_setters = next(iter(vocab.values()))
+                token_to_test_setters = tokenizer.convert_ids_to_tokens(
+                    token_id_to_test_setters, skip_special_tokens=False
+                )
+
+                for attr in attributes_list:
+                    setattr(tokenizer, attr + "_id", None)
+                    self.assertEqual(getattr(tokenizer, attr), None)
+                    self.assertEqual(getattr(tokenizer, attr + "_id"), None)
+
+                    setattr(tokenizer, attr + "_id", token_id_to_test_setters)
+                    self.assertEqual(getattr(tokenizer, attr), token_to_test_setters)
+                    self.assertEqual(getattr(tokenizer, attr + "_id"), token_id_to_test_setters)
+
+                setattr(tokenizer, "additional_special_tokens_ids", [])
+                self.assertListEqual(getattr(tokenizer, "additional_special_tokens"), [])
+                self.assertListEqual(getattr(tokenizer, "additional_special_tokens_ids"), [])
+
+                setattr(tokenizer, "additional_special_tokens_ids", [token_id_to_test_setters])
+                self.assertListEqual(getattr(tokenizer, "additional_special_tokens"), [token_to_test_setters])
+                self.assertListEqual(getattr(tokenizer, "additional_special_tokens_ids"), [token_id_to_test_setters])
+
     def test_save_and_load_tokenizer(self):
         # safety check on max_len default value so we are sure the test works
         tokenizers = self.get_tokenizers()
diff --git a/utils/documentation_tests.txt b/utils/documentation_tests.txt
index eee14c3374..8e96b81e6d 100644
--- a/utils/documentation_tests.txt
+++ b/utils/documentation_tests.txt
@@ -1,6 +1,9 @@
 docs/source/en/quicktour.mdx
 docs/source/en/task_summary.mdx
 docs/source/en/model_doc/speech_to_text.mdx
+docs/source/en/model_doc/t5.mdx
+docs/source/en/model_doc/t5v1_1.mdx
+docs/source/en/model_doc/byt5.mdx
 docs/source/en/model_doc/tapex.mdx
 src/transformers/generation_utils.py
 src/transformers/models/bart/modeling_bart.py
@@ -16,6 +19,7 @@ src/transformers/models/deit/modeling_deit.py
 src/transformers/models/dpt/modeling_dpt.py
 src/transformers/models/glpn/modeling_glpn.py
 src/transformers/models/gpt2/modeling_gpt2.py
+src/transformers/models/gptj/modeling_gptj.py
 src/transformers/models/hubert/modeling_hubert.py
 src/transformers/models/marian/modeling_marian.py
 src/transformers/models/mbart/modeling_mbart.py