Issue 1:
This is not urgent. There is a ton of deprecation warnings across many modules with pytorch-1.7+ and a few with python-3.8:

(I hard-wrapped the lines to avoid the need to scroll, but it makes somewhat harder to see the warnings):

src/transformers/modeling_deberta.py:18 src/transformers/modeling_deberta.py:18
src/transformers/modeling_deberta.py:18 src/transformers/modeling_deberta.py:18
src/transformers/modeling_deberta.py:18
src/transformers/modeling_deberta.py:18:
DeprecationWarning: Using or importing the ABCs from 'collections' instead of
from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop
working from collections import Sequence

tests/test_logging.py::HfArgumentParserTest::test_integration
tests/test_logging.py:40:
DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
logger.warn(msg)

tests/test_logging.py::HfArgumentParserTest::test_integration
tests/test_logging.py:48:
DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
logger.warn(msg)

tests/test_benchmark.py::BenchmarkTest::test_inference_torchscript
tests/test_modeling_gpt2.py::GPT2ModelTest::test_torchscript
tests/test_modeling_gpt2.py::GPT2ModelTest::test_torchscript_output_attentions
tests/test_modeling_gpt2.py::GPT2ModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_gpt2.py:164:
TracerWarning: Converting a tensor to a Python float might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! w = w / (float(v.size(-1)) ** 0.5)

tests/test_benchmark.py::BenchmarkTest::test_inference_torchscript
tests/test_modeling_gpt2.py::GPT2ModelTest::test_torchscript
tests/test_modeling_gpt2.py::GPT2ModelTest::test_torchscript_output_attentions
tests/test_modeling_gpt2.py::GPT2ModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_gpt2.py:169:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! mask = self.bias[:, :, ns - nd : ns, :ns]

tests/test_modeling_auto.py::AutoModelTest::test_from_identifier_from_model_type
tests/test_modeling_auto.py::AutoModelTest::test_from_pretrained_identifier
src/transformers/modeling_auto.py:821:
FutureWarning: The class `AutoModelWithLMHead` is deprecated and will be removed
in a future version. Please use `AutoModelForCausalLM` for causal language
models, `AutoModelForMaskedLM` for masked language models and
`AutoModelForSeq2SeqLM` for encoder-decoder models. warnings.warn(

tests/test_benchmark_tf.py::TFBenchmarkTest::test_train_no_configs
tests/test_benchmark_tf.py::TFBenchmarkTest::test_train_with_configs
/home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:432:
UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape.
This may consume a large amount of memory. warnings.warn(

tests/test_modeling_albert.py::AlbertModelTest::test_torchscript
tests/test_modeling_albert.py::AlbertModelTest::test_torchscript_output_attentions
tests/test_modeling_albert.py::AlbertModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_albert.py:229:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! position_ids = self.position_ids[:, :seq_length]

tests/test_modeling_albert.py: 3 warnings tests/test_modeling_bert.py: 3
warnings tests/test_modeling_bert_generation.py: 3 warnings
tests/test_modeling_distilbert.py: 2 warnings tests/test_modeling_dpr.py: 3
warnings tests/test_modeling_flaubert.py: 3 warnings
tests/test_modeling_electra.py: 3 warnings tests/test_modeling_layoutlm.py: 3
warnings tests/test_modeling_roberta.py: 3 warnings tests/test_modeling_xlm.py:
3 warnings tests/test_modeling_xlnet.py: 3 warnings
src/transformers/modeling_utils.py:1670:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! input_tensor.shape == tensor_shape for input_tensor
in input_tensors

tests/test_modeling_bert_generation.py: 32 warnings
src/transformers/modeling_bert_generation.py:417:
DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
logger.warn("If you want to use `BertGenerationDecoder` as a standalone, add
`is_decoder=True.`")

tests/test_modeling_bert.py::BertModelTest::test_torchscript
tests/test_modeling_bert.py::BertModelTest::test_torchscript_output_attentions
tests/test_modeling_bert.py::BertModelTest::test_torchscript_output_hidden_state
tests/test_modeling_dpr.py::DPRModelTest::test_torchscript
tests/test_modeling_dpr.py::DPRModelTest::test_torchscript_output_attentions
tests/test_modeling_dpr.py::DPRModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_bert.py:191:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! position_ids = self.position_ids[:, :seq_length]

tests/test_modeling_bart.py::BARTModelTest::test_torchscript
tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_attentions
tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_bart.py:175:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! if decoder_padding_mask is not None and
decoder_padding_mask.shape[1] > 1:

tests/test_modeling_bart.py: 3 warnings tests/test_modeling_flaubert.py: 3
warnings tests/test_modeling_fsmt.py: 3 warnings tests/test_modeling_roberta.py:
3 warnings tests/test_modeling_xlm.py: 3 warnings
/home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/nn/functional.py:1836:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert padding_idx < weight.size(0), 'Padding_idx
must be within num_embeddings'

tests/test_modeling_bart.py::BARTModelTest::test_torchscript
tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_attentions
tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_bart.py:720:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert key_padding_mask is None or
key_padding_mask.shape == (bsz, src_len)

tests/test_modeling_bart.py::BARTModelTest::test_torchscript
tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_attentions
tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_bart.py:722:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert attn_weights.size() == (bsz * self.num_heads,
tgt_len, src_len)

tests/test_modeling_bart.py::BARTModelTest::test_torchscript
tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_attentions
tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_bart.py:740:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert attn_output.size() == (bsz * self.num_heads,
tgt_len, self.head_dim)

tests/test_modeling_bart.py::BARTModelTest::test_torchscript
tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_attentions
tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_bart.py:287:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! if torch.isinf(x).any() or torch.isnan(x).any():

tests/test_modeling_bart.py::BARTModelTest::test_torchscript
tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_attentions
tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_bart.py:1190:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! if len(torch.unique(eos_mask.sum(1))) > 1:

tests/test_modeling_common.py::UtilsFunctionsTest::test_top_k_top_p_filtering
tests/test_modeling_common.py:1196:
UserWarning: This overload of nonzero is deprecated: nonzero() Consider using
one of the following signatures instead: nonzero(*, bool as_tuple) (Triggered
internally at /pytorch/torch/csrc/utils/python_arg_parser.cpp:882.)
non_inf_idx = (output != -float("inf")).nonzero().to(device=torch_device)

tests/test_modeling_bert_generation.py::BertGenerationEncoderTest::test_torchscript
tests/test_modeling_bert_generation.py::BertGenerationEncoderTest::test_torchscript_output_attentions
tests/test_modeling_bert_generation.py::BertGenerationEncoderTest::test_torchscript_output_hidden_state
src/transformers/modeling_bert_generation.py:156:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! position_ids = self.position_ids[:, :seq_length]

tests/test_modeling_flaubert.py: 14 warnings tests/test_modeling_xlm.py: 14
warnings
src/transformers/modeling_xlm.py:1220:
FutureWarning: The `lengths` parameter cannot be used with the XLM multiple
choice models. Please use the attention mask instead. warnings.warn(

tests/test_modeling_flaubert.py::FlaubertModelTest::test_flaubert_lm_head
tests/test_modeling_flaubert.py::FlaubertModelTest::test_model_outputs_equivalence
tests/test_modeling_xlm.py::XLMModelTest::test_model_outputs_equivalence
tests/test_modeling_xlm.py::XLMModelTest::test_xlm_lm_head
/home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/nn/_reduction.py:14:
UserWarning: reduction='elementwise_mean' is deprecated, please use
reduction='mean' instead. warnings.warn("reduction='elementwise_mean' is
deprecated, please use reduction='mean' instead.")

tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript
tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_attentions
tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_flaubert.py:188:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert lengths.size(0) == bs

tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript
tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_attentions
tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_flaubert.py:189:
TracerWarning: Converting a tensor to a Python number might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert lengths.max().item() <= slen

tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript
tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_attentions
tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_flaubert.py:189:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert lengths.max().item() <= slen

tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript
tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_attentions
tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_hidden_state
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_attentions
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_xlm.py:95:
TracerWarning: Converting a tensor to a Python number might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert lengths.max().item() <= slen

tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript
tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_attentions
tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_hidden_state
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_attentions
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_xlm.py:95:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert lengths.max().item() <= slen

tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript
tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_attentions
tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_hidden_state
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_attentions
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_xlm.py:106:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert mask.size() == (bs, slen)

tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript
tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_attentions
tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_fsmt.py:1224:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! if max_pos > self.weight.size(0):

tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript
tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_attentions
tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_fsmt.py:763:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert embed_dim == self.embed_dim

tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript
tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_attentions
tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_fsmt.py:764:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert list(query.size()) == [tgt_len, bsz,
embed_dim]

tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript
tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_attentions
tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_fsmt.py:805:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert attn_weights.size() == (bsz * self.num_heads,
tgt_len, src_len)

tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript
tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_attentions
tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_fsmt.py:814:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert key_padding_mask is None or
key_padding_mask.size()[:2] == (

tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript
tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_attentions
tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_fsmt.py:833:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert attn_output.size() == (bsz * self.num_heads,
tgt_len, self.head_dim)

tests/test_modeling_gpt2.py::GPT2ModelTest::test_gpt2_model_att_mask_past
tests/test_modeling_gpt2.py::GPT2ModelTest::test_gpt2_model_past
tests/test_modeling_gpt2.py::GPT2ModelTest::test_gpt2_model_past_large_inputs
src/transformers/modeling_gpt2.py:530:
FutureWarning: The `past` argument is deprecated and will be removed in a future
version, use `past_key_values` instead. warnings.warn(

tests/test_modeling_electra.py::ElectraModelTest::test_torchscript
tests/test_modeling_electra.py::ElectraModelTest::test_torchscript_output_attentions
tests/test_modeling_electra.py::ElectraModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_electra.py:180:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! position_ids = self.position_ids[:, :seq_length]

tests/test_modeling_layoutlm.py::LayoutLMModelTest::test_torchscript
tests/test_modeling_layoutlm.py::LayoutLMModelTest::test_torchscript_output_attentions
tests/test_modeling_layoutlm.py::LayoutLMModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_layoutlm.py:87:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! position_ids = self.position_ids[:, :seq_length]

tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript
tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_attentions
tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_hidden_state
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript_output_attentions
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript_output_hidden_state
/home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/tensor.py:547:
TracerWarning: torch.tensor results are registered as constants in the trace.
You can safely ignore this warning if you use this function to create tensors
out of constant variables that would be the same every time you call this
function. In any other case, this might cause the trace to be incorrect. return
torch.tensor(other, dtype=dtype, device=self.device) ** self

tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript
tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_attentions
tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_hidden_state
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript_output_attentions
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_funnel.py:314:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! num_remove = shift * len(pooled_pos)

tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript
tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_attentions
tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_hidden_state
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript_output_attentions
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_funnel.py:638:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! pooling_flag = pooling_flag and block_index > 0

tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript
tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_attentions
tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_hidden_state
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript_output_attentions
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_funnel.py:481:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! shift = 2 if q_head.shape[1] != context_len else 1

tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript
tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_attentions
tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_hidden_state
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript_output_attentions
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_funnel.py:431:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! positional_attn = positional_attn[..., :context_len]

tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript
tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_attentions
tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_funnel.py:678:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! output = output[:, : target_len - 1]

tests/test_modeling_gpt2.py::GPT2ModelTest::test_torchscript
tests/test_modeling_gpt2.py::GPT2ModelTest::test_torchscript_output_attentions
tests/test_modeling_gpt2.py::GPT2ModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_gpt2.py:1058:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! pooled_logits = logits[range(batch_size),
sequence_lengths]

tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript
tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript_output_attentions
tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_openai.py:467:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! position_ids = self.position_ids[None, :
input_shape[-1]]

tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript
tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript_output_attentions
tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_openai.py:180:
TracerWarning: Converting a tensor to a Python float might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! w = w / math.sqrt(v.size(-1))

tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript
tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript_output_attentions
tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_openai.py:183:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! b = self.bias[:, :, : w.size(-2), : w.size(-1)]

tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript
tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript_output_attentions
tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_openai.py:823:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! pooled_logits = logits[range(batch_size),
sequence_lengths]

tests/test_modeling_rag.py: 12 warnings tests/test_retrieval_rag.py: 1 warning
src/transformers/tokenization_utils_base.py:613:
UserWarning: To copy construct from a tensor, it is recommended to use
sourceTensor.clone().detach() or
sourceTensor.clone().detach().requires_grad_(True), rather than
torch.tensor(sourceTensor). tensor = as_tensor(value)

tests/test_modeling_reformer.py: 58 warnings tests/test_modeling_transfo_xl.py:
18 warnings
/home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/nn/modules/container.py:434:
UserWarning: Setting attributes on ParameterList is not supported.
warnings.warn("Setting attributes on ParameterList is not supported.")

tests/test_modeling_mobilebert.py::MobileBertModelTest::test_torchscript
tests/test_modeling_mobilebert.py::MobileBertModelTest::test_torchscript_output_attentions
tests/test_modeling_mobilebert.py::MobileBertModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_mobilebert.py:192:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! position_ids = self.position_ids[:, :seq_length]

tests/test_modeling_mobilebert.py::MobileBertModelTest::test_torchscript
tests/test_modeling_mobilebert.py::MobileBertModelTest::test_torchscript_output_attentions
tests/test_modeling_mobilebert.py::MobileBertModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_mobilebert.py:534:
TracerWarning: torch.tensor results are registered as constants in the trace.
You can safely ignore this warning if you use this function to create tensors
out of constant variables that would be the same every time you call this
function. In any other case, this might cause the trace to be incorrect.
torch.tensor(1000),

tests/test_modeling_reformer.py::ReformerLSHAttnModelTest::test_reformer_cached_inference
src/transformers/modeling_reformer.py:899:
UserWarning: This overload of nonzero is deprecated: nonzero() Consider using
one of the following signatures instead: nonzero(*, bool as_tuple) (Triggered
internally at /pytorch/torch/csrc/utils/python_arg_parser.cpp:882.)
relevant_bucket_idx = (bucket_idx == (bucket_idx.shape[-1] - 1)).nonzero()

tests/test_modeling_t5.py::T5ModelTest::test_export_to_onnx
tests/test_modeling_t5.py::T5ModelTest::test_torchscript_output_attentions
tests/test_modeling_t5.py::T5ModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_utils.py:244:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! if causal_mask.shape[1] < attention_mask.shape[1]:

tests/test_modeling_t5.py: 95 warnings
/home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/onnx/utils.py:760:
DeprecationWarning: an integer is required (got type
torch._C._onnx.TensorProtoDataType). Implicit conversion to integers using
__int__ is deprecated, and may be removed in a future version of Python.
return getattr(node, kind + "_")(name, value)

tests/test_modeling_t5.py::T5ModelTest::test_export_to_onnx
tests/test_modeling_t5.py::T5ModelTest::test_export_to_onnx
tests/test_modeling_t5.py::T5ModelTest::test_export_to_onnx
/home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/onnx/symbolic_opset9.py:1638:
DeprecationWarning: an integer is required (got type float). Implicit conversion
to integers using __int__ is deprecated, and may be removed in a future version
of Python. value_t=torch.tensor([fill_value],
dtype=sym_help.scalar_type_to_pytorch_type[dtype]))

tests/test_modeling_tf_auto.py::TFAutoModelTest::test_from_identifier_from_model_type
tests/test_modeling_tf_auto.py::TFAutoModelTest::test_from_pretrained_identifier
src/transformers/modeling_tf_auto.py:697:
FutureWarning: The class `TFAutoModelWithLMHead` is deprecated and will be
removed in a future version. Please use `TFAutoModelForCausalLM` for causal
language models, `TFAutoModelForMaskedLM` for masked language models and
`TFAutoModelForSeq2SeqLM` for encoder-decoder models. warnings.warn(

tests/test_modeling_squeezebert.py::SqueezeBertModelTest::test_torchscript_output_attentions
tests/test_modeling_squeezebert.py::SqueezeBertModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_squeezebert.py:78:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! position_ids = self.position_ids[:, :seq_length]

tests/test_modeling_tf_flaubert.py: 9 warnings tests/test_modeling_tf_xlm.py: 9
warnings
src/transformers/modeling_tf_xlm.py:994:
FutureWarning: The `lengths` parameter cannot be used with the XLM multiple
choice models. Please use the attention mask instead. warnings.warn(

tests/test_modeling_tf_flaubert.py::TFFlaubertModelTest::test_graph_mode
tests/test_modeling_tf_xlm.py::TFXLMModelTest::test_graph_mode
/home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/tensorflow/python/autograph/impl/api.py:493:
FutureWarning: The `lengths` parameter cannot be used with the XLM multiple
choice models. Please use the attention mask instead. return
py_builtins.overload_of(f)(*args)

tests/test_modeling_tf_xlnet.py::TFXLNetModelTest::test_compile_tf_model
tests/test_modeling_tf_xlnet.py::TFXLNetModelTest::test_config
tests/test_modeling_tf_xlnet.py::TFXLNetModelTest::test_keras_save_load
tests/test_modeling_xlnet.py::XLNetModelTest::test_config
tests/test_modeling_xlnet.py::XLNetModelTest::test_correct_missing_keys
tests/test_modeling_tf_xlnet.py::TFXLNetModelTest::test_save_load
tests/test_modeling_tf_xlnet.py::TFXLNetModelTest::test_train_pipeline_custom_model
tests/test_modeling_xlnet.py::XLNetModelTest::test_save_load
src/transformers/configuration_xlnet.py:205:
FutureWarning: This config doesn't use attention memories, a core feature of
XLNet. Consider setting `mem_len` to a non-zero value, for example `xlnet =
XLNetLMHeadModel.from_pretrained('xlnet-base-cased'', mem_len=1024)`, for
accurate training performance as well as an order of magnitude faster inference.
Starting from version 3.5.0, the default parameter will be 1024, following the
implementation in https://arxiv.org/abs/1906.08237 warnings.warn(

tests/test_modeling_xlm.py::XLMModelTest::test_torchscript
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_attentions
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_xlm.py:531:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert lengths.size(0) == bs

tests/test_modeling_xlm.py::XLMModelTest::test_torchscript
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_attentions
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_xlm.py:532:
TracerWarning: Converting a tensor to a Python number might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert lengths.max().item() <= slen

tests/test_modeling_xlm.py::XLMModelTest::test_torchscript
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_attentions
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_xlm.py:532:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert lengths.max().item() <= slen

tests/test_modeling_xlm.py::XLMModelTest::test_torchscript
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_attentions
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_xlm.py:546:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! position_ids = self.position_ids[:, :slen]

tests/test_optimization.py::OptimizationTest::test_adafactor
src/transformers/optimization.py:512:
UserWarning: This overload of add_ is deprecated: add_(Number alpha, Tensor
other) Consider using one of the following signatures instead: add_(Tensor
other, *, Number alpha) (Triggered internally at
/pytorch/torch/csrc/utils/python_arg_parser.cpp:882.)
exp_avg_sq.mul_(beta2t).add_(1.0 - beta2t, update)

tests/test_optimization.py::ScheduleInitTest::test_schedulers
/home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/optim/lr_scheduler.py:247:
UserWarning: To get the last learning rate computed by the scheduler, please
use `get_last_lr()`. warnings.warn("To get the last learning rate computed by
the scheduler, "

tests/test_optimization.py::ScheduleInitTest::test_schedulers
/home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/optim/lr_scheduler.py:131:
UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`.
In PyTorch 1.1.0 and later, you should call them in the opposite order:
`optimizer.step()` before `lr_scheduler.step()`. Failure to do this will
result in PyTorch skipping the first value of the learning rate schedule. See
more details at
https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate
warnings.warn("Detected call of `lr_scheduler.step()` before
`optimizer.step()`. "

tests/test_optimization.py::ScheduleInitTest::test_schedulers
/home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/optim/lr_scheduler.py:216:
UserWarning: Please also save or load the state of the optimizer when saving
or loading the scheduler. warnings.warn(SAVE_STATE_WARNING, UserWarning)

tests/test_optimization.py::ScheduleInitTest::test_schedulers
/home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/optim/lr_scheduler.py:234:
UserWarning: Please also save or load the state of the optimizer when saving
or loading the scheduler. warnings.warn(SAVE_STATE_WARNING, UserWarning)

tests/test_tokenization_auto.py::AutoTokenizerTest::test_tokenizer_identifier_with_correct_config
tests/test_tokenization_mbart.py::MBartEnroIntegrationTest::test_batch_fairseq_parity
tests/test_tokenization_t5.py::T5TokenizationTest::test_empty_target_text
tests/test_tokenization_t5.py::T5TokenizationTest::test_eos_in_input
tests/test_tokenization_t5.py::T5TokenizationTest::test_max_target_length
tests/test_tokenization_t5.py::T5TokenizationTest::test_outputs_not_longer_than_maxlen
tests/test_tokenization_t5.py::T5TokenizationTest::test_prepare_seq2seq_batch
src/transformers/tokenization_utils_base.py:1421:
FutureWarning: The `max_len` attribute has been deprecated and will be removed
in a future version, use `model_max_length` instead. warnings.warn(

tests/test_tokenization_albert.py: 2 warnings tests/test_tokenization_bart.py: 2
warnings tests/test_tokenization_bert.py: 2 warnings
tests/test_tokenization_bert_generation.py: 1 warning
tests/test_tokenization_bertweet.py: 1 warning
tests/test_tokenization_blenderbot.py: 1 warning
tests/test_tokenization_ctrl.py: 1 warning tests/test_tokenization_camembert.py:
2 warnings tests/test_tokenization_distilbert.py: 4 warnings
tests/test_tokenization_dpr.py: 8 warnings tests/test_tokenization_fsmt.py: 1
warning tests/test_tokenization_funnel.py: 2 warnings
tests/test_tokenization_herbert.py: 2 warnings tests/test_tokenization_gpt2.py:
1 warning tests/test_tokenization_layoutlm.py: 2 warnings
tests/test_tokenization_marian.py: 1 warning tests/test_tokenization_lxmert.py:
2 warnings tests/test_tokenization_mbart.py: 2 warnings
tests/test_tokenization_pegasus.py: 2 warnings
tests/test_tokenization_openai.py: 1 warning tests/test_tokenization_phobert.py:
1 warning tests/test_tokenization_deberta.py: 1 warning
tests/test_tokenization_prophetnet.py: 1 warning
tests/test_tokenization_reformer.py: 1 warning
tests/test_tokenization_squeezebert.py: 4 warnings
tests/test_tokenization_t5.py: 2 warnings tests/test_tokenization_roberta.py: 2
warnings tests/test_tokenization_transfo_xl.py: 1 warning
tests/test_tokenization_xlm.py: 1 warning
tests/test_tokenization_xlm_prophetnet.py: 1 warning
tests/test_tokenization_xlnet.py: 2 warnings
tests/test_tokenization_xlm_roberta.py: 2 warnings
src/transformers/tokenization_utils_base.py:2025:
FutureWarning: The `pad_to_max_length` argument is deprecated and will be
removed in a future version, use `padding=True` or `padding='longest'` to pad to
the longest sequence in the batch, or use `padding='max_length'` to pad to a max
length. In this case, you can give a specific length with `max_length` (e.g.
`max_length=45`) or leave max_length to None to pad to the maximal input size of
the model (e.g. 512 for Bert). warnings.warn(

tests/test_tokenization_t5.py::T5TokenizationTest::test_eos_in_input
tests/test_tokenization_t5.py::T5TokenizationTest::test_eos_treatment
src/transformers/tokenization_t5.py:183:
UserWarning: This sequence already has 
. In future versions this behavior
may lead to duplicated eos tokens being added. warnings.warn(

tests/test_trainer.py: 44 warnings
/home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/nn/parallel/_functions.py:64:
UserWarning: Was asked to gather along dimension 0, but all input tensors were
scalars; will instead unsqueeze and return a vector. warnings.warn('Was asked
to gather along dimension 0, but all '

tests/test_trainer.py::TrainerIntegrationTest::test_can_resume_training
tests/test_trainer_callback.py::TrainerCallbackTest::test_event_flow
/home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/cuda/nccl.py:48:
DeprecationWarning: Using or importing the ABCs from 'collections' instead of
from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop
working if not isinstance(inputs, collections.Container) or isinstance(inputs,
torch.Tensor):

-- Docs: https://docs.pytest.org/en/stable/warnings.html

@LysandreJik

Issue 2:
System Info

- `transformers` version: 4.20.1
- Platform: Windows-10-10.0.19044-SP0
- Python version: 3.8.12
- Huggingface_hub version: 0.4.0
- PyTorch version (GPU?): 1.11.0 (True)
- Tensorflow version (GPU?): 2.8.0 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

Who can help?

@patrickvonplaten

Information

The official example scripts

My own modified scripts

Tasks

An officially supported task in the examples folder (such as GLUE/SQuAD, ...)

My own task or dataset (give details below)

Reproduction

duplication case: the outputs should not contain the same sequence.

>>> from transformers import GPT2LMHeadModel, GPT2Tokenizer

>>> model = GPT2LMHeadModel.from_pretrained("gpt2")
>>> tokenizer = GPT2Tokenizer.from_pretrained("gpt2")

>>> force_word = "are"

>>> force_words_ids = [
>>> tokenizer([force_word], add_prefix_space=True, add_special_tokens=False).input_ids,
>>> ]

>>> starting_text = ["The soldiers"]

>>> input_ids = tokenizer(starting_text, return_tensors="pt").input_ids

>>> outputs = model.generate(
>>> input_ids,
>>> max_new_tokens=3,
>>> force_words_ids=force_words_ids,
>>> num_beams=10,
>>> num_return_sequences=10,
>>> no_repeat_ngram_size=1,
>>> remove_invalid_values=True,
>>> )

>>> outputs = outputs[:, input_ids.shape[-1]:]
>>> print("Output:\n" + 100 * '-')
>>> for s in tokenizer.batch_decode(outputs, skip_special_tokens=True):
>>> print(s)
>>> import collections
>>> print(collections.Counter(map(tuple, outputs.tolist())).most_common(1))
Output:
----------------------------------------------------------------------------------------------------
, who are
, who were
who are in
, who had
, who are
who are fighting
who are still
who are killed
who are not
who are on
[((11, 508, 389), 2)]

weird case: the output looks weird, repeat progressing constraints with unreasonable tokens

>>> from transformers import GPT2LMHeadModel, GPT2Tokenizer

>>> model = GPT2LMHeadModel.from_pretrained("gpt2")
>>> tokenizer = GPT2Tokenizer.from_pretrained("gpt2")

>>> force_word = "were not allowed"

>>> force_words_ids = [
>>> tokenizer([force_word], add_prefix_space=True, add_special_tokens=False).input_ids,
>>> ]

>>> starting_text = ["The soldiers"]

>>> input_ids = tokenizer(starting_text, return_tensors="pt").input_ids

>>> outputs = model.generate(
>>> input_ids,
>>> force_words_ids=force_words_ids,
>>> num_beams=10,
>>> num_return_sequences=10,
>>> no_repeat_ngram_size=1,
>>> remove_invalid_values=True,
>>> )

>>> print("Output:\n" + 100 * '-')
>>> for s in tokenizer.batch_decode(outputs, skip_special_tokens=True):
>>> print(s)
Output:
----------------------------------------------------------------------------------------------------
The soldiers, who were wearing were not in were not allowed to leave the barracks. The commander of
The soldiers, who were wearing were not in were not allowed to leave the barracks. The military said
The soldiers, who were wearing were not in were not allowed to enter the building. The military said
The soldiers, who were wearing were not in were not allowed to enter the building. The police said
The soldiers, who were wearing were not in were not allowed to leave the barracks. The commander said
The soldiers, who were wearing were not in were not allowed to enter the building. The police had
The soldiers, who were wearing were not in were not allowed to leave the barracks. The army said
The soldiers, who were wearing were not in were not allowed to enter the building. The police then
The soldiers, who were wearing were not in were not allowed to leave the barracks. The commander told
The soldiers, who were wearing were not in were not allowed to enter the building. The police and

Expected behavior

1. I believe the bug is due to the insufficient initialization of [`track_new["new_seqs"]`](https://github.com/huggingface/transformers/blob/3b00b623b7cad9e1b7c71c97fff24a0286b37045/src/transformers/generation_beam_search.py#L658).
It appears if the top-k hypothesis also advances the constraints.
Then this hypothesis may appear multiple times in the beam as the example output.
The fix is one line in-place.
Updated to `track_new = {"new_seqs": full_hypotheses.tolist(), "new_states": [], "new_indices": [], "new_tokens": [], "new_scores": []}`

2. I believe the bug is due to the incorrect value assignment of [`scores_for_all_vocab`](https://github.com/huggingface/transformers/blob/3b00b623b7cad9e1b7c71c97fff24a0286b37045/src/transformers/generation_utils.py#L3223),
what stores the next **token scores**.
`scores_for_all_vocab` is first passed to [`constrained_beam_scorer.process()`](https://github.com/huggingface/transformers/blob/3b00b623b7cad9e1b7c71c97fff24a0286b37045/src/transformers/generation_utils.py#L3262), and later passed to [`constrained_beam_scorer.step_sentence_constraint()`](https://github.com/huggingface/transformers/blob/3b00b623b7cad9e1b7c71c97fff24a0286b37045/src/transformers/generation_beam_search.py#L596) as `vocab_scores`.
Within its scope, `vocab_scores` is sliced to [`this_batch_token_scores`](https://github.com/huggingface/transformers/blob/3b00b623b7cad9e1b7c71c97fff24a0286b37045/src/transformers/generation_beam_search.py#L654).
`this_batch_token_scores` is finally added to [`track_new["new_scores"]`](https://github.com/huggingface/transformers/blob/3b00b623b7cad9e1b7c71c97fff24a0286b37045/src/transformers/generation_beam_search.py#L686).
However, the derived [`new_scores`](https://github.com/huggingface/transformers/blob/3b00b623b7cad9e1b7c71c97fff24a0286b37045/src/transformers/generation_beam_search.py#L727) is concated with [`sent_beam_scores`](https://github.com/huggingface/transformers/blob/3b00b623b7cad9e1b7c71c97fff24a0286b37045/src/transformers/generation_beam_search.py#L731),
reveal it should represent **beam scores**.
The **token scores** is larger than the expected **beam scores** as ignoring past token scores.
So the unfulfilled hypothesis will advance with an unexpected higher score, and dominate the beam as the example output.
The fix is also straightforward in-place.

# scores_for_all_vocab = next_token_scores_processed.clone()

next_token_scores = next_token_scores_processed + beam_scores[:, None].expand_as(next_token_scores)

scores_for_all_vocab = next_token_scores.clone()