Repository: huggingface/transformers

Issue:
ValueError: could not broadcast input array from shape (2816,384) into shape (2698,384)
the command to reproduce:
cd huggingface-transformers/examples/pytorch/question-answering
python -m torch.distributed.launch --nproc_per_node=8 ./run_qa.py \
		  --model_name_or_path roberta-large \
		  --dataset_name squad  \
		  --do_train --do_eval \
		  --per_device_train_batch_size 16 \
		  --per_device_eval_batch_size 256 \
		  --learning_rate 3e-5 \
		  --num_train_epochs 2 \
		  --max_seq_length 384 \
		  --doc_stride 128 \
		  --output_dir test_result2/$trials --overwrite_output_dir \
		  --logging_dir test_result2/$trials/tensorboard --logging_first_step --logging_steps 50 \
                  --fp16



i tried add "--max_eval_samples 10240", this will fix the error, while the AUC result is quite low(exact_match  = 4.9414, f1 = 8.9784). and when i ran with 1gpu, the above command can succeed(exact_match  = 88.5336, f1 = 94.3266)


the full error is "File "./transformers/src/transformers/trainer_pt_utils.py", line 410, in _nested_set_tensors
    i * slice_len : (i + 1) * slice_len
    i * slice_len : (i + 1) * slice_len
ValueError: could not broadcast input array from shape (2816,384) into shape (2698,384)"


Discussion:
@sgugger i saw you have a PR to fix similar error, could you help to take a look?


You should upgrade your version of Transformers: this code is not used anymore inside the `Trainer`.
i am installing transformers from source, the issue still threr.

and issue throw from https://github.com/huggingface/transformers/blob/86d5fb0b360e68de46d40265e7c707fe68c8015b/src/transformers/trainer_pt_utils.py#L411, looks like the code is still used in master?



Resolving patch:
diff --git a/examples/pytorch/question-answering/trainer_qa.py b/examples/pytorch/question-answering/trainer_qa.py
--- a/examples/pytorch/question-answering/trainer_qa.py
+++ b/examples/pytorch/question-answering/trainer_qa.py
@@ -39,8 +39,9 @@ def evaluate(self, eval_dataset=None, eval_examples=None, ignore_keys=None):
         # Temporarily disable metric computation, we will do it in the loop here.
         compute_metrics = self.compute_metrics
         self.compute_metrics = None
+        eval_loop = self.prediction_loop if self.args.use_legacy_prediction_loop else self.evaluation_loop
         try:
-            output = self.prediction_loop(
+            output = eval_loop(
                 eval_dataloader,
                 description="Evaluation",
                 # No point gathering the predictions if there are no metrics, otherwise we defer to
@@ -72,8 +73,9 @@ def predict(self, predict_dataset, predict_examples, ignore_keys=None):
         # Temporarily disable metric computation, we will do it in the loop here.
         compute_metrics = self.compute_metrics
         self.compute_metrics = None
+        eval_loop = self.prediction_loop if self.args.use_legacy_prediction_loop else self.evaluation_loop
         try:
-            output = self.prediction_loop(
+            output = eval_loop(
                 predict_dataloader,
                 description="Prediction",
                 # No point gathering the predictions if there are no metrics, otherwise we defer to


Test patch: