2022-05-04 13:30:45 INFO Running runs: []
2022-05-04 13:30:45 INFO Agent received command: run
2022-05-04 13:30:45 INFO Agent starting run with config:
	eval_split_name: test
	eval_steps: 500
	evaluation_strategy: steps
	generation_max_length: 40
	generation_num_beams: 1
	gradient_accumulation_steps: 8
	greater_is_better: True
	hidden_dropout: 0.17305159310134854
	language: fr.en
	learning_rate: 0.00012335092351490598
	logging_steps: 1
	max_duration_in_seconds: 20
	metric_for_best_model: bleu
	model_name_or_path: ./
	num_train_epochs: 3
	output_dir: ./
	per_device_eval_batch_size: 8
	per_device_train_batch_size: 8
	save_steps: 500
	task: covost2
	warmup_steps: 500
2022-05-04 13:30:45 INFO About to run command: python3 run_xtreme_s.py --overwrite_output_dir --freeze_feature_encoder --gradient_checkpointing --predict_with_generate --fp16 --group_by_length --do_train --do_eval --load_best_model_at_end --push_to_hub --use_auth_token --eval_split_name=test --eval_steps=500 --evaluation_strategy=steps --generation_max_length=40 --generation_num_beams=1 --gradient_accumulation_steps=8 --greater_is_better=True --hidden_dropout=0.17305159310134854 --language=fr.en --learning_rate=0.00012335092351490598 --logging_steps=1 --max_duration_in_seconds=20 --metric_for_best_model=bleu --model_name_or_path=./ --num_train_epochs=3 --output_dir=./ --per_device_eval_batch_size=8 --per_device_train_batch_size=8 --save_steps=500 --task=covost2 --warmup_steps=500
2022-05-04 13:30:50 INFO Running runs: ['w4rlzz90']
2022-05-05 09:24:36 ERROR 500 response executing GraphQL.
2022-05-05 09:24:36 ERROR {"errors":[{"message":"context deadline exceeded"}]}
2022-05-05 09:25:32 ERROR 500 response executing GraphQL.
2022-05-05 09:25:32 ERROR {"errors":[{"message":"context deadline exceeded"}]}
2022-05-05 16:55:14 INFO Cleaning up finished run: w4rlzz90
2022-05-05 16:55:15 INFO Agent received command: run
2022-05-05 16:55:15 INFO Agent starting run with config:
	eval_split_name: test
	eval_steps: 500
	evaluation_strategy: steps
	generation_max_length: 40
	generation_num_beams: 1
	gradient_accumulation_steps: 8
	greater_is_better: True
	hidden_dropout: 0.0565117244178473
	language: fr.en
	learning_rate: 0.0004100464310609422
	logging_steps: 1
	max_duration_in_seconds: 20
	metric_for_best_model: bleu
	model_name_or_path: ./
	num_train_epochs: 3
	output_dir: ./
	per_device_eval_batch_size: 8
	per_device_train_batch_size: 8
	save_steps: 500
	task: covost2
	warmup_steps: 500
2022-05-05 16:55:15 INFO About to run command: python3 run_xtreme_s.py --overwrite_output_dir --freeze_feature_encoder --gradient_checkpointing --predict_with_generate --fp16 --group_by_length --do_train --do_eval --load_best_model_at_end --push_to_hub --use_auth_token --eval_split_name=test --eval_steps=500 --evaluation_strategy=steps --generation_max_length=40 --generation_num_beams=1 --gradient_accumulation_steps=8 --greater_is_better=True --hidden_dropout=0.0565117244178473 --language=fr.en --learning_rate=0.0004100464310609422 --logging_steps=1 --max_duration_in_seconds=20 --metric_for_best_model=bleu --model_name_or_path=./ --num_train_epochs=3 --output_dir=./ --per_device_eval_batch_size=8 --per_device_train_batch_size=8 --save_steps=500 --task=covost2 --warmup_steps=500
2022-05-05 16:55:20 INFO Running runs: ['npwlqyuz']
2022-05-05 16:59:27 INFO Cleaning up finished run: npwlqyuz
2022-05-05 16:59:28 INFO Agent received command: run
2022-05-05 16:59:28 INFO Agent starting run with config:
	eval_split_name: test
	eval_steps: 500
	evaluation_strategy: steps
	generation_max_length: 40
	generation_num_beams: 1
	gradient_accumulation_steps: 8
	greater_is_better: True
	hidden_dropout: 0.09631139674620667
	language: fr.en
	learning_rate: 0.0001294077232279209
	logging_steps: 1
	max_duration_in_seconds: 20
	metric_for_best_model: bleu
	model_name_or_path: ./
	num_train_epochs: 3
	output_dir: ./
	per_device_eval_batch_size: 8
	per_device_train_batch_size: 8
	save_steps: 500
	task: covost2
	warmup_steps: 500
2022-05-05 16:59:28 INFO About to run command: python3 run_xtreme_s.py --overwrite_output_dir --freeze_feature_encoder --gradient_checkpointing --predict_with_generate --fp16 --group_by_length --do_train --do_eval --load_best_model_at_end --push_to_hub --use_auth_token --eval_split_name=test --eval_steps=500 --evaluation_strategy=steps --generation_max_length=40 --generation_num_beams=1 --gradient_accumulation_steps=8 --greater_is_better=True --hidden_dropout=0.09631139674620667 --language=fr.en --learning_rate=0.0001294077232279209 --logging_steps=1 --max_duration_in_seconds=20 --metric_for_best_model=bleu --model_name_or_path=./ --num_train_epochs=3 --output_dir=./ --per_device_eval_batch_size=8 --per_device_train_batch_size=8 --save_steps=500 --task=covost2 --warmup_steps=500
2022-05-05 16:59:33 INFO Running runs: ['58ade1nh']
2022-05-05 17:02:30 INFO Cleaning up finished run: 58ade1nh
2022-05-05 17:02:31 INFO Agent received command: run
2022-05-05 17:02:31 INFO Agent starting run with config:
	eval_split_name: test
	eval_steps: 500
	evaluation_strategy: steps
	generation_max_length: 40
	generation_num_beams: 1
	gradient_accumulation_steps: 8
	greater_is_better: True
	hidden_dropout: 0.055807655334937395
	language: fr.en
	learning_rate: 0.0002523256445833689
	logging_steps: 1
	max_duration_in_seconds: 20
	metric_for_best_model: bleu
	model_name_or_path: ./
	num_train_epochs: 3
	output_dir: ./
	per_device_eval_batch_size: 8
	per_device_train_batch_size: 8
	save_steps: 500
	task: covost2
	warmup_steps: 500
2022-05-05 17:02:31 INFO About to run command: python3 run_xtreme_s.py --overwrite_output_dir --freeze_feature_encoder --gradient_checkpointing --predict_with_generate --fp16 --group_by_length --do_train --do_eval --load_best_model_at_end --push_to_hub --use_auth_token --eval_split_name=test --eval_steps=500 --evaluation_strategy=steps --generation_max_length=40 --generation_num_beams=1 --gradient_accumulation_steps=8 --greater_is_better=True --hidden_dropout=0.055807655334937395 --language=fr.en --learning_rate=0.0002523256445833689 --logging_steps=1 --max_duration_in_seconds=20 --metric_for_best_model=bleu --model_name_or_path=./ --num_train_epochs=3 --output_dir=./ --per_device_eval_batch_size=8 --per_device_train_batch_size=8 --save_steps=500 --task=covost2 --warmup_steps=500
2022-05-05 17:02:36 INFO Running runs: ['2zmxvr9t']
2022-05-05 17:05:37 INFO Cleaning up finished run: 2zmxvr9t
2022-05-05 17:05:38 INFO Agent received command: run
2022-05-05 17:05:38 INFO Agent starting run with config:
	eval_split_name: test
	eval_steps: 500
	evaluation_strategy: steps
	generation_max_length: 40
	generation_num_beams: 1
	gradient_accumulation_steps: 8
	greater_is_better: True
	hidden_dropout: 0.08376731063580722
	language: fr.en
	learning_rate: 0.00023509256443134124
	logging_steps: 1
	max_duration_in_seconds: 20
	metric_for_best_model: bleu
	model_name_or_path: ./
	num_train_epochs: 3
	output_dir: ./
	per_device_eval_batch_size: 8
	per_device_train_batch_size: 8
	save_steps: 500
	task: covost2
	warmup_steps: 500
2022-05-05 17:05:38 INFO About to run command: python3 run_xtreme_s.py --overwrite_output_dir --freeze_feature_encoder --gradient_checkpointing --predict_with_generate --fp16 --group_by_length --do_train --do_eval --load_best_model_at_end --push_to_hub --use_auth_token --eval_split_name=test --eval_steps=500 --evaluation_strategy=steps --generation_max_length=40 --generation_num_beams=1 --gradient_accumulation_steps=8 --greater_is_better=True --hidden_dropout=0.08376731063580722 --language=fr.en --learning_rate=0.00023509256443134124 --logging_steps=1 --max_duration_in_seconds=20 --metric_for_best_model=bleu --model_name_or_path=./ --num_train_epochs=3 --output_dir=./ --per_device_eval_batch_size=8 --per_device_train_batch_size=8 --save_steps=500 --task=covost2 --warmup_steps=500
2022-05-05 17:05:43 INFO Running runs: ['s79deysf']
2022-05-05 17:08:40 ERROR Detected 5 failed runs in a row, shutting down.
2022-05-05 17:08:40 INFO To change this value set WANDB_AGENT_MAX_INITIAL_FAILURES=val
2022-05-05 17:31:30 INFO Running runs: []
2022-05-05 17:31:30 INFO Agent received command: run
2022-05-05 17:31:30 INFO Agent starting run with config:
	eval_split_name: test
	eval_steps: 500
	evaluation_strategy: steps
	generation_max_length: 40
	generation_num_beams: 1
	gradient_accumulation_steps: 16
	greater_is_better: True
	hidden_dropout: 0.2
	language: fr.en
	learning_rate: 0.0006039131612823546
	logging_steps: 1
	max_duration_in_seconds: 20
	metric_for_best_model: bleu
	model_name_or_path: ./
	num_train_epochs: 3
	output_dir: ./
	per_device_eval_batch_size: 8
	per_device_train_batch_size: 8
	save_steps: 500
	task: covost2
	warmup_steps: 500
2022-05-05 17:31:30 INFO About to run command: python3 run_xtreme_s.py --overwrite_output_dir --freeze_feature_encoder --gradient_checkpointing --predict_with_generate --fp16 --group_by_length --do_train --do_eval --load_best_model_at_end --push_to_hub --use_auth_token --eval_split_name=test --eval_steps=500 --evaluation_strategy=steps --generation_max_length=40 --generation_num_beams=1 --gradient_accumulation_steps=16 --greater_is_better=True --hidden_dropout=0.2 --language=fr.en --learning_rate=0.0006039131612823546 --logging_steps=1 --max_duration_in_seconds=20 --metric_for_best_model=bleu --model_name_or_path=./ --num_train_epochs=3 --output_dir=./ --per_device_eval_batch_size=8 --per_device_train_batch_size=8 --save_steps=500 --task=covost2 --warmup_steps=500
2022-05-05 17:31:35 INFO Running runs: ['3sxltsfz']
2022-05-05 17:37:18 INFO Running runs: []
2022-05-05 17:37:19 INFO Agent received command: run
2022-05-05 17:37:19 INFO Agent starting run with config:
	eval_split_name: test
	eval_steps: 500
	evaluation_strategy: steps
	generation_max_length: 40
	generation_num_beams: 1
	gradient_accumulation_steps: 16
	greater_is_better: True
	hidden_dropout: 0.2
	language: fr.en
	learning_rate: 0.0003
	logging_steps: 1
	max_duration_in_seconds: 20
	metric_for_best_model: bleu
	model_name_or_path: ./
	num_train_epochs: 3
	output_dir: ./
	per_device_eval_batch_size: 8
	per_device_train_batch_size: 8
	save_steps: 500
	task: covost2
	warmup_steps: 500
2022-05-05 17:37:19 INFO About to run command: python3 run_xtreme_s.py --overwrite_output_dir --freeze_feature_encoder --gradient_checkpointing --predict_with_generate --fp16 --group_by_length --do_train --do_eval --load_best_model_at_end --push_to_hub --use_auth_token --eval_split_name=test --eval_steps=500 --evaluation_strategy=steps --generation_max_length=40 --generation_num_beams=1 --gradient_accumulation_steps=16 --greater_is_better=True --hidden_dropout=0.2 --language=fr.en --learning_rate=0.0003 --logging_steps=1 --max_duration_in_seconds=20 --metric_for_best_model=bleu --model_name_or_path=./ --num_train_epochs=3 --output_dir=./ --per_device_eval_batch_size=8 --per_device_train_batch_size=8 --save_steps=500 --task=covost2 --warmup_steps=500
2022-05-05 17:37:24 INFO Running runs: ['i9acyhfo']
2022-05-06 01:57:37 ERROR 500 response executing GraphQL.
2022-05-06 01:57:37 ERROR {"error":"driver: bad connection"}
2022-05-06 07:05:53 ERROR 500 response executing GraphQL.
2022-05-06 07:05:53 ERROR {"error":"driver: bad connection"}
2022-05-06 07:06:49 ERROR 500 response executing GraphQL.
2022-05-06 07:06:49 ERROR {"errors":[{"message":"context deadline exceeded"}]}
2022-05-06 09:14:19 ERROR 500 response executing GraphQL.
2022-05-06 09:14:19 ERROR {"errors":[{"message":"context deadline exceeded"}]}
2022-05-06 09:15:15 ERROR 500 response executing GraphQL.
2022-05-06 09:15:15 ERROR {"errors":[{"message":"context deadline exceeded"}]}