File size: 8,212 Bytes
9915204 731244f |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 |
2021-07-14 22:58:20,643 INFO MainThread:600323 [wandb_setup.py:_flush():69] setting env: {}
2021-07-14 22:58:20,643 INFO MainThread:600323 [wandb_setup.py:_flush():69] setting login settings: {}
2021-07-14 22:58:20,643 INFO MainThread:600323 [wandb_init.py:_log_setup():337] Logging user logs to /home/dat/pino-roberta-base/wandb/run-20210714_225820-1dpoijkp/logs/debug.log
2021-07-14 22:58:20,643 INFO MainThread:600323 [wandb_init.py:_log_setup():338] Logging internal logs to /home/dat/pino-roberta-base/wandb/run-20210714_225820-1dpoijkp/logs/debug-internal.log
2021-07-14 22:58:20,643 INFO MainThread:600323 [wandb_init.py:init():370] calling init triggers
2021-07-14 22:58:20,643 INFO MainThread:600323 [wandb_init.py:init():375] wandb.init called with sweep_config: {}
config: {}
2021-07-14 22:58:20,643 INFO MainThread:600323 [wandb_init.py:init():419] starting backend
2021-07-14 22:58:20,643 INFO MainThread:600323 [backend.py:_multiprocessing_setup():70] multiprocessing start_methods=fork,spawn,forkserver, using: spawn
2021-07-14 22:58:20,687 INFO MainThread:600323 [backend.py:ensure_launched():135] starting backend process...
2021-07-14 22:58:20,729 INFO MainThread:600323 [backend.py:ensure_launched():139] started backend process with pid: 601574
2021-07-14 22:58:20,731 INFO MainThread:600323 [wandb_init.py:init():424] backend started and connected
2021-07-14 22:58:20,734 INFO MainThread:600323 [wandb_init.py:init():472] updated telemetry
2021-07-14 22:58:20,735 INFO MainThread:600323 [wandb_init.py:init():491] communicating current version
2021-07-14 22:58:21,375 INFO MainThread:600323 [wandb_init.py:init():496] got version response
2021-07-14 22:58:21,375 INFO MainThread:600323 [wandb_init.py:init():504] communicating run to backend with 30 second timeout
2021-07-14 22:58:21,602 INFO MainThread:600323 [wandb_init.py:init():529] starting run threads in backend
2021-07-14 22:58:22,739 INFO MainThread:600323 [wandb_run.py:_console_start():1623] atexit reg
2021-07-14 22:58:22,740 INFO MainThread:600323 [wandb_run.py:_redirect():1497] redirect: SettingsConsole.REDIRECT
2021-07-14 22:58:22,740 INFO MainThread:600323 [wandb_run.py:_redirect():1502] Redirecting console.
2021-07-14 22:58:22,742 INFO MainThread:600323 [wandb_run.py:_redirect():1558] Redirects installed.
2021-07-14 22:58:22,742 INFO MainThread:600323 [wandb_init.py:init():554] run started, returning control to user process
2021-07-14 22:58:22,750 INFO MainThread:600323 [wandb_run.py:_config_callback():872] config_cb None None {'output_dir': './', 'overwrite_output_dir': True, 'do_train': False, 'do_eval': False, 'do_predict': False, 'evaluation_strategy': 'IntervalStrategy.NO', 'prediction_loss_only': False, 'per_device_train_batch_size': 2, 'per_device_eval_batch_size': 2, 'per_gpu_train_batch_size': None, 'per_gpu_eval_batch_size': None, 'gradient_accumulation_steps': 8, 'eval_accumulation_steps': None, 'learning_rate': 5e-05, 'weight_decay': 0.0095, 'adam_beta1': 0.9, 'adam_beta2': 0.98, 'adam_epsilon': 1e-08, 'max_grad_norm': 1.0, 'num_train_epochs': 5.0, 'max_steps': -1, 'lr_scheduler_type': 'SchedulerType.LINEAR', 'warmup_ratio': 0.0, 'warmup_steps': 10000, 'log_level': -1, 'log_level_replica': -1, 'log_on_each_node': True, 'logging_dir': './runs/Jul14_22-58-13_t1v-n-f5c06ea1-w-0', 'logging_strategy': 'IntervalStrategy.STEPS', 'logging_first_step': False, 'logging_steps': 500, 'save_strategy': 'IntervalStrategy.STEPS', 'save_steps': 20000, 'save_total_limit': 5, 'save_on_each_node': False, 'no_cuda': False, 'seed': 42, 'fp16': False, 'fp16_opt_level': 'O1', 'fp16_backend': 'auto', 'fp16_full_eval': False, 'local_rank': -1, 'tpu_num_cores': None, 'tpu_metrics_debug': False, 'debug': [], 'dataloader_drop_last': False, 'eval_steps': 20000, 'dataloader_num_workers': 0, 'past_index': -1, 'run_name': './', 'disable_tqdm': False, 'remove_unused_columns': True, 'label_names': None, 'load_best_model_at_end': False, 'metric_for_best_model': None, 'greater_is_better': None, 'ignore_data_skip': False, 'sharded_ddp': [], 'deepspeed': None, 'label_smoothing_factor': 0.0, 'adafactor': False, 'group_by_length': False, 'length_column_name': 'length', 'report_to': ['tensorboard', 'wandb'], 'ddp_find_unused_parameters': None, 'dataloader_pin_memory': True, 'skip_memory_metrics': True, 'use_legacy_prediction_loop': False, 'push_to_hub': True, 'resume_from_checkpoint': None, 'push_to_hub_model_id': '', 'push_to_hub_organization': None, 'push_to_hub_token': None, 'mp_parameters': ''}
2021-07-14 22:58:22,752 INFO MainThread:600323 [wandb_run.py:_config_callback():872] config_cb None None {'model_name_or_path': None, 'model_type': 'big_bird', 'config_name': './', 'tokenizer_name': './', 'cache_dir': None, 'use_fast_tokenizer': True, 'dtype': 'float32', 'save_optimizer': True}
2021-07-14 22:58:22,753 INFO MainThread:600323 [wandb_run.py:_config_callback():872] config_cb None None {'dataset_name': None, 'dataset_config_name': None, 'train_ref_file': None, 'validation_ref_file': None, 'overwrite_cache': False, 'validation_split_percentage': 5, 'max_seq_length': 4096, 'preprocessing_num_workers': 96, 'mlm_probability': 0.15, 'pad_to_max_length': False, 'line_by_line': False, 'max_eval_samples': 2000}
2021-07-14 23:07:48,138 INFO MainThread:600323 [wandb_run.py:_atexit_cleanup():1593] got exitcode: 1
2021-07-14 23:07:48,139 INFO MainThread:600323 [wandb_run.py:_restore():1565] restore
2021-07-14 23:07:50,441 INFO MainThread:600323 [wandb_run.py:_wait_for_finish():1715] got exit ret: file_counts {
wandb_count: 1
}
pusher_stats {
uploaded_bytes: 1448
total_bytes: 1448
}
2021-07-14 23:07:50,747 INFO MainThread:600323 [wandb_run.py:_wait_for_finish():1715] got exit ret: file_counts {
wandb_count: 1
}
pusher_stats {
uploaded_bytes: 1448
total_bytes: 1448
}
2021-07-14 23:07:50,849 INFO MainThread:600323 [wandb_run.py:_wait_for_finish():1715] got exit ret: file_counts {
wandb_count: 5
}
pusher_stats {
uploaded_bytes: 1448
total_bytes: 11717
}
2021-07-14 23:07:50,951 INFO MainThread:600323 [wandb_run.py:_wait_for_finish():1715] got exit ret: file_counts {
wandb_count: 5
}
pusher_stats {
uploaded_bytes: 1448
total_bytes: 11717
}
2021-07-14 23:07:51,054 INFO MainThread:600323 [wandb_run.py:_wait_for_finish():1715] got exit ret: file_counts {
wandb_count: 5
}
pusher_stats {
uploaded_bytes: 11717
total_bytes: 11717
}
2021-07-14 23:07:51,156 INFO MainThread:600323 [wandb_run.py:_wait_for_finish():1715] got exit ret: file_counts {
wandb_count: 5
}
pusher_stats {
uploaded_bytes: 11717
total_bytes: 11717
}
2021-07-14 23:07:51,258 INFO MainThread:600323 [wandb_run.py:_wait_for_finish():1715] got exit ret: file_counts {
wandb_count: 5
}
pusher_stats {
uploaded_bytes: 11717
total_bytes: 11717
}
2021-07-14 23:07:51,360 INFO MainThread:600323 [wandb_run.py:_wait_for_finish():1715] got exit ret: file_counts {
wandb_count: 5
}
pusher_stats {
uploaded_bytes: 11717
total_bytes: 11717
}
2021-07-14 23:07:51,462 INFO MainThread:600323 [wandb_run.py:_wait_for_finish():1715] got exit ret: file_counts {
wandb_count: 5
}
pusher_stats {
uploaded_bytes: 11717
total_bytes: 11717
}
2021-07-14 23:07:51,564 INFO MainThread:600323 [wandb_run.py:_wait_for_finish():1715] got exit ret: file_counts {
wandb_count: 5
}
pusher_stats {
uploaded_bytes: 11717
total_bytes: 11717
}
2021-07-14 23:07:51,665 INFO MainThread:600323 [wandb_run.py:_wait_for_finish():1715] got exit ret: file_counts {
wandb_count: 5
}
pusher_stats {
uploaded_bytes: 11717
total_bytes: 11717
}
2021-07-14 23:07:51,942 INFO MainThread:600323 [wandb_run.py:_wait_for_finish():1715] got exit ret: file_counts {
wandb_count: 5
}
pusher_stats {
uploaded_bytes: 11717
total_bytes: 11717
}
2021-07-14 23:07:52,044 INFO MainThread:600323 [wandb_run.py:_wait_for_finish():1715] got exit ret: done: true
exit_result {
}
file_counts {
wandb_count: 5
}
pusher_stats {
uploaded_bytes: 11717
total_bytes: 11717
}
2021-07-14 23:07:53,341 INFO MainThread:600323 [wandb_run.py:_show_files():1937] logging synced files
|