File size: 8,202 Bytes
f291f93 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 |
2021-07-13 11:11:11,770 INFO MainThread:346512 [wandb_setup.py:_flush():69] setting env: {}
2021-07-13 11:11:11,770 INFO MainThread:346512 [wandb_setup.py:_flush():69] setting login settings: {}
2021-07-13 11:11:11,770 INFO MainThread:346512 [wandb_init.py:_log_setup():337] Logging user logs to /home/dat/pino-roberta-base/wandb/run-20210713_111111-3cwslju0/logs/debug.log
2021-07-13 11:11:11,770 INFO MainThread:346512 [wandb_init.py:_log_setup():338] Logging internal logs to /home/dat/pino-roberta-base/wandb/run-20210713_111111-3cwslju0/logs/debug-internal.log
2021-07-13 11:11:11,770 INFO MainThread:346512 [wandb_init.py:init():370] calling init triggers
2021-07-13 11:11:11,770 INFO MainThread:346512 [wandb_init.py:init():375] wandb.init called with sweep_config: {}
config: {}
2021-07-13 11:11:11,770 INFO MainThread:346512 [wandb_init.py:init():419] starting backend
2021-07-13 11:11:11,771 INFO MainThread:346512 [backend.py:_multiprocessing_setup():70] multiprocessing start_methods=fork,spawn,forkserver, using: spawn
2021-07-13 11:11:11,846 INFO MainThread:346512 [backend.py:ensure_launched():135] starting backend process...
2021-07-13 11:11:11,894 INFO MainThread:346512 [backend.py:ensure_launched():139] started backend process with pid: 348072
2021-07-13 11:11:11,896 INFO MainThread:346512 [wandb_init.py:init():424] backend started and connected
2021-07-13 11:11:11,899 INFO MainThread:346512 [wandb_init.py:init():472] updated telemetry
2021-07-13 11:11:11,900 INFO MainThread:346512 [wandb_init.py:init():491] communicating current version
2021-07-13 11:11:12,551 INFO MainThread:346512 [wandb_init.py:init():496] got version response
2021-07-13 11:11:12,551 INFO MainThread:346512 [wandb_init.py:init():504] communicating run to backend with 30 second timeout
2021-07-13 11:11:12,741 INFO MainThread:346512 [wandb_init.py:init():529] starting run threads in backend
2021-07-13 11:11:14,038 INFO MainThread:346512 [wandb_run.py:_console_start():1623] atexit reg
2021-07-13 11:11:14,039 INFO MainThread:346512 [wandb_run.py:_redirect():1497] redirect: SettingsConsole.REDIRECT
2021-07-13 11:11:14,039 INFO MainThread:346512 [wandb_run.py:_redirect():1502] Redirecting console.
2021-07-13 11:11:14,041 INFO MainThread:346512 [wandb_run.py:_redirect():1558] Redirects installed.
2021-07-13 11:11:14,041 INFO MainThread:346512 [wandb_init.py:init():554] run started, returning control to user process
2021-07-13 11:11:14,047 INFO MainThread:346512 [wandb_run.py:_config_callback():872] config_cb None None {'output_dir': './', 'overwrite_output_dir': True, 'do_train': False, 'do_eval': False, 'do_predict': False, 'evaluation_strategy': 'IntervalStrategy.NO', 'prediction_loss_only': False, 'per_device_train_batch_size': 2, 'per_device_eval_batch_size': 2, 'per_gpu_train_batch_size': None, 'per_gpu_eval_batch_size': None, 'gradient_accumulation_steps': 4, 'eval_accumulation_steps': None, 'learning_rate': 5e-05, 'weight_decay': 0.0095, 'adam_beta1': 0.9, 'adam_beta2': 0.98, 'adam_epsilon': 1e-08, 'max_grad_norm': 1.0, 'num_train_epochs': 5.0, 'max_steps': -1, 'lr_scheduler_type': 'SchedulerType.LINEAR', 'warmup_ratio': 0.0, 'warmup_steps': 10, 'log_level': -1, 'log_level_replica': -1, 'log_on_each_node': True, 'logging_dir': './runs/Jul13_11-10-23_t1v-n-f5c06ea1-w-0', 'logging_strategy': 'IntervalStrategy.STEPS', 'logging_first_step': False, 'logging_steps': 500, 'save_strategy': 'IntervalStrategy.STEPS', 'save_steps': 20000, 'save_total_limit': 5, 'save_on_each_node': False, 'no_cuda': False, 'seed': 42, 'fp16': False, 'fp16_opt_level': 'O1', 'fp16_backend': 'auto', 'fp16_full_eval': False, 'local_rank': -1, 'tpu_num_cores': None, 'tpu_metrics_debug': False, 'debug': [], 'dataloader_drop_last': False, 'eval_steps': 100001, 'dataloader_num_workers': 0, 'past_index': -1, 'run_name': './', 'disable_tqdm': False, 'remove_unused_columns': True, 'label_names': None, 'load_best_model_at_end': False, 'metric_for_best_model': None, 'greater_is_better': None, 'ignore_data_skip': False, 'sharded_ddp': [], 'deepspeed': None, 'label_smoothing_factor': 0.0, 'adafactor': False, 'group_by_length': False, 'length_column_name': 'length', 'report_to': ['tensorboard', 'wandb'], 'ddp_find_unused_parameters': None, 'dataloader_pin_memory': True, 'skip_memory_metrics': True, 'use_legacy_prediction_loop': False, 'push_to_hub': True, 'resume_from_checkpoint': None, 'push_to_hub_model_id': '', 'push_to_hub_organization': None, 'push_to_hub_token': None, 'mp_parameters': ''}
2021-07-13 11:11:14,049 INFO MainThread:346512 [wandb_run.py:_config_callback():872] config_cb None None {'model_name_or_path': None, 'model_type': 'big_bird', 'config_name': './', 'tokenizer_name': './', 'cache_dir': None, 'use_fast_tokenizer': True, 'dtype': 'float32'}
2021-07-13 11:11:14,050 INFO MainThread:346512 [wandb_run.py:_config_callback():872] config_cb None None {'dataset_name': None, 'dataset_config_name': None, 'train_file': None, 'validation_file': None, 'train_ref_file': None, 'validation_ref_file': None, 'overwrite_cache': False, 'validation_split_percentage': 5, 'max_seq_length': 4096, 'preprocessing_num_workers': 64, 'mlm_probability': 0.15, 'pad_to_max_length': False, 'line_by_line': False}
2021-07-13 11:14:25,148 INFO MainThread:346512 [wandb_run.py:_atexit_cleanup():1593] got exitcode: 1
2021-07-13 11:14:25,149 INFO MainThread:346512 [wandb_run.py:_restore():1565] restore
2021-07-13 11:14:27,505 INFO MainThread:346512 [wandb_run.py:_wait_for_finish():1715] got exit ret: file_counts {
wandb_count: 1
}
pusher_stats {
uploaded_bytes: 1365
total_bytes: 1365
}
2021-07-13 11:14:27,706 INFO MainThread:346512 [wandb_run.py:_wait_for_finish():1715] got exit ret: file_counts {
wandb_count: 1
}
pusher_stats {
uploaded_bytes: 1365
total_bytes: 1365
}
2021-07-13 11:14:27,902 INFO MainThread:346512 [wandb_run.py:_wait_for_finish():1715] got exit ret: file_counts {
wandb_count: 1
}
pusher_stats {
uploaded_bytes: 1365
total_bytes: 1365
}
2021-07-13 11:14:28,841 INFO MainThread:346512 [wandb_run.py:_wait_for_finish():1715] got exit ret: file_counts {
wandb_count: 3
}
pusher_stats {
uploaded_bytes: 1365
total_bytes: 6624
}
2021-07-13 11:14:28,944 INFO MainThread:346512 [wandb_run.py:_wait_for_finish():1715] got exit ret: file_counts {
wandb_count: 5
}
pusher_stats {
uploaded_bytes: 1365
total_bytes: 11231
}
2021-07-13 11:14:29,047 INFO MainThread:346512 [wandb_run.py:_wait_for_finish():1715] got exit ret: file_counts {
wandb_count: 5
}
pusher_stats {
uploaded_bytes: 11231
total_bytes: 11231
}
2021-07-13 11:14:29,150 INFO MainThread:346512 [wandb_run.py:_wait_for_finish():1715] got exit ret: file_counts {
wandb_count: 5
}
pusher_stats {
uploaded_bytes: 11231
total_bytes: 11231
}
2021-07-13 11:14:29,252 INFO MainThread:346512 [wandb_run.py:_wait_for_finish():1715] got exit ret: file_counts {
wandb_count: 5
}
pusher_stats {
uploaded_bytes: 11231
total_bytes: 11231
}
2021-07-13 11:14:29,354 INFO MainThread:346512 [wandb_run.py:_wait_for_finish():1715] got exit ret: file_counts {
wandb_count: 5
}
pusher_stats {
uploaded_bytes: 11231
total_bytes: 11231
}
2021-07-13 11:14:29,480 INFO MainThread:346512 [wandb_run.py:_wait_for_finish():1715] got exit ret: file_counts {
wandb_count: 5
}
pusher_stats {
uploaded_bytes: 11231
total_bytes: 11231
}
2021-07-13 11:14:29,583 INFO MainThread:346512 [wandb_run.py:_wait_for_finish():1715] got exit ret: file_counts {
wandb_count: 5
}
pusher_stats {
uploaded_bytes: 11231
total_bytes: 11231
}
2021-07-13 11:14:29,950 INFO MainThread:346512 [wandb_run.py:_wait_for_finish():1715] got exit ret: file_counts {
wandb_count: 5
}
pusher_stats {
uploaded_bytes: 11231
total_bytes: 11231
}
2021-07-13 11:14:30,053 INFO MainThread:346512 [wandb_run.py:_wait_for_finish():1715] got exit ret: done: true
exit_result {
}
file_counts {
wandb_count: 5
}
pusher_stats {
uploaded_bytes: 11231
total_bytes: 11231
}
2021-07-13 11:14:31,336 INFO MainThread:346512 [wandb_run.py:_show_files():1937] logging synced files
|