[2023-05-16 23:59:31,098] [WARNING] [runner.py:191:fetch_hostfile] Unable to find hostfile, will proceed with training with local resources only. [2023-05-16 23:59:31,161] [INFO] [runner.py:541:main] cmd = /home/users/liyurui/environment/deepspeed/bin/python -u -m deepspeed.launcher.launch --world_info=eyJsb2NhbGhvc3QiOiBbMCwgMSwgMiwgMywgNCwgNSwgNiwgN119 --master_addr=127.0.0.1 --master_port=12346 --enable_each_rank_log=None main.py --data_path Dahoas/rm-static --data_split 2,4,4 --actor_model_name_or_path ../step1_supervised_finetuning/output/ --critic_model_name_or_path ../step2_reward_model_finetuning/output/ --num_padding_at_beginning 1 --per_device_train_batch_size 4 --per_device_mini_train_batch_size 4 --generation_batch_numbers 1 --ppo_epochs 1 --max_answer_seq_len 256 --max_prompt_seq_len 256 --actor_learning_rate 9.65e-7 --critic_learning_rate 5e-6 --actor_weight_decay 0.1 --critic_weight_decay 0.1 --num_train_epochs 1 --lr_scheduler_type cosine --gradient_accumulation_steps 1 --actor_gradient_checkpointing --disable_actor_dropout --num_warmup_steps 100 --deepspeed --seed 1234 --enable_hybrid_engine --actor_zero_stage 3 --critic_zero_stage 3 --output_dir ./output [2023-05-16 23:59:33,164] [INFO] [launch.py:229:main] WORLD INFO DICT: {'localhost': [0, 1, 2, 3, 4, 5, 6, 7]} [2023-05-16 23:59:33,164] [INFO] [launch.py:235:main] nnodes=1, num_local_procs=8, node_rank=0 [2023-05-16 23:59:33,164] [INFO] [launch.py:246:main] global_rank_mapping=defaultdict(, {'localhost': [0, 1, 2, 3, 4, 5, 6, 7]}) [2023-05-16 23:59:33,164] [INFO] [launch.py:247:main] dist_world_size=8 [2023-05-16 23:59:33,165] [INFO] [launch.py:249:main] Setting CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 [2023-05-16 23:59:37,593] [INFO] [comm.py:622:init_distributed] Initializing TorchBackend in DeepSpeed with backend nccl ************************[start] Initializing Actor Model [start] ************************* [2023-05-17 00:00:00,449] [INFO] [partition_parameters.py:454:__exit__] finished initializing model with 6.86B parameters Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... Detected CUDA files, patching ldflags Emitting ninja build file /root/.cache/torch_extensions/py39_cu117/fused_adam/build.ninja... Building extension module fused_adam... Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N) ninja: no work to do. Loading extension module fused_adam... Time to load fused_adam op: 0.4698362350463867 seconds Loading extension module fused_adam... Time to load fused_adam op: 0.40274715423583984 seconds Loading extension module fused_adam... Time to load fused_adam op: 0.4026913642883301 seconds Loading extension module fused_adam... Time to load fused_adam op: 0.30251479148864746 seconds Loading extension module fused_adam... Time to load fused_adam op: 0.502511739730835 seconds Loading extension module fused_adam... Time to load fused_adam op: 0.5026230812072754 seconds Loading extension module fused_adam... Time to load fused_adam op: 0.5025103092193604 seconds [2023-05-17 00:00:04,282] [INFO] [logging.py:96:log_dist] [Rank 0] DeepSpeed info: version=0.9.2, git-hash=unknown, git-branch=unknown Loading extension module fused_adam... Time to load fused_adam op: 0.4024386405944824 seconds [2023-05-17 00:00:04,325] [INFO] [logging.py:96:log_dist] [Rank 0] DeepSpeed Flops Profiler Enabled: False [2023-05-17 00:00:04,326] [INFO] [logging.py:96:log_dist] [Rank 0] Removing param_group that has no 'params' in the client Optimizer [2023-05-17 00:00:04,326] [INFO] [logging.py:96:log_dist] [Rank 0] Using client Optimizer as basic optimizer Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... [2023-05-17 00:00:04,361] [INFO] [logging.py:96:log_dist] [Rank 0] DeepSpeed Basic Optimizer = FusedAdam [2023-05-17 00:00:04,361] [INFO] [utils.py:54:is_zero_supported_optimizer] Checking ZeRO support for optimizer=FusedAdam type= [2023-05-17 00:00:04,361] [INFO] [logging.py:96:log_dist] [Rank 0] Creating fp16 ZeRO stage 3 optimizer, MiCS is enabled False, Hierarchical params gather False [2023-05-17 00:00:04,361] [INFO] [logging.py:96:log_dist] [Rank 0] Creating torch.float16 ZeRO stage 3 optimizer Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... Emitting ninja build file /root/.cache/torch_extensions/py39_cu117/utils/build.ninja... Building extension module utils... Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N) [2023-05-17 00:00:04,724] [INFO] [utils.py:785:see_memory_usage] Stage 3 initialize beginning [2023-05-17 00:00:04,725] [INFO] [utils.py:786:see_memory_usage] MA 1.6 GB Max_MA 2.7 GB CA 11.32 GB Max_CA 11 GB [2023-05-17 00:00:04,725] [INFO] [utils.py:793:see_memory_usage] CPU Virtual Memory: used = 106.81 GB, percent = 7.1% [2023-05-17 00:00:04,727] [INFO] [stage3.py:113:__init__] Reduce bucket size 500,000,000 [2023-05-17 00:00:04,727] [INFO] [stage3.py:114:__init__] Prefetch bucket size 30000000 Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... ninja: no work to do. Loading extension module utils... Time to load utils op: 0.4342000484466553 seconds Loading extension module utils... Time to load utils op: 0.10159897804260254 seconds Loading extension module utils... Loading extension module utils... Time to load utils op: 0.5019791126251221 seconds Time to load utils op: 0.5018725395202637 seconds Loading extension module utils... Time to load utils op: 0.5018723011016846 seconds Loading extension module utils... Loading extension module utils... Time to load utils op: 0.5026967525482178 seconds Time to load utils op: 0.5020692348480225 seconds Loading extension module utils... Time to load utils op: 0.5018832683563232 seconds [2023-05-17 00:00:05,101] [INFO] [utils.py:785:see_memory_usage] DeepSpeedZeRoOffload initialize [begin] [2023-05-17 00:00:05,101] [INFO] [utils.py:786:see_memory_usage] MA 1.6 GB Max_MA 1.6 GB CA 11.32 GB Max_CA 11 GB [2023-05-17 00:00:05,102] [INFO] [utils.py:793:see_memory_usage] CPU Virtual Memory: used = 106.81 GB, percent = 7.1% Parameter Offload: Total persistent parameters: 1187840 in 290 params [2023-05-17 00:00:05,396] [INFO] [utils.py:785:see_memory_usage] DeepSpeedZeRoOffload initialize [end] [2023-05-17 00:00:05,397] [INFO] [utils.py:786:see_memory_usage] MA 1.6 GB Max_MA 1.6 GB CA 11.32 GB Max_CA 11 GB [2023-05-17 00:00:05,397] [INFO] [utils.py:793:see_memory_usage] CPU Virtual Memory: used = 106.82 GB, percent = 7.1% [2023-05-17 00:00:05,656] [INFO] [utils.py:785:see_memory_usage] Before creating fp16 partitions [2023-05-17 00:00:05,656] [INFO] [utils.py:786:see_memory_usage] MA 1.6 GB Max_MA 1.6 GB CA 11.32 GB Max_CA 11 GB [2023-05-17 00:00:05,657] [INFO] [utils.py:793:see_memory_usage] CPU Virtual Memory: used = 106.81 GB, percent = 7.1% [2023-05-17 00:00:08,334] [INFO] [utils.py:785:see_memory_usage] After creating fp16 partitions: 2 [2023-05-17 00:00:08,335] [INFO] [utils.py:786:see_memory_usage] MA 1.6 GB Max_MA 1.6 GB CA 1.68 GB Max_CA 11 GB [2023-05-17 00:00:08,335] [INFO] [utils.py:793:see_memory_usage] CPU Virtual Memory: used = 106.84 GB, percent = 7.1% [2023-05-17 00:00:08,613] [INFO] [utils.py:785:see_memory_usage] Before creating fp32 partitions [2023-05-17 00:00:08,614] [INFO] [utils.py:786:see_memory_usage] MA 1.6 GB Max_MA 1.6 GB CA 1.68 GB Max_CA 2 GB [2023-05-17 00:00:08,614] [INFO] [utils.py:793:see_memory_usage] CPU Virtual Memory: used = 106.83 GB, percent = 7.1% [2023-05-17 00:00:08,932] [INFO] [utils.py:785:see_memory_usage] After creating fp32 partitions [2023-05-17 00:00:08,933] [INFO] [utils.py:786:see_memory_usage] MA 4.7 GB Max_MA 6.25 GB CA 6.33 GB Max_CA 6 GB [2023-05-17 00:00:08,933] [INFO] [utils.py:793:see_memory_usage] CPU Virtual Memory: used = 106.84 GB, percent = 7.1% [2023-05-17 00:00:09,264] [INFO] [utils.py:785:see_memory_usage] Before initializing optimizer states [2023-05-17 00:00:09,265] [INFO] [utils.py:786:see_memory_usage] MA 4.7 GB Max_MA 4.7 GB CA 6.33 GB Max_CA 6 GB [2023-05-17 00:00:09,265] [INFO] [utils.py:793:see_memory_usage] CPU Virtual Memory: used = 106.84 GB, percent = 7.1% [2023-05-17 00:00:09,593] [INFO] [utils.py:785:see_memory_usage] After initializing optimizer states [2023-05-17 00:00:09,594] [INFO] [utils.py:786:see_memory_usage] MA 10.9 GB Max_MA 14.0 GB CA 15.64 GB Max_CA 16 GB [2023-05-17 00:00:09,594] [INFO] [utils.py:793:see_memory_usage] CPU Virtual Memory: used = 106.85 GB, percent = 7.1% [2023-05-17 00:00:09,594] [INFO] [stage3.py:392:_setup_for_real_optimizer] optimizer state initialized Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module utils, skipping build step... Loading extension module utils... Time to load utils op: 0.00046706199645996094 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module utils, skipping build step... Loading extension module utils... Time to load utils op: 0.0005366802215576172 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module utils, skipping build step... Loading extension module utils... Time to load utils op: 0.0008149147033691406 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module utils, skipping build step... Loading extension module utils... Time to load utils op: 0.0005071163177490234 seconds No modifications detected for re-loaded extension module utils, skipping build step... Loading extension module utils... Time to load utils op: 0.0015435218811035156 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module utils, skipping build step... Loading extension module utils... Time to load utils op: 0.0007567405700683594 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module utils, skipping build step... Loading extension module utils... Time to load utils op: 0.0008368492126464844 seconds [2023-05-17 00:00:10,144] [INFO] [utils.py:785:see_memory_usage] After initializing ZeRO optimizer [2023-05-17 00:00:10,145] [INFO] [utils.py:786:see_memory_usage] MA 13.38 GB Max_MA 14.15 GB CA 15.64 GB Max_CA 16 GB [2023-05-17 00:00:10,145] [INFO] [utils.py:793:see_memory_usage] CPU Virtual Memory: used = 106.84 GB, percent = 7.1% [2023-05-17 00:00:10,146] [INFO] [logging.py:96:log_dist] [Rank 0] DeepSpeed Final Optimizer = FusedAdam [2023-05-17 00:00:10,146] [INFO] [logging.py:96:log_dist] [Rank 0] DeepSpeed using client LR scheduler [2023-05-17 00:00:10,146] [INFO] [logging.py:96:log_dist] [Rank 0] DeepSpeed LR Scheduler = [2023-05-17 00:00:10,146] [INFO] [logging.py:96:log_dist] [Rank 0] step=0, skipped=0, lr=[0.0, 0.0], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 00:00:10,147] [INFO] [config.py:955:print] DeepSpeedEngine configuration: [2023-05-17 00:00:10,147] [INFO] [config.py:959:print] activation_checkpointing_config { "partition_activations": false, "contiguous_memory_optimization": false, "cpu_checkpointing": false, "number_checkpoints": null, "synchronize_checkpoint_boundary": false, "profile": false } [2023-05-17 00:00:10,147] [INFO] [config.py:959:print] aio_config ................... {'block_size': 1048576, 'queue_depth': 8, 'thread_count': 1, 'single_submit': False, 'overlap_events': True} [2023-05-17 00:00:10,147] [INFO] [config.py:959:print] amp_enabled .................. False [2023-05-17 00:00:10,147] [INFO] [config.py:959:print] amp_params ................... False [2023-05-17 00:00:10,147] [INFO] [config.py:959:print] autotuning_config ............ { "enabled": false, "start_step": null, "end_step": null, "metric_path": null, "arg_mappings": null, "metric": "throughput", "model_info": null, "results_dir": "autotuning_results", "exps_dir": "autotuning_exps", "overwrite": true, "fast": true, "start_profile_step": 3, "end_profile_step": 5, "tuner_type": "gridsearch", "tuner_early_stopping": 5, "tuner_num_trials": 50, "model_info_path": null, "mp_size": 1, "max_train_batch_size": null, "min_train_batch_size": 1, "max_train_micro_batch_size_per_gpu": 1.024000e+03, "min_train_micro_batch_size_per_gpu": 1, "num_tuning_micro_batch_sizes": 3 } [2023-05-17 00:00:10,147] [INFO] [config.py:959:print] bfloat16_enabled ............. False [2023-05-17 00:00:10,148] [INFO] [config.py:959:print] checkpoint_parallel_write_pipeline False [2023-05-17 00:00:10,148] [INFO] [config.py:959:print] checkpoint_tag_validation_enabled True [2023-05-17 00:00:10,148] [INFO] [config.py:959:print] checkpoint_tag_validation_fail False [2023-05-17 00:00:10,148] [INFO] [config.py:959:print] comms_config ................. [2023-05-17 00:00:10,148] [INFO] [config.py:959:print] communication_data_type ...... None [2023-05-17 00:00:10,148] [INFO] [config.py:959:print] compression_config ........... {'weight_quantization': {'shared_parameters': {'enabled': False, 'quantizer_kernel': False, 'schedule_offset': 0, 'quantize_groups': 1, 'quantize_verbose': False, 'quantization_type': 'symmetric', 'quantize_weight_in_forward': False, 'rounding': 'nearest', 'fp16_mixed_quantize': False, 'quantize_change_ratio': 0.001}, 'different_groups': {}}, 'activation_quantization': {'shared_parameters': {'enabled': False, 'quantization_type': 'symmetric', 'range_calibration': 'dynamic', 'schedule_offset': 1000}, 'different_groups': {}}, 'sparse_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'row_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'head_pruning': {'shared_parameters': {'enabled': False, 'method': 'topk', 'schedule_offset': 1000}, 'different_groups': {}}, 'channel_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'layer_reduction': {'enabled': False}} [2023-05-17 00:00:10,148] [INFO] [config.py:959:print] curriculum_enabled_legacy .... False [2023-05-17 00:00:10,148] [INFO] [config.py:959:print] curriculum_params_legacy ..... False [2023-05-17 00:00:10,148] [INFO] [config.py:959:print] data_efficiency_config ....... {'enabled': False, 'seed': 1234, 'data_sampling': {'enabled': False, 'num_epochs': 1000, 'num_workers': 0, 'curriculum_learning': {'enabled': False}}, 'data_routing': {'enabled': False, 'random_ltd': {'enabled': False, 'layer_token_lr_schedule': {'enabled': False}}}} [2023-05-17 00:00:10,148] [INFO] [config.py:959:print] data_efficiency_enabled ...... False [2023-05-17 00:00:10,148] [INFO] [config.py:959:print] dataloader_drop_last ......... False [2023-05-17 00:00:10,148] [INFO] [config.py:959:print] disable_allgather ............ False [2023-05-17 00:00:10,148] [INFO] [config.py:959:print] dump_state ................... False [2023-05-17 00:00:10,148] [INFO] [config.py:959:print] dynamic_loss_scale_args ...... {'init_scale': 65536, 'scale_window': 100, 'delayed_shift': 2, 'min_scale': 1} [2023-05-17 00:00:10,148] [INFO] [config.py:959:print] eigenvalue_enabled ........... False [2023-05-17 00:00:10,148] [INFO] [config.py:959:print] eigenvalue_gas_boundary_resolution 1 [2023-05-17 00:00:10,148] [INFO] [config.py:959:print] eigenvalue_layer_name ........ bert.encoder.layer [2023-05-17 00:00:10,148] [INFO] [config.py:959:print] eigenvalue_layer_num ......... 0 [2023-05-17 00:00:10,148] [INFO] [config.py:959:print] eigenvalue_max_iter .......... 100 [2023-05-17 00:00:10,148] [INFO] [config.py:959:print] eigenvalue_stability ......... 1e-06 [2023-05-17 00:00:10,148] [INFO] [config.py:959:print] eigenvalue_tol ............... 0.01 [2023-05-17 00:00:10,148] [INFO] [config.py:959:print] eigenvalue_verbose ........... False [2023-05-17 00:00:10,148] [INFO] [config.py:959:print] elasticity_enabled ........... False [2023-05-17 00:00:10,148] [INFO] [config.py:959:print] flops_profiler_config ........ { "enabled": false, "profile_step": 1, "module_depth": -1, "top_modules": 1, "detailed": true, "output_file": null } [2023-05-17 00:00:10,148] [INFO] [config.py:959:print] fp16_auto_cast ............... False [2023-05-17 00:00:10,148] [INFO] [config.py:959:print] fp16_enabled ................. True [2023-05-17 00:00:10,148] [INFO] [config.py:959:print] fp16_master_weights_and_gradients False [2023-05-17 00:00:10,148] [INFO] [config.py:959:print] global_rank .................. 0 [2023-05-17 00:00:10,148] [INFO] [config.py:959:print] grad_accum_dtype ............. None [2023-05-17 00:00:10,148] [INFO] [config.py:959:print] gradient_accumulation_steps .. 1 [2023-05-17 00:00:10,148] [INFO] [config.py:959:print] gradient_clipping ............ 1.0 [2023-05-17 00:00:10,148] [INFO] [config.py:959:print] gradient_predivide_factor .... 1.0 [2023-05-17 00:00:10,148] [INFO] [config.py:959:print] hybrid_engine ................ enabled=True max_out_tokens=512 inference_tp_size=1 release_inference_cache=False pin_parameters=True tp_gather_partition_size=8 [2023-05-17 00:00:10,148] [INFO] [config.py:959:print] initial_dynamic_scale ........ 65536 [2023-05-17 00:00:10,148] [INFO] [config.py:959:print] load_universal_checkpoint .... False [2023-05-17 00:00:10,148] [INFO] [config.py:959:print] loss_scale ................... 0 [2023-05-17 00:00:10,148] [INFO] [config.py:959:print] memory_breakdown ............. False [2023-05-17 00:00:10,148] [INFO] [config.py:959:print] mics_hierarchial_params_gather False [2023-05-17 00:00:10,148] [INFO] [config.py:959:print] mics_shard_size .............. -1 [2023-05-17 00:00:10,148] [INFO] [config.py:959:print] monitor_config ............... tensorboard=TensorBoardConfig(enabled=False, output_path='', job_name='DeepSpeedJobName') wandb=WandbConfig(enabled=False, group=None, team=None, project='deepspeed') csv_monitor=CSVConfig(enabled=False, output_path='', job_name='DeepSpeedJobName') enabled=False [2023-05-17 00:00:10,149] [INFO] [config.py:959:print] nebula_config ................ { "enabled": false, "persistent_storage_path": null, "persistent_time_interval": 100, "num_of_version_in_retention": 2, "enable_nebula_load": true, "load_path": null } [2023-05-17 00:00:10,149] [INFO] [config.py:959:print] optimizer_legacy_fusion ...... False [2023-05-17 00:00:10,149] [INFO] [config.py:959:print] optimizer_name ............... None [2023-05-17 00:00:10,149] [INFO] [config.py:959:print] optimizer_params ............. None [2023-05-17 00:00:10,149] [INFO] [config.py:959:print] pipeline ..................... {'stages': 'auto', 'partition': 'best', 'seed_layers': False, 'activation_checkpoint_interval': 0} [2023-05-17 00:00:10,149] [INFO] [config.py:959:print] pld_enabled .................. False [2023-05-17 00:00:10,149] [INFO] [config.py:959:print] pld_params ................... False [2023-05-17 00:00:10,149] [INFO] [config.py:959:print] prescale_gradients ........... False [2023-05-17 00:00:10,149] [INFO] [config.py:959:print] scheduler_name ............... None [2023-05-17 00:00:10,149] [INFO] [config.py:959:print] scheduler_params ............. None [2023-05-17 00:00:10,149] [INFO] [config.py:959:print] sparse_attention ............. None [2023-05-17 00:00:10,149] [INFO] [config.py:959:print] sparse_gradients_enabled ..... False [2023-05-17 00:00:10,149] [INFO] [config.py:959:print] steps_per_print .............. 10 [2023-05-17 00:00:10,149] [INFO] [config.py:959:print] train_batch_size ............. 32 [2023-05-17 00:00:10,149] [INFO] [config.py:959:print] train_micro_batch_size_per_gpu 4 [2023-05-17 00:00:10,149] [INFO] [config.py:959:print] use_node_local_storage ....... False [2023-05-17 00:00:10,149] [INFO] [config.py:959:print] wall_clock_breakdown ......... False [2023-05-17 00:00:10,149] [INFO] [config.py:959:print] world_size ................... 8 [2023-05-17 00:00:10,149] [INFO] [config.py:959:print] zero_allow_untested_optimizer False [2023-05-17 00:00:10,149] [INFO] [config.py:959:print] zero_config .................. stage=3 contiguous_gradients=True reduce_scatter=True reduce_bucket_size=500,000,000 allgather_partitions=True allgather_bucket_size=500,000,000 overlap_comm=True load_from_fp32_weights=True elastic_checkpoint=False offload_param=DeepSpeedZeroOffloadParamConfig(device='none', nvme_path=None, buffer_count=5, buffer_size=100,000,000, max_in_cpu=1,000,000,000, pin_memory=False) offload_optimizer=DeepSpeedZeroOffloadOptimizerConfig(device='none', nvme_path=None, buffer_count=4, pin_memory=False, pipeline=False, pipeline_read=False, pipeline_write=False, fast_init=False) sub_group_size=1,000,000,000 cpu_offload_param=None cpu_offload_use_pin_memory=None cpu_offload=None prefetch_bucket_size=30000000 param_persistence_threshold=10000 model_persistence_threshold=sys.maxsize max_live_parameters=30000000 max_reuse_distance=1,000,000,000 gather_16bit_weights_on_model_save=False stage3_gather_fp16_weights_on_model_save=False ignore_unused_parameters=True legacy_stage1=False round_robin_gradients=False mics_shard_size=-1 mics_hierarchical_params_gather=False memory_efficient_linear=False [2023-05-17 00:00:10,149] [INFO] [config.py:959:print] zero_enabled ................. True [2023-05-17 00:00:10,149] [INFO] [config.py:959:print] zero_force_ds_cpu_optimizer .. True [2023-05-17 00:00:10,149] [INFO] [config.py:959:print] zero_optimization_stage ...... 3 [2023-05-17 00:00:10,149] [INFO] [config.py:945:print_user_config] json = { "train_batch_size": 32, "train_micro_batch_size_per_gpu": 4, "steps_per_print": 10, "zero_optimization": { "stage": 3, "offload_param": { "device": "none" }, "offload_optimizer": { "device": "none" }, "stage3_param_persistence_threshold": 1.000000e+04, "stage3_max_live_parameters": 3.000000e+07, "stage3_prefetch_bucket_size": 3.000000e+07, "memory_efficient_linear": false }, "fp16": { "enabled": true, "loss_scale_window": 100 }, "gradient_clipping": 1.0, "prescale_gradients": false, "wall_clock_breakdown": false, "hybrid_engine": { "enabled": true, "max_out_tokens": 512, "inference_tp_size": 1, "release_inference_cache": false, "pin_parameters": true, "tp_gather_partition_size": 8 } } Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module utils, skipping build step... Loading extension module utils... Time to load utils op: 0.0004086494445800781 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... Detected CUDA files, patching ldflags Emitting ninja build file /root/.cache/torch_extensions/py39_cu117/transformer_inference/build.ninja... Building extension module transformer_inference... Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N) ninja: no work to do. Loading extension module transformer_inference... Time to load transformer_inference op: 0.537193775177002 seconds Loading extension module transformer_inference... Time to load transformer_inference op: 0.5119471549987793 seconds Loading extension module transformer_inference... Loading extension module transformer_inference... Time to load transformer_inference op: 0.6015143394470215 seconds Time to load transformer_inference op: 0.6009294986724854 seconds [2023-05-17 00:00:10,976] [INFO] [logging.py:96:log_dist] [Rank 0] DeepSpeed-Inference config: {'layer_id': 0, 'hidden_size': 4096, 'intermediate_size': 16384, 'heads': 32, 'num_hidden_layers': -1, 'fp16': True, 'pre_layer_norm': True, 'local_rank': -1, 'stochastic_mode': False, 'epsilon': 1e-05, 'mp_size': 1, 'q_int8': False, 'scale_attention': True, 'triangular_masking': True, 'local_attention': False, 'window_size': 1, 'rotary_dim': -1, 'rotate_half': False, 'rotate_every_two': True, 'return_tuple': True, 'mlp_after_attn': True, 'mlp_act_func_type': , 'specialized_mode': False, 'training_mp_size': 1, 'bigscience_bloom': False, 'max_out_tokens': 512, 'min_out_tokens': 512, 'scale_attn_by_inverse_layer_idx': False, 'enable_qkv_quantization': False, 'use_mup': False, 'return_single_tuple': False, 'set_empty_params': True, 'transposed_mode': True} Loading extension module transformer_inference... Time to load transformer_inference op: 0.5978825092315674 seconds Loading extension module transformer_inference... Loading extension module transformer_inference... Time to load transformer_inference op: 0.6001217365264893 seconds Time to load transformer_inference op: 0.6004347801208496 seconds Loading extension module transformer_inference... Time to load transformer_inference op: 0.6028428077697754 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module transformer_inference, skipping build step... Loading extension module transformer_inference... Time to load transformer_inference op: 0.09887290000915527 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module transformer_inference, skipping build step... Loading extension module transformer_inference... Time to load transformer_inference op: 0.10450315475463867 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module transformer_inference, skipping build step... Loading extension module transformer_inference... Time to load transformer_inference op: 0.08766388893127441 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module transformer_inference, skipping build step... Loading extension module transformer_inference... Time to load transformer_inference op: 0.09647250175476074 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module transformer_inference, skipping build step... Loading extension module transformer_inference... No modifications detected for re-loaded extension module transformer_inference, skipping build step... Loading extension module transformer_inference... Time to load transformer_inference op: 0.12305784225463867 seconds Time to load transformer_inference op: 0.12316346168518066 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module transformer_inference, skipping build step... Loading extension module transformer_inference... Time to load transformer_inference op: 0.09761524200439453 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module transformer_inference, skipping build step... Loading extension module transformer_inference... Time to load transformer_inference op: 0.13199210166931152 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module transformer_inference, skipping build step... Loading extension module transformer_inference... Time to load transformer_inference op: 0.09572124481201172 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module transformer_inference, skipping build step... Loading extension module transformer_inference... Time to load transformer_inference op: 0.08826041221618652 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module transformer_inference, skipping build step... Loading extension module transformer_inference... Time to load transformer_inference op: 0.10236310958862305 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module transformer_inference, skipping build step... Loading extension module transformer_inference... No modifications detected for re-loaded extension module transformer_inference, skipping build step... Loading extension module transformer_inference... Time to load transformer_inference op: 0.09708261489868164 seconds Time to load transformer_inference op: 0.10012531280517578 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module transformer_inference, skipping build step... Loading extension module transformer_inference... Time to load transformer_inference op: 0.10323905944824219 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module transformer_inference, skipping build step... Loading extension module transformer_inference... Time to load transformer_inference op: 0.10862135887145996 seconds ******************[end] Initialized Actor Model [end] (duration: 20.30s)****************** *************************[start] Initializing Ref Model [start] ************************** No modifications detected for re-loaded extension module transformer_inference, skipping build step... Loading extension module transformer_inference... Time to load transformer_inference op: 0.09238219261169434 seconds [2023-05-17 00:00:20,435] [INFO] [partition_parameters.py:454:__exit__] finished initializing model with 13.73B parameters Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module utils, skipping build step... Loading extension module utils... Time to load utils op: 0.0006401538848876953 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module utils, skipping build step... Loading extension module utils... Time to load utils op: 0.0005936622619628906 seconds [2023-05-17 00:00:24,410] [INFO] [logging.py:96:log_dist] [Rank 0] DeepSpeed info: version=0.9.2, git-hash=unknown, git-branch=unknown [2023-05-17 00:00:24,426] [INFO] [logging.py:96:log_dist] [Rank 0] DeepSpeed Flops Profiler Enabled: False [2023-05-17 00:00:24,427] [INFO] [logging.py:96:log_dist] [Rank 0] Creating ZeRO Offload Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module utils, skipping build step... Loading extension module utils... Time to load utils op: 0.0007390975952148438 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module utils, skipping build step... Loading extension module utils... Time to load utils op: 0.0006525516510009766 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module utils, skipping build step... Loading extension module utils... Time to load utils op: 0.0006296634674072266 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module utils, skipping build step... Loading extension module utils... Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... Time to load utils op: 0.0008893013000488281 seconds No modifications detected for re-loaded extension module utils, skipping build step... Loading extension module utils... Time to load utils op: 0.0007762908935546875 seconds [2023-05-17 00:00:24,785] [INFO] [utils.py:785:see_memory_usage] DeepSpeedZeRoOffload initialize [begin] [2023-05-17 00:00:24,786] [INFO] [utils.py:786:see_memory_usage] MA 15.08 GB Max_MA 16.18 GB CA 25.18 GB Max_CA 25 GB [2023-05-17 00:00:24,786] [INFO] [utils.py:793:see_memory_usage] CPU Virtual Memory: used = 107.54 GB, percent = 7.1% Parameter Offload: Total persistent parameters: 1187840 in 290 params [2023-05-17 00:00:25,033] [INFO] [utils.py:785:see_memory_usage] DeepSpeedZeRoOffload initialize [end] [2023-05-17 00:00:25,034] [INFO] [utils.py:786:see_memory_usage] MA 15.08 GB Max_MA 15.08 GB CA 25.18 GB Max_CA 25 GB [2023-05-17 00:00:25,034] [INFO] [utils.py:793:see_memory_usage] CPU Virtual Memory: used = 108.26 GB, percent = 7.2% [2023-05-17 00:00:25,035] [INFO] [config.py:955:print] DeepSpeedEngine configuration: [2023-05-17 00:00:25,035] [INFO] [config.py:959:print] activation_checkpointing_config { "partition_activations": false, "contiguous_memory_optimization": false, "cpu_checkpointing": false, "number_checkpoints": null, "synchronize_checkpoint_boundary": false, "profile": false } [2023-05-17 00:00:25,036] [INFO] [config.py:959:print] aio_config ................... {'block_size': 1048576, 'queue_depth': 8, 'thread_count': 1, 'single_submit': False, 'overlap_events': True} [2023-05-17 00:00:25,036] [INFO] [config.py:959:print] amp_enabled .................. False [2023-05-17 00:00:25,036] [INFO] [config.py:959:print] amp_params ................... False [2023-05-17 00:00:25,036] [INFO] [config.py:959:print] autotuning_config ............ { "enabled": false, "start_step": null, "end_step": null, "metric_path": null, "arg_mappings": null, "metric": "throughput", "model_info": null, "results_dir": "autotuning_results", "exps_dir": "autotuning_exps", "overwrite": true, "fast": true, "start_profile_step": 3, "end_profile_step": 5, "tuner_type": "gridsearch", "tuner_early_stopping": 5, "tuner_num_trials": 50, "model_info_path": null, "mp_size": 1, "max_train_batch_size": null, "min_train_batch_size": 1, "max_train_micro_batch_size_per_gpu": 1.024000e+03, "min_train_micro_batch_size_per_gpu": 1, "num_tuning_micro_batch_sizes": 3 } [2023-05-17 00:00:25,036] [INFO] [config.py:959:print] bfloat16_enabled ............. False [2023-05-17 00:00:25,036] [INFO] [config.py:959:print] checkpoint_parallel_write_pipeline False [2023-05-17 00:00:25,036] [INFO] [config.py:959:print] checkpoint_tag_validation_enabled True [2023-05-17 00:00:25,036] [INFO] [config.py:959:print] checkpoint_tag_validation_fail False [2023-05-17 00:00:25,036] [INFO] [config.py:959:print] comms_config ................. [2023-05-17 00:00:25,036] [INFO] [config.py:959:print] communication_data_type ...... None [2023-05-17 00:00:25,036] [INFO] [config.py:959:print] compression_config ........... {'weight_quantization': {'shared_parameters': {'enabled': False, 'quantizer_kernel': False, 'schedule_offset': 0, 'quantize_groups': 1, 'quantize_verbose': False, 'quantization_type': 'symmetric', 'quantize_weight_in_forward': False, 'rounding': 'nearest', 'fp16_mixed_quantize': False, 'quantize_change_ratio': 0.001}, 'different_groups': {}}, 'activation_quantization': {'shared_parameters': {'enabled': False, 'quantization_type': 'symmetric', 'range_calibration': 'dynamic', 'schedule_offset': 1000}, 'different_groups': {}}, 'sparse_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'row_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'head_pruning': {'shared_parameters': {'enabled': False, 'method': 'topk', 'schedule_offset': 1000}, 'different_groups': {}}, 'channel_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'layer_reduction': {'enabled': False}} [2023-05-17 00:00:25,036] [INFO] [config.py:959:print] curriculum_enabled_legacy .... False [2023-05-17 00:00:25,036] [INFO] [config.py:959:print] curriculum_params_legacy ..... False [2023-05-17 00:00:25,036] [INFO] [config.py:959:print] data_efficiency_config ....... {'enabled': False, 'seed': 1234, 'data_sampling': {'enabled': False, 'num_epochs': 1000, 'num_workers': 0, 'curriculum_learning': {'enabled': False}}, 'data_routing': {'enabled': False, 'random_ltd': {'enabled': False, 'layer_token_lr_schedule': {'enabled': False}}}} [2023-05-17 00:00:25,036] [INFO] [config.py:959:print] data_efficiency_enabled ...... False [2023-05-17 00:00:25,036] [INFO] [config.py:959:print] dataloader_drop_last ......... False [2023-05-17 00:00:25,036] [INFO] [config.py:959:print] disable_allgather ............ False [2023-05-17 00:00:25,036] [INFO] [config.py:959:print] dump_state ................... False [2023-05-17 00:00:25,036] [INFO] [config.py:959:print] dynamic_loss_scale_args ...... None [2023-05-17 00:00:25,036] [INFO] [config.py:959:print] eigenvalue_enabled ........... False [2023-05-17 00:00:25,036] [INFO] [config.py:959:print] eigenvalue_gas_boundary_resolution 1 [2023-05-17 00:00:25,036] [INFO] [config.py:959:print] eigenvalue_layer_name ........ bert.encoder.layer [2023-05-17 00:00:25,036] [INFO] [config.py:959:print] eigenvalue_layer_num ......... 0 [2023-05-17 00:00:25,036] [INFO] [config.py:959:print] eigenvalue_max_iter .......... 100 [2023-05-17 00:00:25,036] [INFO] [config.py:959:print] eigenvalue_stability ......... 1e-06 [2023-05-17 00:00:25,036] [INFO] [config.py:959:print] eigenvalue_tol ............... 0.01 [2023-05-17 00:00:25,036] [INFO] [config.py:959:print] eigenvalue_verbose ........... False [2023-05-17 00:00:25,036] [INFO] [config.py:959:print] elasticity_enabled ........... False [2023-05-17 00:00:25,036] [INFO] [config.py:959:print] flops_profiler_config ........ { "enabled": false, "profile_step": 1, "module_depth": -1, "top_modules": 1, "detailed": true, "output_file": null } [2023-05-17 00:00:25,037] [INFO] [config.py:959:print] fp16_auto_cast ............... False [2023-05-17 00:00:25,037] [INFO] [config.py:959:print] fp16_enabled ................. True [2023-05-17 00:00:25,037] [INFO] [config.py:959:print] fp16_master_weights_and_gradients False [2023-05-17 00:00:25,037] [INFO] [config.py:959:print] global_rank .................. 0 [2023-05-17 00:00:25,037] [INFO] [config.py:959:print] grad_accum_dtype ............. None [2023-05-17 00:00:25,037] [INFO] [config.py:959:print] gradient_accumulation_steps .. 1 [2023-05-17 00:00:25,037] [INFO] [config.py:959:print] gradient_clipping ............ 1.0 [2023-05-17 00:00:25,037] [INFO] [config.py:959:print] gradient_predivide_factor .... 1.0 [2023-05-17 00:00:25,037] [INFO] [config.py:959:print] hybrid_engine ................ enabled=False max_out_tokens=512 inference_tp_size=1 release_inference_cache=False pin_parameters=True tp_gather_partition_size=8 [2023-05-17 00:00:25,037] [INFO] [config.py:959:print] initial_dynamic_scale ........ 65536 [2023-05-17 00:00:25,037] [INFO] [config.py:959:print] load_universal_checkpoint .... False [2023-05-17 00:00:25,037] [INFO] [config.py:959:print] loss_scale ................... 0 [2023-05-17 00:00:25,037] [INFO] [config.py:959:print] memory_breakdown ............. False [2023-05-17 00:00:25,037] [INFO] [config.py:959:print] mics_hierarchial_params_gather False [2023-05-17 00:00:25,037] [INFO] [config.py:959:print] mics_shard_size .............. -1 [2023-05-17 00:00:25,037] [INFO] [config.py:959:print] monitor_config ............... tensorboard=TensorBoardConfig(enabled=False, output_path='', job_name='DeepSpeedJobName') wandb=WandbConfig(enabled=False, group=None, team=None, project='deepspeed') csv_monitor=CSVConfig(enabled=False, output_path='', job_name='DeepSpeedJobName') enabled=False [2023-05-17 00:00:25,037] [INFO] [config.py:959:print] nebula_config ................ { "enabled": false, "persistent_storage_path": null, "persistent_time_interval": 100, "num_of_version_in_retention": 2, "enable_nebula_load": true, "load_path": null } [2023-05-17 00:00:25,037] [INFO] [config.py:959:print] optimizer_legacy_fusion ...... False [2023-05-17 00:00:25,037] [INFO] [config.py:959:print] optimizer_name ............... None [2023-05-17 00:00:25,037] [INFO] [config.py:959:print] optimizer_params ............. None [2023-05-17 00:00:25,037] [INFO] [config.py:959:print] pipeline ..................... {'stages': 'auto', 'partition': 'best', 'seed_layers': False, 'activation_checkpoint_interval': 0} [2023-05-17 00:00:25,037] [INFO] [config.py:959:print] pld_enabled .................. False [2023-05-17 00:00:25,037] [INFO] [config.py:959:print] pld_params ................... False [2023-05-17 00:00:25,037] [INFO] [config.py:959:print] prescale_gradients ........... False [2023-05-17 00:00:25,037] [INFO] [config.py:959:print] scheduler_name ............... None [2023-05-17 00:00:25,037] [INFO] [config.py:959:print] scheduler_params ............. None [2023-05-17 00:00:25,037] [INFO] [config.py:959:print] sparse_attention ............. None [2023-05-17 00:00:25,037] [INFO] [config.py:959:print] sparse_gradients_enabled ..... False [2023-05-17 00:00:25,037] [INFO] [config.py:959:print] steps_per_print .............. 10 [2023-05-17 00:00:25,037] [INFO] [config.py:959:print] train_batch_size ............. 32 [2023-05-17 00:00:25,037] [INFO] [config.py:959:print] train_micro_batch_size_per_gpu 4 [2023-05-17 00:00:25,037] [INFO] [config.py:959:print] use_node_local_storage ....... False [2023-05-17 00:00:25,037] [INFO] [config.py:959:print] wall_clock_breakdown ......... False [2023-05-17 00:00:25,037] [INFO] [config.py:959:print] world_size ................... 8 [2023-05-17 00:00:25,037] [INFO] [config.py:959:print] zero_allow_untested_optimizer False [2023-05-17 00:00:25,037] [INFO] [config.py:959:print] zero_config .................. stage=3 contiguous_gradients=True reduce_scatter=True reduce_bucket_size=500,000,000 allgather_partitions=True allgather_bucket_size=500,000,000 overlap_comm=True load_from_fp32_weights=True elastic_checkpoint=False offload_param=DeepSpeedZeroOffloadParamConfig(device='none', nvme_path=None, buffer_count=5, buffer_size=100,000,000, max_in_cpu=1,000,000,000, pin_memory=False) offload_optimizer=None sub_group_size=1,000,000,000 cpu_offload_param=None cpu_offload_use_pin_memory=None cpu_offload=None prefetch_bucket_size=50,000,000 param_persistence_threshold=10000 model_persistence_threshold=sys.maxsize max_live_parameters=1,000,000,000 max_reuse_distance=1,000,000,000 gather_16bit_weights_on_model_save=False stage3_gather_fp16_weights_on_model_save=False ignore_unused_parameters=True legacy_stage1=False round_robin_gradients=False mics_shard_size=-1 mics_hierarchical_params_gather=False memory_efficient_linear=False [2023-05-17 00:00:25,037] [INFO] [config.py:959:print] zero_enabled ................. True [2023-05-17 00:00:25,037] [INFO] [config.py:959:print] zero_force_ds_cpu_optimizer .. True [2023-05-17 00:00:25,038] [INFO] [config.py:959:print] zero_optimization_stage ...... 3 [2023-05-17 00:00:25,038] [INFO] [config.py:945:print_user_config] json = { "train_batch_size": 32, "train_micro_batch_size_per_gpu": 4, "steps_per_print": 10, "zero_optimization": { "stage": 3, "stage3_param_persistence_threshold": 1.000000e+04, "offload_param": { "device": "none" }, "memory_efficient_linear": false }, "fp16": { "enabled": true }, "gradient_clipping": 1.0, "prescale_gradients": false, "wall_clock_breakdown": false } Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module utils, skipping build step... Loading extension module utils... Time to load utils op: 0.00039887428283691406 seconds *******************[end] Initialized Ref Model [end] (duration: 13.40s)******************* ************************[start] Initializing Critic Model [start] ************************ Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module fused_adam, skipping build step... Loading extension module fused_adam... Time to load fused_adam op: 0.0017757415771484375 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module fused_adam, skipping build step... Loading extension module fused_adam... Time to load fused_adam op: 0.0017540454864501953 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module fused_adam, skipping build step... Loading extension module fused_adam... Time to load fused_adam op: 0.001493692398071289 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module fused_adam, skipping build step... Loading extension module fused_adam... Time to load fused_adam op: 0.001377105712890625 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module fused_adam, skipping build step... Loading extension module fused_adam... Time to load fused_adam op: 0.0017273426055908203 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module fused_adam, skipping build step... Loading extension module fused_adam... Time to load fused_adam op: 0.001779794692993164 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module fused_adam, skipping build step... Loading extension module fused_adam... Time to load fused_adam op: 0.001394033432006836 seconds [2023-05-17 00:00:39,333] [INFO] [logging.py:96:log_dist] [Rank 0] DeepSpeed info: version=0.9.2, git-hash=unknown, git-branch=unknown Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module fused_adam, skipping build step... Loading extension module fused_adam... Time to load fused_adam op: 0.0013842582702636719 seconds [2023-05-17 00:00:51,130] [INFO] [logging.py:96:log_dist] [Rank 0] DeepSpeed Flops Profiler Enabled: False [2023-05-17 00:00:51,132] [INFO] [logging.py:96:log_dist] [Rank 0] Removing param_group that has no 'params' in the client Optimizer [2023-05-17 00:00:51,132] [INFO] [logging.py:96:log_dist] [Rank 0] Using client Optimizer as basic optimizer [2023-05-17 00:00:51,149] [INFO] [logging.py:96:log_dist] [Rank 0] DeepSpeed Basic Optimizer = FusedAdam [2023-05-17 00:00:51,149] [INFO] [utils.py:54:is_zero_supported_optimizer] Checking ZeRO support for optimizer=FusedAdam type= [2023-05-17 00:00:51,149] [INFO] [logging.py:96:log_dist] [Rank 0] Creating fp16 ZeRO stage 3 optimizer, MiCS is enabled False, Hierarchical params gather False [2023-05-17 00:00:51,149] [INFO] [logging.py:96:log_dist] [Rank 0] Creating torch.float16 ZeRO stage 3 optimizer Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module utils, skipping build step... Loading extension module utils... Time to load utils op: 0.0007483959197998047 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module utils, skipping build step... Loading extension module utils... Time to load utils op: 0.0007741451263427734 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module utils, skipping build step... Loading extension module utils... No modifications detected for re-loaded extension module utils, skipping build step... Loading extension module utils... Time to load utils op: 0.0007581710815429688 seconds Time to load utils op: 0.000766754150390625 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module utils, skipping build step... Loading extension module utils... Time to load utils op: 0.0020132064819335938 seconds No modifications detected for re-loaded extension module utils, skipping build step... Loading extension module utils... Time to load utils op: 0.002331972122192383 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module utils, skipping build step... Loading extension module utils... Time to load utils op: 0.001508474349975586 seconds [2023-05-17 00:00:51,516] [INFO] [utils.py:785:see_memory_usage] Stage 3 initialize beginning [2023-05-17 00:00:51,517] [INFO] [utils.py:786:see_memory_usage] MA 15.69 GB Max_MA 15.69 GB CA 25.18 GB Max_CA 25 GB [2023-05-17 00:00:51,517] [INFO] [utils.py:793:see_memory_usage] CPU Virtual Memory: used = 116.25 GB, percent = 7.7% [2023-05-17 00:00:51,519] [INFO] [stage3.py:113:__init__] Reduce bucket size 500,000,000 [2023-05-17 00:00:51,519] [INFO] [stage3.py:114:__init__] Prefetch bucket size 30000000 Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module utils, skipping build step... Loading extension module utils... Time to load utils op: 0.00040221214294433594 seconds [2023-05-17 00:00:51,764] [INFO] [utils.py:785:see_memory_usage] DeepSpeedZeRoOffload initialize [begin] [2023-05-17 00:00:51,765] [INFO] [utils.py:786:see_memory_usage] MA 15.69 GB Max_MA 15.69 GB CA 25.18 GB Max_CA 25 GB [2023-05-17 00:00:51,765] [INFO] [utils.py:793:see_memory_usage] CPU Virtual Memory: used = 116.25 GB, percent = 7.7% Parameter Offload: Total persistent parameters: 320000 in 241 params [2023-05-17 00:00:52,121] [INFO] [utils.py:785:see_memory_usage] DeepSpeedZeRoOffload initialize [end] [2023-05-17 00:00:52,122] [INFO] [utils.py:786:see_memory_usage] MA 15.15 GB Max_MA 15.7 GB CA 25.25 GB Max_CA 25 GB [2023-05-17 00:00:52,122] [INFO] [utils.py:793:see_memory_usage] CPU Virtual Memory: used = 116.26 GB, percent = 7.7% [2023-05-17 00:00:52,367] [INFO] [utils.py:785:see_memory_usage] Before creating fp16 partitions [2023-05-17 00:00:52,368] [INFO] [utils.py:786:see_memory_usage] MA 15.15 GB Max_MA 15.15 GB CA 25.25 GB Max_CA 25 GB [2023-05-17 00:00:52,368] [INFO] [utils.py:793:see_memory_usage] CPU Virtual Memory: used = 116.26 GB, percent = 7.7% [2023-05-17 00:00:53,079] [INFO] [utils.py:785:see_memory_usage] After creating fp16 partitions: 2 [2023-05-17 00:00:53,081] [INFO] [utils.py:786:see_memory_usage] MA 15.15 GB Max_MA 15.15 GB CA 19.39 GB Max_CA 25 GB [2023-05-17 00:00:53,081] [INFO] [utils.py:793:see_memory_usage] CPU Virtual Memory: used = 116.51 GB, percent = 7.7% [2023-05-17 00:00:53,349] [INFO] [utils.py:785:see_memory_usage] Before creating fp32 partitions [2023-05-17 00:00:53,350] [INFO] [utils.py:786:see_memory_usage] MA 15.15 GB Max_MA 15.15 GB CA 19.39 GB Max_CA 19 GB [2023-05-17 00:00:53,350] [INFO] [utils.py:793:see_memory_usage] CPU Virtual Memory: used = 116.28 GB, percent = 7.7% [2023-05-17 00:00:53,596] [INFO] [utils.py:785:see_memory_usage] After creating fp32 partitions [2023-05-17 00:00:53,597] [INFO] [utils.py:786:see_memory_usage] MA 15.31 GB Max_MA 15.38 GB CA 19.39 GB Max_CA 19 GB [2023-05-17 00:00:53,597] [INFO] [utils.py:793:see_memory_usage] CPU Virtual Memory: used = 116.28 GB, percent = 7.7% [2023-05-17 00:00:53,861] [INFO] [utils.py:785:see_memory_usage] Before initializing optimizer states [2023-05-17 00:00:53,862] [INFO] [utils.py:786:see_memory_usage] MA 15.31 GB Max_MA 15.31 GB CA 19.39 GB Max_CA 19 GB [2023-05-17 00:00:53,862] [INFO] [utils.py:793:see_memory_usage] CPU Virtual Memory: used = 116.29 GB, percent = 7.7% [2023-05-17 00:00:54,125] [INFO] [utils.py:785:see_memory_usage] After initializing optimizer states [2023-05-17 00:00:54,126] [INFO] [utils.py:786:see_memory_usage] MA 15.62 GB Max_MA 15.77 GB CA 19.39 GB Max_CA 19 GB [2023-05-17 00:00:54,126] [INFO] [utils.py:793:see_memory_usage] CPU Virtual Memory: used = 116.28 GB, percent = 7.7% [2023-05-17 00:00:54,126] [INFO] [stage3.py:392:_setup_for_real_optimizer] optimizer state initialized Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module utils, skipping build step... Loading extension module utils... Time to load utils op: 0.0005028247833251953 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module utils, skipping build step... Loading extension module utils... Time to load utils op: 0.0006129741668701172 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module utils, skipping build step... Loading extension module utils... Time to load utils op: 0.0006966590881347656 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module utils, skipping build step... Loading extension module utils... Time to load utils op: 0.0010144710540771484 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module utils, skipping build step... Loading extension module utils... Time to load utils op: 0.0006630420684814453 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module utils, skipping build step... Loading extension module utils... Time to load utils op: 0.0004916191101074219 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module utils, skipping build step... Loading extension module utils... Time to load utils op: 0.0004532337188720703 seconds [2023-05-17 00:00:54,636] [INFO] [utils.py:785:see_memory_usage] After initializing ZeRO optimizer [2023-05-17 00:00:54,637] [INFO] [utils.py:786:see_memory_usage] MA 16.62 GB Max_MA 16.72 GB CA 20.33 GB Max_CA 20 GB [2023-05-17 00:00:54,637] [INFO] [utils.py:793:see_memory_usage] CPU Virtual Memory: used = 116.98 GB, percent = 7.7% [2023-05-17 00:00:54,637] [INFO] [logging.py:96:log_dist] [Rank 0] DeepSpeed Final Optimizer = FusedAdam [2023-05-17 00:00:54,637] [INFO] [logging.py:96:log_dist] [Rank 0] DeepSpeed using client LR scheduler [2023-05-17 00:00:54,638] [INFO] [logging.py:96:log_dist] [Rank 0] DeepSpeed LR Scheduler = [2023-05-17 00:00:54,638] [INFO] [logging.py:96:log_dist] [Rank 0] step=0, skipped=0, lr=[0.0, 0.0], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 00:00:54,638] [INFO] [config.py:955:print] DeepSpeedEngine configuration: [2023-05-17 00:00:54,638] [INFO] [config.py:959:print] activation_checkpointing_config { "partition_activations": false, "contiguous_memory_optimization": false, "cpu_checkpointing": false, "number_checkpoints": null, "synchronize_checkpoint_boundary": false, "profile": false } [2023-05-17 00:00:54,638] [INFO] [config.py:959:print] aio_config ................... {'block_size': 1048576, 'queue_depth': 8, 'thread_count': 1, 'single_submit': False, 'overlap_events': True} [2023-05-17 00:00:54,638] [INFO] [config.py:959:print] amp_enabled .................. False [2023-05-17 00:00:54,638] [INFO] [config.py:959:print] amp_params ................... False [2023-05-17 00:00:54,639] [INFO] [config.py:959:print] autotuning_config ............ { "enabled": false, "start_step": null, "end_step": null, "metric_path": null, "arg_mappings": null, "metric": "throughput", "model_info": null, "results_dir": "autotuning_results", "exps_dir": "autotuning_exps", "overwrite": true, "fast": true, "start_profile_step": 3, "end_profile_step": 5, "tuner_type": "gridsearch", "tuner_early_stopping": 5, "tuner_num_trials": 50, "model_info_path": null, "mp_size": 1, "max_train_batch_size": null, "min_train_batch_size": 1, "max_train_micro_batch_size_per_gpu": 1.024000e+03, "min_train_micro_batch_size_per_gpu": 1, "num_tuning_micro_batch_sizes": 3 } [2023-05-17 00:00:54,639] [INFO] [config.py:959:print] bfloat16_enabled ............. False [2023-05-17 00:00:54,639] [INFO] [config.py:959:print] checkpoint_parallel_write_pipeline False [2023-05-17 00:00:54,639] [INFO] [config.py:959:print] checkpoint_tag_validation_enabled True [2023-05-17 00:00:54,639] [INFO] [config.py:959:print] checkpoint_tag_validation_fail False [2023-05-17 00:00:54,639] [INFO] [config.py:959:print] comms_config ................. [2023-05-17 00:00:54,639] [INFO] [config.py:959:print] communication_data_type ...... None [2023-05-17 00:00:54,639] [INFO] [config.py:959:print] compression_config ........... {'weight_quantization': {'shared_parameters': {'enabled': False, 'quantizer_kernel': False, 'schedule_offset': 0, 'quantize_groups': 1, 'quantize_verbose': False, 'quantization_type': 'symmetric', 'quantize_weight_in_forward': False, 'rounding': 'nearest', 'fp16_mixed_quantize': False, 'quantize_change_ratio': 0.001}, 'different_groups': {}}, 'activation_quantization': {'shared_parameters': {'enabled': False, 'quantization_type': 'symmetric', 'range_calibration': 'dynamic', 'schedule_offset': 1000}, 'different_groups': {}}, 'sparse_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'row_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'head_pruning': {'shared_parameters': {'enabled': False, 'method': 'topk', 'schedule_offset': 1000}, 'different_groups': {}}, 'channel_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'layer_reduction': {'enabled': False}} [2023-05-17 00:00:54,639] [INFO] [config.py:959:print] curriculum_enabled_legacy .... False [2023-05-17 00:00:54,639] [INFO] [config.py:959:print] curriculum_params_legacy ..... False [2023-05-17 00:00:54,639] [INFO] [config.py:959:print] data_efficiency_config ....... {'enabled': False, 'seed': 1234, 'data_sampling': {'enabled': False, 'num_epochs': 1000, 'num_workers': 0, 'curriculum_learning': {'enabled': False}}, 'data_routing': {'enabled': False, 'random_ltd': {'enabled': False, 'layer_token_lr_schedule': {'enabled': False}}}} [2023-05-17 00:00:54,639] [INFO] [config.py:959:print] data_efficiency_enabled ...... False [2023-05-17 00:00:54,639] [INFO] [config.py:959:print] dataloader_drop_last ......... False [2023-05-17 00:00:54,639] [INFO] [config.py:959:print] disable_allgather ............ False [2023-05-17 00:00:54,639] [INFO] [config.py:959:print] dump_state ................... False [2023-05-17 00:00:54,639] [INFO] [config.py:959:print] dynamic_loss_scale_args ...... {'init_scale': 65536, 'scale_window': 100, 'delayed_shift': 2, 'min_scale': 1} [2023-05-17 00:00:54,639] [INFO] [config.py:959:print] eigenvalue_enabled ........... False [2023-05-17 00:00:54,639] [INFO] [config.py:959:print] eigenvalue_gas_boundary_resolution 1 [2023-05-17 00:00:54,639] [INFO] [config.py:959:print] eigenvalue_layer_name ........ bert.encoder.layer [2023-05-17 00:00:54,639] [INFO] [config.py:959:print] eigenvalue_layer_num ......... 0 [2023-05-17 00:00:54,639] [INFO] [config.py:959:print] eigenvalue_max_iter .......... 100 [2023-05-17 00:00:54,639] [INFO] [config.py:959:print] eigenvalue_stability ......... 1e-06 [2023-05-17 00:00:54,639] [INFO] [config.py:959:print] eigenvalue_tol ............... 0.01 [2023-05-17 00:00:54,639] [INFO] [config.py:959:print] eigenvalue_verbose ........... False [2023-05-17 00:00:54,639] [INFO] [config.py:959:print] elasticity_enabled ........... False [2023-05-17 00:00:54,639] [INFO] [config.py:959:print] flops_profiler_config ........ { "enabled": false, "profile_step": 1, "module_depth": -1, "top_modules": 1, "detailed": true, "output_file": null } [2023-05-17 00:00:54,639] [INFO] [config.py:959:print] fp16_auto_cast ............... False [2023-05-17 00:00:54,639] [INFO] [config.py:959:print] fp16_enabled ................. True [2023-05-17 00:00:54,639] [INFO] [config.py:959:print] fp16_master_weights_and_gradients False [2023-05-17 00:00:54,639] [INFO] [config.py:959:print] global_rank .................. 0 [2023-05-17 00:00:54,639] [INFO] [config.py:959:print] grad_accum_dtype ............. None [2023-05-17 00:00:54,639] [INFO] [config.py:959:print] gradient_accumulation_steps .. 1 [2023-05-17 00:00:54,639] [INFO] [config.py:959:print] gradient_clipping ............ 1.0 [2023-05-17 00:00:54,640] [INFO] [config.py:959:print] gradient_predivide_factor .... 1.0 [2023-05-17 00:00:54,640] [INFO] [config.py:959:print] hybrid_engine ................ enabled=False max_out_tokens=512 inference_tp_size=1 release_inference_cache=False pin_parameters=True tp_gather_partition_size=8 [2023-05-17 00:00:54,640] [INFO] [config.py:959:print] initial_dynamic_scale ........ 65536 [2023-05-17 00:00:54,640] [INFO] [config.py:959:print] load_universal_checkpoint .... False [2023-05-17 00:00:54,640] [INFO] [config.py:959:print] loss_scale ................... 0 [2023-05-17 00:00:54,640] [INFO] [config.py:959:print] memory_breakdown ............. False [2023-05-17 00:00:54,640] [INFO] [config.py:959:print] mics_hierarchial_params_gather False [2023-05-17 00:00:54,640] [INFO] [config.py:959:print] mics_shard_size .............. -1 [2023-05-17 00:00:54,640] [INFO] [config.py:959:print] monitor_config ............... tensorboard=TensorBoardConfig(enabled=False, output_path='', job_name='DeepSpeedJobName') wandb=WandbConfig(enabled=False, group=None, team=None, project='deepspeed') csv_monitor=CSVConfig(enabled=False, output_path='', job_name='DeepSpeedJobName') enabled=False [2023-05-17 00:00:54,640] [INFO] [config.py:959:print] nebula_config ................ { "enabled": false, "persistent_storage_path": null, "persistent_time_interval": 100, "num_of_version_in_retention": 2, "enable_nebula_load": true, "load_path": null } [2023-05-17 00:00:54,640] [INFO] [config.py:959:print] optimizer_legacy_fusion ...... False [2023-05-17 00:00:54,640] [INFO] [config.py:959:print] optimizer_name ............... None [2023-05-17 00:00:54,640] [INFO] [config.py:959:print] optimizer_params ............. None [2023-05-17 00:00:54,640] [INFO] [config.py:959:print] pipeline ..................... {'stages': 'auto', 'partition': 'best', 'seed_layers': False, 'activation_checkpoint_interval': 0} [2023-05-17 00:00:54,640] [INFO] [config.py:959:print] pld_enabled .................. False [2023-05-17 00:00:54,640] [INFO] [config.py:959:print] pld_params ................... False [2023-05-17 00:00:54,640] [INFO] [config.py:959:print] prescale_gradients ........... False [2023-05-17 00:00:54,640] [INFO] [config.py:959:print] scheduler_name ............... None [2023-05-17 00:00:54,640] [INFO] [config.py:959:print] scheduler_params ............. None [2023-05-17 00:00:54,640] [INFO] [config.py:959:print] sparse_attention ............. None [2023-05-17 00:00:54,640] [INFO] [config.py:959:print] sparse_gradients_enabled ..... False [2023-05-17 00:00:54,640] [INFO] [config.py:959:print] steps_per_print .............. 10 [2023-05-17 00:00:54,640] [INFO] [config.py:959:print] train_batch_size ............. 32 [2023-05-17 00:00:54,640] [INFO] [config.py:959:print] train_micro_batch_size_per_gpu 4 [2023-05-17 00:00:54,640] [INFO] [config.py:959:print] use_node_local_storage ....... False [2023-05-17 00:00:54,640] [INFO] [config.py:959:print] wall_clock_breakdown ......... False [2023-05-17 00:00:54,640] [INFO] [config.py:959:print] world_size ................... 8 [2023-05-17 00:00:54,640] [INFO] [config.py:959:print] zero_allow_untested_optimizer False [2023-05-17 00:00:54,640] [INFO] [config.py:959:print] zero_config .................. stage=3 contiguous_gradients=True reduce_scatter=True reduce_bucket_size=500,000,000 allgather_partitions=True allgather_bucket_size=500,000,000 overlap_comm=True load_from_fp32_weights=True elastic_checkpoint=False offload_param=DeepSpeedZeroOffloadParamConfig(device='none', nvme_path=None, buffer_count=5, buffer_size=100,000,000, max_in_cpu=1,000,000,000, pin_memory=False) offload_optimizer=DeepSpeedZeroOffloadOptimizerConfig(device='none', nvme_path=None, buffer_count=4, pin_memory=False, pipeline=False, pipeline_read=False, pipeline_write=False, fast_init=False) sub_group_size=1,000,000,000 cpu_offload_param=None cpu_offload_use_pin_memory=None cpu_offload=None prefetch_bucket_size=30000000 param_persistence_threshold=10000 model_persistence_threshold=sys.maxsize max_live_parameters=30000000 max_reuse_distance=1,000,000,000 gather_16bit_weights_on_model_save=False stage3_gather_fp16_weights_on_model_save=False ignore_unused_parameters=True legacy_stage1=False round_robin_gradients=False mics_shard_size=-1 mics_hierarchical_params_gather=False memory_efficient_linear=False [2023-05-17 00:00:54,640] [INFO] [config.py:959:print] zero_enabled ................. True [2023-05-17 00:00:54,640] [INFO] [config.py:959:print] zero_force_ds_cpu_optimizer .. True [2023-05-17 00:00:54,640] [INFO] [config.py:959:print] zero_optimization_stage ...... 3 [2023-05-17 00:00:54,640] [INFO] [config.py:945:print_user_config] json = { "train_batch_size": 32, "train_micro_batch_size_per_gpu": 4, "steps_per_print": 10, "zero_optimization": { "stage": 3, "offload_param": { "device": "none" }, "offload_optimizer": { "device": "none" }, "stage3_param_persistence_threshold": 1.000000e+04, "stage3_max_live_parameters": 3.000000e+07, "stage3_prefetch_bucket_size": 3.000000e+07, "memory_efficient_linear": false }, "fp16": { "enabled": true, "loss_scale_window": 100 }, "gradient_clipping": 1.0, "prescale_gradients": false, "wall_clock_breakdown": false, "hybrid_engine": { "enabled": false, "max_out_tokens": 512, "inference_tp_size": 1, "release_inference_cache": false, "pin_parameters": true, "tp_gather_partition_size": 8 } } Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module utils, skipping build step... Loading extension module utils... Time to load utils op: 0.0003383159637451172 seconds *****************[end] Initialized Critic Model [end] (duration: 29.60s)****************** ************************[start] Initializing Reward Model [start] ************************ [2023-05-17 00:01:11,766] [INFO] [logging.py:96:log_dist] [Rank 0] DeepSpeed info: version=0.9.2, git-hash=unknown, git-branch=unknown Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module utils, skipping build step... Loading extension module utils... Time to load utils op: 0.0009508132934570312 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module utils, skipping build step... Loading extension module utils... Time to load utils op: 0.0009582042694091797 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module utils, skipping build step... Loading extension module utils... Time to load utils op: 0.0008373260498046875 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module utils, skipping build step... Loading extension module utils... Time to load utils op: 0.000774383544921875 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module utils, skipping build step... Loading extension module utils... Time to load utils op: 0.0010979175567626953 seconds Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module utils, skipping build step... Loading extension module utils... Time to load utils op: 0.0009930133819580078 seconds [2023-05-17 00:01:20,804] [INFO] [logging.py:96:log_dist] [Rank 0] DeepSpeed Flops Profiler Enabled: False [2023-05-17 00:01:20,806] [INFO] [logging.py:96:log_dist] [Rank 0] Creating ZeRO Offload Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module utils, skipping build step... Loading extension module utils... Time to load utils op: 0.0008032321929931641 seconds [2023-05-17 00:01:21,186] [INFO] [utils.py:785:see_memory_usage] DeepSpeedZeRoOffload initialize [begin] [2023-05-17 00:01:21,186] [INFO] [utils.py:786:see_memory_usage] MA 17.24 GB Max_MA 17.24 GB CA 20.33 GB Max_CA 20 GB [2023-05-17 00:01:21,187] [INFO] [utils.py:793:see_memory_usage] CPU Virtual Memory: used = 122.2 GB, percent = 8.1% Parameter Offload: Total persistent parameters: 320000 in 241 params [2023-05-17 00:01:21,573] [INFO] [utils.py:785:see_memory_usage] DeepSpeedZeRoOffload initialize [end] [2023-05-17 00:01:21,574] [INFO] [utils.py:786:see_memory_usage] MA 16.7 GB Max_MA 17.25 GB CA 20.39 GB Max_CA 20 GB [2023-05-17 00:01:21,574] [INFO] [utils.py:793:see_memory_usage] CPU Virtual Memory: used = 122.21 GB, percent = 8.1% [2023-05-17 00:01:21,575] [INFO] [config.py:955:print] DeepSpeedEngine configuration: [2023-05-17 00:01:21,575] [INFO] [config.py:959:print] activation_checkpointing_config { "partition_activations": false, "contiguous_memory_optimization": false, "cpu_checkpointing": false, "number_checkpoints": null, "synchronize_checkpoint_boundary": false, "profile": false } [2023-05-17 00:01:21,575] [INFO] [config.py:959:print] aio_config ................... {'block_size': 1048576, 'queue_depth': 8, 'thread_count': 1, 'single_submit': False, 'overlap_events': True} [2023-05-17 00:01:21,575] [INFO] [config.py:959:print] amp_enabled .................. False [2023-05-17 00:01:21,575] [INFO] [config.py:959:print] amp_params ................... False [2023-05-17 00:01:21,576] [INFO] [config.py:959:print] autotuning_config ............ { "enabled": false, "start_step": null, "end_step": null, "metric_path": null, "arg_mappings": null, "metric": "throughput", "model_info": null, "results_dir": "autotuning_results", "exps_dir": "autotuning_exps", "overwrite": true, "fast": true, "start_profile_step": 3, "end_profile_step": 5, "tuner_type": "gridsearch", "tuner_early_stopping": 5, "tuner_num_trials": 50, "model_info_path": null, "mp_size": 1, "max_train_batch_size": null, "min_train_batch_size": 1, "max_train_micro_batch_size_per_gpu": 1.024000e+03, "min_train_micro_batch_size_per_gpu": 1, "num_tuning_micro_batch_sizes": 3 } [2023-05-17 00:01:21,576] [INFO] [config.py:959:print] bfloat16_enabled ............. False [2023-05-17 00:01:21,576] [INFO] [config.py:959:print] checkpoint_parallel_write_pipeline False [2023-05-17 00:01:21,576] [INFO] [config.py:959:print] checkpoint_tag_validation_enabled True [2023-05-17 00:01:21,576] [INFO] [config.py:959:print] checkpoint_tag_validation_fail False [2023-05-17 00:01:21,576] [INFO] [config.py:959:print] comms_config ................. [2023-05-17 00:01:21,576] [INFO] [config.py:959:print] communication_data_type ...... None [2023-05-17 00:01:21,576] [INFO] [config.py:959:print] compression_config ........... {'weight_quantization': {'shared_parameters': {'enabled': False, 'quantizer_kernel': False, 'schedule_offset': 0, 'quantize_groups': 1, 'quantize_verbose': False, 'quantization_type': 'symmetric', 'quantize_weight_in_forward': False, 'rounding': 'nearest', 'fp16_mixed_quantize': False, 'quantize_change_ratio': 0.001}, 'different_groups': {}}, 'activation_quantization': {'shared_parameters': {'enabled': False, 'quantization_type': 'symmetric', 'range_calibration': 'dynamic', 'schedule_offset': 1000}, 'different_groups': {}}, 'sparse_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'row_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'head_pruning': {'shared_parameters': {'enabled': False, 'method': 'topk', 'schedule_offset': 1000}, 'different_groups': {}}, 'channel_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'layer_reduction': {'enabled': False}} [2023-05-17 00:01:21,576] [INFO] [config.py:959:print] curriculum_enabled_legacy .... False [2023-05-17 00:01:21,576] [INFO] [config.py:959:print] curriculum_params_legacy ..... False [2023-05-17 00:01:21,576] [INFO] [config.py:959:print] data_efficiency_config ....... {'enabled': False, 'seed': 1234, 'data_sampling': {'enabled': False, 'num_epochs': 1000, 'num_workers': 0, 'curriculum_learning': {'enabled': False}}, 'data_routing': {'enabled': False, 'random_ltd': {'enabled': False, 'layer_token_lr_schedule': {'enabled': False}}}} [2023-05-17 00:01:21,576] [INFO] [config.py:959:print] data_efficiency_enabled ...... False [2023-05-17 00:01:21,576] [INFO] [config.py:959:print] dataloader_drop_last ......... False [2023-05-17 00:01:21,576] [INFO] [config.py:959:print] disable_allgather ............ False [2023-05-17 00:01:21,576] [INFO] [config.py:959:print] dump_state ................... False [2023-05-17 00:01:21,576] [INFO] [config.py:959:print] dynamic_loss_scale_args ...... None [2023-05-17 00:01:21,576] [INFO] [config.py:959:print] eigenvalue_enabled ........... False [2023-05-17 00:01:21,576] [INFO] [config.py:959:print] eigenvalue_gas_boundary_resolution 1 [2023-05-17 00:01:21,576] [INFO] [config.py:959:print] eigenvalue_layer_name ........ bert.encoder.layer [2023-05-17 00:01:21,576] [INFO] [config.py:959:print] eigenvalue_layer_num ......... 0 [2023-05-17 00:01:21,576] [INFO] [config.py:959:print] eigenvalue_max_iter .......... 100 [2023-05-17 00:01:21,576] [INFO] [config.py:959:print] eigenvalue_stability ......... 1e-06 [2023-05-17 00:01:21,576] [INFO] [config.py:959:print] eigenvalue_tol ............... 0.01 [2023-05-17 00:01:21,576] [INFO] [config.py:959:print] eigenvalue_verbose ........... False [2023-05-17 00:01:21,576] [INFO] [config.py:959:print] elasticity_enabled ........... False [2023-05-17 00:01:21,576] [INFO] [config.py:959:print] flops_profiler_config ........ { "enabled": false, "profile_step": 1, "module_depth": -1, "top_modules": 1, "detailed": true, "output_file": null } [2023-05-17 00:01:21,576] [INFO] [config.py:959:print] fp16_auto_cast ............... False [2023-05-17 00:01:21,576] [INFO] [config.py:959:print] fp16_enabled ................. True [2023-05-17 00:01:21,576] [INFO] [config.py:959:print] fp16_master_weights_and_gradients False [2023-05-17 00:01:21,576] [INFO] [config.py:959:print] global_rank .................. 0 [2023-05-17 00:01:21,576] [INFO] [config.py:959:print] grad_accum_dtype ............. None [2023-05-17 00:01:21,576] [INFO] [config.py:959:print] gradient_accumulation_steps .. 1 [2023-05-17 00:01:21,576] [INFO] [config.py:959:print] gradient_clipping ............ 1.0 [2023-05-17 00:01:21,576] [INFO] [config.py:959:print] gradient_predivide_factor .... 1.0 [2023-05-17 00:01:21,576] [INFO] [config.py:959:print] hybrid_engine ................ enabled=False max_out_tokens=512 inference_tp_size=1 release_inference_cache=False pin_parameters=True tp_gather_partition_size=8 [2023-05-17 00:01:21,576] [INFO] [config.py:959:print] initial_dynamic_scale ........ 65536 [2023-05-17 00:01:21,576] [INFO] [config.py:959:print] load_universal_checkpoint .... False [2023-05-17 00:01:21,577] [INFO] [config.py:959:print] loss_scale ................... 0 [2023-05-17 00:01:21,577] [INFO] [config.py:959:print] memory_breakdown ............. False [2023-05-17 00:01:21,577] [INFO] [config.py:959:print] mics_hierarchial_params_gather False [2023-05-17 00:01:21,577] [INFO] [config.py:959:print] mics_shard_size .............. -1 [2023-05-17 00:01:21,577] [INFO] [config.py:959:print] monitor_config ............... tensorboard=TensorBoardConfig(enabled=False, output_path='', job_name='DeepSpeedJobName') wandb=WandbConfig(enabled=False, group=None, team=None, project='deepspeed') csv_monitor=CSVConfig(enabled=False, output_path='', job_name='DeepSpeedJobName') enabled=False [2023-05-17 00:01:21,577] [INFO] [config.py:959:print] nebula_config ................ { "enabled": false, "persistent_storage_path": null, "persistent_time_interval": 100, "num_of_version_in_retention": 2, "enable_nebula_load": true, "load_path": null } [2023-05-17 00:01:21,577] [INFO] [config.py:959:print] optimizer_legacy_fusion ...... False [2023-05-17 00:01:21,577] [INFO] [config.py:959:print] optimizer_name ............... None [2023-05-17 00:01:21,577] [INFO] [config.py:959:print] optimizer_params ............. None [2023-05-17 00:01:21,577] [INFO] [config.py:959:print] pipeline ..................... {'stages': 'auto', 'partition': 'best', 'seed_layers': False, 'activation_checkpoint_interval': 0} [2023-05-17 00:01:21,577] [INFO] [config.py:959:print] pld_enabled .................. False [2023-05-17 00:01:21,577] [INFO] [config.py:959:print] pld_params ................... False [2023-05-17 00:01:21,577] [INFO] [config.py:959:print] prescale_gradients ........... False [2023-05-17 00:01:21,577] [INFO] [config.py:959:print] scheduler_name ............... None [2023-05-17 00:01:21,577] [INFO] [config.py:959:print] scheduler_params ............. None [2023-05-17 00:01:21,577] [INFO] [config.py:959:print] sparse_attention ............. None [2023-05-17 00:01:21,577] [INFO] [config.py:959:print] sparse_gradients_enabled ..... False [2023-05-17 00:01:21,577] [INFO] [config.py:959:print] steps_per_print .............. 10 [2023-05-17 00:01:21,577] [INFO] [config.py:959:print] train_batch_size ............. 32 [2023-05-17 00:01:21,577] [INFO] [config.py:959:print] train_micro_batch_size_per_gpu 4 [2023-05-17 00:01:21,577] [INFO] [config.py:959:print] use_node_local_storage ....... False [2023-05-17 00:01:21,577] [INFO] [config.py:959:print] wall_clock_breakdown ......... False [2023-05-17 00:01:21,577] [INFO] [config.py:959:print] world_size ................... 8 [2023-05-17 00:01:21,577] [INFO] [config.py:959:print] zero_allow_untested_optimizer False [2023-05-17 00:01:21,577] [INFO] [config.py:959:print] zero_config .................. stage=3 contiguous_gradients=True reduce_scatter=True reduce_bucket_size=500,000,000 allgather_partitions=True allgather_bucket_size=500,000,000 overlap_comm=True load_from_fp32_weights=True elastic_checkpoint=False offload_param=DeepSpeedZeroOffloadParamConfig(device='none', nvme_path=None, buffer_count=5, buffer_size=100,000,000, max_in_cpu=1,000,000,000, pin_memory=False) offload_optimizer=None sub_group_size=1,000,000,000 cpu_offload_param=None cpu_offload_use_pin_memory=None cpu_offload=None prefetch_bucket_size=50,000,000 param_persistence_threshold=10000 model_persistence_threshold=sys.maxsize max_live_parameters=1,000,000,000 max_reuse_distance=1,000,000,000 gather_16bit_weights_on_model_save=False stage3_gather_fp16_weights_on_model_save=False ignore_unused_parameters=True legacy_stage1=False round_robin_gradients=False mics_shard_size=-1 mics_hierarchical_params_gather=False memory_efficient_linear=False [2023-05-17 00:01:21,577] [INFO] [config.py:959:print] zero_enabled ................. True [2023-05-17 00:01:21,577] [INFO] [config.py:959:print] zero_force_ds_cpu_optimizer .. True [2023-05-17 00:01:21,577] [INFO] [config.py:959:print] zero_optimization_stage ...... 3 [2023-05-17 00:01:21,577] [INFO] [config.py:945:print_user_config] json = { "train_batch_size": 32, "train_micro_batch_size_per_gpu": 4, "steps_per_print": 10, "zero_optimization": { "stage": 3, "stage3_param_persistence_threshold": 1.000000e+04, "offload_param": { "device": "none" }, "memory_efficient_linear": false }, "fp16": { "enabled": true }, "gradient_clipping": 1.0, "prescale_gradients": false, "wall_clock_breakdown": false } Using /root/.cache/torch_extensions/py39_cu117 as PyTorch extensions root... No modifications detected for re-loaded extension module utils, skipping build step... Loading extension module utils... Time to load utils op: 0.00037026405334472656 seconds *****************[end] Initialized Reward Model [end] (duration: 26.94s)****************** ***** Running training ***** Beginning of Epoch 1/1, Total Generation Batches 954 ------------------------------------------------------ Free memory : 48.290222 (GigaBytes) Total memory: 79.346863 (GigaBytes) Requested memory: 1.218750 (GigaBytes) Setting maximum total tokens (input + output) to 512 WorkSpace: 0x7f334c000000 ------------------------------------------------------ [2023-05-17 00:01:35,638] [INFO] [loss_scaler.py:188:update_scale] [deepspeed] OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 65536, but hysteresis is 2. Reducing hysteresis to 1 [2023-05-17 00:01:36,826] [INFO] [loss_scaler.py:188:update_scale] [deepspeed] OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 65536, but hysteresis is 2. Reducing hysteresis to 1 epoch: 0|step: 0|ppo_ep: 1|act_loss: -0.053009033203125|cri_loss: 0.055511474609375|unsuper_loss: 0.0 average reward score: -3.73046875 ------------------------------------------------------------------------------------- |E2E latency=14.99s |Gather latency=0.17s (1.13%) |Generate time=10.43s (69.59%) |Training time=3.18s (21.20%) |Others=1.38 (9.21%)|CurSamplesPerSec=2.13 |AvgSamplesPerSec=2.13 [2023-05-17 00:01:51,616] [INFO] [loss_scaler.py:181:update_scale] [deepspeed] OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 65536, reducing to 32768 [2023-05-17 00:01:52,993] [INFO] [loss_scaler.py:181:update_scale] [deepspeed] OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 65536, reducing to 32768 epoch: 0|step: 1|ppo_ep: 1|act_loss: -0.09442138671875|cri_loss: 0.167236328125|unsuper_loss: 0.0 average reward score: -4.6796875 ------------------------------------------------------------------------------------- |E2E latency=16.16s |Gather latency=0.97s (5.99%) |Generate time=8.77s (54.24%) |Training time=4.11s (25.46%) |Others=3.28 (20.30%)|CurSamplesPerSec=1.98 |AvgSamplesPerSec=2.05 epoch: 0|step: 2|ppo_ep: 1|act_loss: -0.0179443359375|cri_loss: 0.0399169921875|unsuper_loss: 0.0 average reward score: -3.576171875 ------------------------------------------------------------------------------------- |E2E latency=13.81s |Gather latency=0.96s (6.94%) |Generate time=8.69s (62.94%) |Training time=2.54s (18.40%) |Others=2.58 (18.66%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.14 epoch: 0|step: 3|ppo_ep: 1|act_loss: 0.0714111328125|cri_loss: 0.056304931640625|unsuper_loss: 0.0 average reward score: -2.802734375 ------------------------------------------------------------------------------------- |E2E latency=13.90s |Gather latency=0.83s (5.95%) |Generate time=8.68s (62.43%) |Training time=2.40s (17.26%) |Others=2.82 (20.31%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.17 epoch: 0|step: 4|ppo_ep: 1|act_loss: -0.053741455078125|cri_loss: 0.04583740234375|unsuper_loss: 0.0 average reward score: -2.955078125 ------------------------------------------------------------------------------------- |E2E latency=13.89s |Gather latency=0.61s (4.39%) |Generate time=8.95s (64.47%) |Training time=2.44s (17.56%) |Others=2.50 (17.97%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.20 epoch: 0|step: 5|ppo_ep: 1|act_loss: -0.0164947509765625|cri_loss: 0.0199737548828125|unsuper_loss: 0.0 average reward score: -3.15234375 ------------------------------------------------------------------------------------- |E2E latency=13.72s |Gather latency=0.99s (7.22%) |Generate time=8.58s (62.56%) |Training time=2.38s (17.37%) |Others=2.75 (20.07%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.22 epoch: 0|step: 6|ppo_ep: 1|act_loss: -0.076904296875|cri_loss: 0.046783447265625|unsuper_loss: 0.0 average reward score: -2.224609375 ------------------------------------------------------------------------------------- |E2E latency=13.51s |Gather latency=0.99s (7.32%) |Generate time=8.70s (64.35%) |Training time=2.30s (16.98%) |Others=2.52 (18.66%)|CurSamplesPerSec=2.37 |AvgSamplesPerSec=2.24 epoch: 0|step: 7|ppo_ep: 1|act_loss: -0.04901123046875|cri_loss: 0.031494140625|unsuper_loss: 0.0 average reward score: -4.140625 ------------------------------------------------------------------------------------- |E2E latency=14.09s |Gather latency=0.62s (4.37%) |Generate time=9.19s (65.25%) |Training time=2.38s (16.91%) |Others=2.51 (17.84%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.24 [2023-05-17 00:03:30,129] [INFO] [loss_scaler.py:181:update_scale] [deepspeed] OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 32768, reducing to 16384 epoch: 0|step: 8|ppo_ep: 1|act_loss: -0.0889892578125|cri_loss: 0.053985595703125|unsuper_loss: 0.0 average reward score: -2.810546875 ------------------------------------------------------------------------------------- |E2E latency=14.22s |Gather latency=0.76s (5.35%) |Generate time=9.19s (64.62%) |Training time=2.30s (16.15%) |Others=2.73 (19.23%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.24 [2023-05-17 00:03:43,234] [INFO] [logging.py:96:log_dist] [Rank 0] step=10, skipped=2, lr=[7.72e-08, 7.72e-08], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 00:03:43,235] [INFO] [timer.py:199:stop] epoch=0/micro_step=10/global_step=10, RunningAvgSamplesPerSec=28.043070409862878, CurrSamplesPerSec=27.650999012771344, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 00:03:43,936] [INFO] [logging.py:96:log_dist] [Rank 0] step=10, skipped=3, lr=[3.5000000000000004e-07, 3.5000000000000004e-07], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 9|ppo_ep: 1|act_loss: -0.034698486328125|cri_loss: 0.032257080078125|unsuper_loss: 0.0 average reward score: -3.171875 ------------------------------------------------------------------------------------- |E2E latency=13.81s |Gather latency=0.44s (3.21%) |Generate time=8.73s (63.21%) |Training time=2.53s (18.32%) |Others=2.55 (18.47%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.25 epoch: 0|step: 10|ppo_ep: 1|act_loss: 0.00807952880859375|cri_loss: 0.023406982421875|unsuper_loss: 0.0 average reward score: -3.001953125 ------------------------------------------------------------------------------------- |E2E latency=14.31s |Gather latency=0.64s (4.49%) |Generate time=9.18s (64.17%) |Training time=2.47s (17.22%) |Others=2.66 (18.61%)|CurSamplesPerSec=2.24 |AvgSamplesPerSec=2.25 epoch: 0|step: 11|ppo_ep: 1|act_loss: 0.0274200439453125|cri_loss: 0.032135009765625|unsuper_loss: 0.0 average reward score: -3.1015625 ------------------------------------------------------------------------------------- |E2E latency=14.12s |Gather latency=1.20s (8.47%) |Generate time=9.05s (64.06%) |Training time=2.46s (17.43%) |Others=2.61 (18.51%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.25 epoch: 0|step: 12|ppo_ep: 1|act_loss: 0.07421875|cri_loss: 0.0228729248046875|unsuper_loss: 0.0 average reward score: -3.66015625 ------------------------------------------------------------------------------------- |E2E latency=13.76s |Gather latency=0.83s (6.07%) |Generate time=8.71s (63.26%) |Training time=2.37s (17.23%) |Others=2.69 (19.52%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.26 epoch: 0|step: 13|ppo_ep: 1|act_loss: -0.06884765625|cri_loss: 0.039276123046875|unsuper_loss: 0.0 average reward score: -2.955078125 ------------------------------------------------------------------------------------- |E2E latency=13.95s |Gather latency=1.02s (7.32%) |Generate time=8.68s (62.22%) |Training time=2.36s (16.95%) |Others=2.91 (20.84%)|CurSamplesPerSec=2.29 |AvgSamplesPerSec=2.26 epoch: 0|step: 14|ppo_ep: 1|act_loss: -0.152099609375|cri_loss: 0.05718994140625|unsuper_loss: 0.0 average reward score: -2.869140625 ------------------------------------------------------------------------------------- |E2E latency=13.94s |Gather latency=0.98s (7.02%) |Generate time=9.04s (64.81%) |Training time=2.35s (16.88%) |Others=2.55 (18.32%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.26 epoch: 0|step: 15|ppo_ep: 1|act_loss: -0.0211181640625|cri_loss: 0.032501220703125|unsuper_loss: 0.0 average reward score: -2.7890625 ------------------------------------------------------------------------------------- |E2E latency=13.93s |Gather latency=0.56s (4.00%) |Generate time=8.77s (62.97%) |Training time=2.38s (17.07%) |Others=2.78 (19.96%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.26 epoch: 0|step: 16|ppo_ep: 1|act_loss: -0.061737060546875|cri_loss: 0.0262298583984375|unsuper_loss: 0.0 average reward score: -3.650390625 ------------------------------------------------------------------------------------- |E2E latency=13.72s |Gather latency=0.71s (5.21%) |Generate time=8.64s (63.03%) |Training time=2.40s (17.47%) |Others=2.67 (19.50%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.27 epoch: 0|step: 17|ppo_ep: 1|act_loss: -0.0182647705078125|cri_loss: 0.037445068359375|unsuper_loss: 0.0 average reward score: -2.77734375 ------------------------------------------------------------------------------------- |E2E latency=14.15s |Gather latency=0.96s (6.76%) |Generate time=9.01s (63.64%) |Training time=2.38s (16.83%) |Others=2.76 (19.53%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.27 epoch: 0|step: 18|ppo_ep: 1|act_loss: -0.060455322265625|cri_loss: 0.039398193359375|unsuper_loss: 0.0 average reward score: -3.609375 ------------------------------------------------------------------------------------- |E2E latency=13.96s |Gather latency=0.86s (6.15%) |Generate time=8.69s (62.27%) |Training time=2.43s (17.39%) |Others=2.84 (20.34%)|CurSamplesPerSec=2.29 |AvgSamplesPerSec=2.27 [2023-05-17 00:06:02,857] [INFO] [logging.py:96:log_dist] [Rank 0] step=20, skipped=2, lr=[1.737e-07, 1.737e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 00:06:02,858] [INFO] [timer.py:199:stop] epoch=0/micro_step=20/global_step=20, RunningAvgSamplesPerSec=28.139316738205743, CurrSamplesPerSec=28.516560898080783, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 00:06:03,621] [INFO] [logging.py:96:log_dist] [Rank 0] step=20, skipped=3, lr=[8.500000000000001e-07, 8.500000000000001e-07], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 19|ppo_ep: 1|act_loss: -0.10137939453125|cri_loss: 0.054840087890625|unsuper_loss: 0.0 average reward score: -3.037109375 ------------------------------------------------------------------------------------- |E2E latency=13.84s |Gather latency=1.14s (8.26%) |Generate time=8.83s (63.82%) |Training time=2.35s (16.95%) |Others=2.66 (19.23%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.27 epoch: 0|step: 20|ppo_ep: 1|act_loss: -0.05181884765625|cri_loss: 0.047515869140625|unsuper_loss: 0.0 average reward score: -3.734375 ------------------------------------------------------------------------------------- |E2E latency=13.67s |Gather latency=0.82s (5.98%) |Generate time=8.71s (63.70%) |Training time=2.31s (16.88%) |Others=2.65 (19.42%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.27 [2023-05-17 00:06:30,200] [INFO] [loss_scaler.py:181:update_scale] [deepspeed] OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 32768, reducing to 16384 epoch: 0|step: 21|ppo_ep: 1|act_loss: -0.125732421875|cri_loss: 0.048492431640625|unsuper_loss: 0.0 average reward score: -3.91015625 ------------------------------------------------------------------------------------- |E2E latency=13.62s |Gather latency=0.87s (6.42%) |Generate time=8.66s (63.58%) |Training time=2.27s (16.65%) |Others=2.69 (19.78%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.28 epoch: 0|step: 22|ppo_ep: 1|act_loss: -0.11114501953125|cri_loss: 0.040863037109375|unsuper_loss: 0.0 average reward score: -3.5859375 ------------------------------------------------------------------------------------- |E2E latency=13.74s |Gather latency=0.88s (6.39%) |Generate time=8.75s (63.72%) |Training time=2.46s (17.89%) |Others=2.53 (18.40%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.28 epoch: 0|step: 23|ppo_ep: 1|act_loss: -0.149169921875|cri_loss: 0.1270751953125|unsuper_loss: 0.0 average reward score: -3.83203125 ------------------------------------------------------------------------------------- |E2E latency=14.10s |Gather latency=1.08s (7.68%) |Generate time=8.92s (63.28%) |Training time=2.43s (17.26%) |Others=2.74 (19.45%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.28 epoch: 0|step: 24|ppo_ep: 1|act_loss: -0.0272674560546875|cri_loss: 0.040740966796875|unsuper_loss: 0.0 average reward score: -3.724609375 ------------------------------------------------------------------------------------- |E2E latency=13.81s |Gather latency=0.93s (6.72%) |Generate time=8.66s (62.70%) |Training time=2.39s (17.27%) |Others=2.77 (20.03%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.28 epoch: 0|step: 25|ppo_ep: 1|act_loss: -0.052154541015625|cri_loss: 0.02447509765625|unsuper_loss: 0.0 average reward score: -2.8828125 ------------------------------------------------------------------------------------- |E2E latency=13.80s |Gather latency=0.71s (5.16%) |Generate time=8.76s (63.48%) |Training time=2.36s (17.06%) |Others=2.69 (19.45%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.28 epoch: 0|step: 26|ppo_ep: 1|act_loss: -0.04742431640625|cri_loss: 0.0408935546875|unsuper_loss: 0.0 average reward score: -3.220703125 ------------------------------------------------------------------------------------- |E2E latency=13.90s |Gather latency=0.81s (5.82%) |Generate time=8.72s (62.73%) |Training time=2.38s (17.13%) |Others=2.80 (20.14%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.28 epoch: 0|step: 27|ppo_ep: 1|act_loss: -0.04864501953125|cri_loss: 0.03656005859375|unsuper_loss: 0.0 average reward score: -3.2265625 ------------------------------------------------------------------------------------- |E2E latency=13.67s |Gather latency=0.58s (4.22%) |Generate time=8.75s (64.01%) |Training time=2.37s (17.32%) |Others=2.55 (18.67%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.29 epoch: 0|step: 28|ppo_ep: 1|act_loss: 0.133056640625|cri_loss: 0.039825439453125|unsuper_loss: 0.0 average reward score: -3.609375 ------------------------------------------------------------------------------------- |E2E latency=13.81s |Gather latency=0.46s (3.34%) |Generate time=8.79s (63.65%) |Training time=2.29s (16.61%) |Others=2.73 (19.74%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.29 [2023-05-17 00:08:20,859] [INFO] [logging.py:96:log_dist] [Rank 0] step=30, skipped=3, lr=[2.6055000000000004e-07, 2.6055000000000004e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 00:08:20,860] [INFO] [timer.py:199:stop] epoch=0/micro_step=30/global_step=30, RunningAvgSamplesPerSec=28.30804235563935, CurrSamplesPerSec=29.461571556261095, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 00:08:21,598] [INFO] [logging.py:96:log_dist] [Rank 0] step=30, skipped=3, lr=[1.3500000000000002e-06, 1.3500000000000002e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 29|ppo_ep: 1|act_loss: -0.04656982421875|cri_loss: 0.06512451171875|unsuper_loss: 0.0 average reward score: -3.16015625 ------------------------------------------------------------------------------------- |E2E latency=13.85s |Gather latency=0.62s (4.50%) |Generate time=8.76s (63.27%) |Training time=2.35s (17.00%) |Others=2.73 (19.73%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.29 epoch: 0|step: 30|ppo_ep: 1|act_loss: 0.0305328369140625|cri_loss: 0.04974365234375|unsuper_loss: 0.0 average reward score: -2.681640625 ------------------------------------------------------------------------------------- |E2E latency=13.72s |Gather latency=0.93s (6.80%) |Generate time=8.76s (63.83%) |Training time=2.34s (17.03%) |Others=2.63 (19.14%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.29 epoch: 0|step: 31|ppo_ep: 1|act_loss: 0.02557373046875|cri_loss: 0.06884765625|unsuper_loss: 0.0 average reward score: -3.4140625 ------------------------------------------------------------------------------------- |E2E latency=13.84s |Gather latency=0.80s (5.75%) |Generate time=8.74s (63.10%) |Training time=2.40s (17.30%) |Others=2.71 (19.60%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.29 epoch: 0|step: 32|ppo_ep: 1|act_loss: 0.00537872314453125|cri_loss: 0.080810546875|unsuper_loss: 0.0 average reward score: -2.5 ------------------------------------------------------------------------------------- |E2E latency=13.82s |Gather latency=1.07s (7.75%) |Generate time=8.89s (64.28%) |Training time=2.29s (16.54%) |Others=2.65 (19.18%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.29 epoch: 0|step: 33|ppo_ep: 1|act_loss: 0.0732421875|cri_loss: 0.05438232421875|unsuper_loss: 0.0 average reward score: -3.28515625 ------------------------------------------------------------------------------------- |E2E latency=13.63s |Gather latency=0.89s (6.50%) |Generate time=8.68s (63.65%) |Training time=2.35s (17.22%) |Others=2.61 (19.14%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.29 epoch: 0|step: 34|ppo_ep: 1|act_loss: 0.11541748046875|cri_loss: 0.06427001953125|unsuper_loss: 0.0 average reward score: -3.521484375 ------------------------------------------------------------------------------------- |E2E latency=13.75s |Gather latency=0.74s (5.36%) |Generate time=8.61s (62.64%) |Training time=2.35s (17.06%) |Others=2.79 (20.30%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.29 epoch: 0|step: 35|ppo_ep: 1|act_loss: 0.0230712890625|cri_loss: 0.048431396484375|unsuper_loss: 0.0 average reward score: -3.52734375 ------------------------------------------------------------------------------------- |E2E latency=13.74s |Gather latency=0.79s (5.76%) |Generate time=8.70s (63.33%) |Training time=2.37s (17.22%) |Others=2.67 (19.45%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.29 epoch: 0|step: 36|ppo_ep: 1|act_loss: -0.0450439453125|cri_loss: 0.04425048828125|unsuper_loss: 0.0 average reward score: -3.1640625 ------------------------------------------------------------------------------------- |E2E latency=13.90s |Gather latency=0.66s (4.76%) |Generate time=8.77s (63.12%) |Training time=2.34s (16.86%) |Others=2.78 (20.02%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.29 epoch: 0|step: 37|ppo_ep: 1|act_loss: 0.02410888671875|cri_loss: 0.034027099609375|unsuper_loss: 0.0 average reward score: -3.671875 ------------------------------------------------------------------------------------- |E2E latency=13.69s |Gather latency=0.66s (4.84%) |Generate time=8.73s (63.78%) |Training time=2.41s (17.57%) |Others=2.55 (18.64%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.29 epoch: 0|step: 38|ppo_ep: 1|act_loss: 0.05804443359375|cri_loss: 0.04437255859375|unsuper_loss: 0.0 average reward score: -3.6171875 ------------------------------------------------------------------------------------- |E2E latency=13.76s |Gather latency=0.68s (4.98%) |Generate time=8.75s (63.62%) |Training time=2.32s (16.82%) |Others=2.69 (19.56%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 [2023-05-17 00:10:38,393] [INFO] [logging.py:96:log_dist] [Rank 0] step=40, skipped=3, lr=[3.5705000000000004e-07, 3.5705000000000004e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 00:10:38,393] [INFO] [timer.py:199:stop] epoch=0/micro_step=40/global_step=40, RunningAvgSamplesPerSec=28.439315402262512, CurrSamplesPerSec=29.81294470562495, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 00:10:39,139] [INFO] [logging.py:96:log_dist] [Rank 0] step=40, skipped=3, lr=[1.85e-06, 1.85e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 39|ppo_ep: 1|act_loss: -0.0780029296875|cri_loss: 0.032318115234375|unsuper_loss: 0.0 average reward score: -3.45703125 ------------------------------------------------------------------------------------- |E2E latency=13.69s |Gather latency=0.72s (5.24%) |Generate time=8.72s (63.68%) |Training time=2.32s (16.97%) |Others=2.65 (19.35%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 40|ppo_ep: 1|act_loss: 0.0011692047119140625|cri_loss: 0.0557861328125|unsuper_loss: 0.0 average reward score: -3.228515625 ------------------------------------------------------------------------------------- |E2E latency=13.48s |Gather latency=0.69s (5.13%) |Generate time=8.66s (64.23%) |Training time=2.24s (16.64%) |Others=2.58 (19.14%)|CurSamplesPerSec=2.37 |AvgSamplesPerSec=2.30 epoch: 0|step: 41|ppo_ep: 1|act_loss: 0.0174713134765625|cri_loss: 0.032470703125|unsuper_loss: 0.0 average reward score: -3.087890625 ------------------------------------------------------------------------------------- |E2E latency=13.66s |Gather latency=1.02s (7.47%) |Generate time=8.86s (64.89%) |Training time=2.29s (16.73%) |Others=2.51 (18.38%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 42|ppo_ep: 1|act_loss: -0.0200042724609375|cri_loss: 0.032806396484375|unsuper_loss: 0.0 average reward score: -3.6328125 ------------------------------------------------------------------------------------- |E2E latency=13.84s |Gather latency=0.82s (5.94%) |Generate time=8.77s (63.39%) |Training time=2.32s (16.77%) |Others=2.75 (19.84%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.30 epoch: 0|step: 43|ppo_ep: 1|act_loss: 0.005084991455078125|cri_loss: 0.03131103515625|unsuper_loss: 0.0 average reward score: -3.591796875 ------------------------------------------------------------------------------------- |E2E latency=13.75s |Gather latency=0.72s (5.21%) |Generate time=8.76s (63.69%) |Training time=2.39s (17.41%) |Others=2.60 (18.90%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 44|ppo_ep: 1|act_loss: -0.045013427734375|cri_loss: 0.039764404296875|unsuper_loss: 0.0 average reward score: -2.7265625 ------------------------------------------------------------------------------------- |E2E latency=13.73s |Gather latency=0.72s (5.23%) |Generate time=8.81s (64.12%) |Training time=2.33s (16.96%) |Others=2.60 (18.92%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 45|ppo_ep: 1|act_loss: 0.048248291015625|cri_loss: 0.05364990234375|unsuper_loss: 0.0 average reward score: -3.3515625 ------------------------------------------------------------------------------------- |E2E latency=13.83s |Gather latency=0.70s (5.05%) |Generate time=8.90s (64.36%) |Training time=2.31s (16.71%) |Others=2.62 (18.93%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.30 epoch: 0|step: 46|ppo_ep: 1|act_loss: -0.0933837890625|cri_loss: 0.032135009765625|unsuper_loss: 0.0 average reward score: -2.9453125 ------------------------------------------------------------------------------------- |E2E latency=13.69s |Gather latency=1.03s (7.49%) |Generate time=8.72s (63.74%) |Training time=2.36s (17.27%) |Others=2.60 (18.99%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 47|ppo_ep: 1|act_loss: -0.0679931640625|cri_loss: 0.034515380859375|unsuper_loss: 0.0 average reward score: -3.330078125 ------------------------------------------------------------------------------------- |E2E latency=13.71s |Gather latency=0.77s (5.58%) |Generate time=8.70s (63.47%) |Training time=2.35s (17.11%) |Others=2.66 (19.43%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 48|ppo_ep: 1|act_loss: -0.0279083251953125|cri_loss: 0.039093017578125|unsuper_loss: 0.0 average reward score: -2.896484375 ------------------------------------------------------------------------------------- |E2E latency=14.16s |Gather latency=1.07s (7.58%) |Generate time=9.21s (65.06%) |Training time=2.40s (16.93%) |Others=2.55 (18.02%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.30 [2023-05-17 00:12:56,109] [INFO] [logging.py:96:log_dist] [Rank 0] step=50, skipped=3, lr=[4.5355000000000003e-07, 4.5355000000000003e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 00:12:56,110] [INFO] [timer.py:199:stop] epoch=0/micro_step=50/global_step=50, RunningAvgSamplesPerSec=28.57613956457749, CurrSamplesPerSec=28.020876344673052, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 00:12:56,911] [INFO] [logging.py:96:log_dist] [Rank 0] step=50, skipped=3, lr=[2.35e-06, 2.35e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 49|ppo_ep: 1|act_loss: 0.042816162109375|cri_loss: 0.0439453125|unsuper_loss: 0.0 average reward score: -3.326171875 ------------------------------------------------------------------------------------- |E2E latency=13.92s |Gather latency=0.66s (4.76%) |Generate time=8.78s (63.11%) |Training time=2.39s (17.18%) |Others=2.74 (19.71%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.30 epoch: 0|step: 50|ppo_ep: 1|act_loss: -0.031280517578125|cri_loss: 0.050048828125|unsuper_loss: 0.0 average reward score: -3.158203125 ------------------------------------------------------------------------------------- |E2E latency=14.17s |Gather latency=0.84s (5.91%) |Generate time=9.05s (63.82%) |Training time=2.42s (17.10%) |Others=2.70 (19.07%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.30 epoch: 0|step: 51|ppo_ep: 1|act_loss: 0.060546875|cri_loss: 0.07574462890625|unsuper_loss: 0.0 average reward score: -3.73046875 ------------------------------------------------------------------------------------- |E2E latency=14.30s |Gather latency=0.95s (6.61%) |Generate time=9.12s (63.76%) |Training time=2.51s (17.58%) |Others=2.67 (18.66%)|CurSamplesPerSec=2.24 |AvgSamplesPerSec=2.30 epoch: 0|step: 52|ppo_ep: 1|act_loss: -0.0143280029296875|cri_loss: 0.0259552001953125|unsuper_loss: 0.0 average reward score: -3.15234375 ------------------------------------------------------------------------------------- |E2E latency=14.29s |Gather latency=0.75s (5.23%) |Generate time=9.13s (63.88%) |Training time=2.33s (16.32%) |Others=2.83 (19.80%)|CurSamplesPerSec=2.24 |AvgSamplesPerSec=2.30 epoch: 0|step: 53|ppo_ep: 1|act_loss: 0.0037136077880859375|cri_loss: 0.038177490234375|unsuper_loss: 0.0 average reward score: -2.986328125 ------------------------------------------------------------------------------------- |E2E latency=13.74s |Gather latency=0.87s (6.31%) |Generate time=8.67s (63.13%) |Training time=2.35s (17.10%) |Others=2.71 (19.76%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 54|ppo_ep: 1|act_loss: -0.0170745849609375|cri_loss: 0.0797119140625|unsuper_loss: 0.0 average reward score: -3.40625 ------------------------------------------------------------------------------------- |E2E latency=14.17s |Gather latency=1.10s (7.74%) |Generate time=9.17s (64.68%) |Training time=2.35s (16.61%) |Others=2.65 (18.71%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.30 epoch: 0|step: 55|ppo_ep: 1|act_loss: 0.02606201171875|cri_loss: 0.039031982421875|unsuper_loss: 0.0 average reward score: -3.00390625 ------------------------------------------------------------------------------------- |E2E latency=13.75s |Gather latency=0.76s (5.52%) |Generate time=8.64s (62.85%) |Training time=2.33s (16.96%) |Others=2.78 (20.20%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 56|ppo_ep: 1|act_loss: 0.0192718505859375|cri_loss: 0.033294677734375|unsuper_loss: 0.0 average reward score: -3.26953125 ------------------------------------------------------------------------------------- |E2E latency=13.78s |Gather latency=0.88s (6.39%) |Generate time=8.68s (62.97%) |Training time=2.38s (17.24%) |Others=2.73 (19.79%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 57|ppo_ep: 1|act_loss: 0.004711151123046875|cri_loss: 0.05224609375|unsuper_loss: 0.0 average reward score: -3.013671875 ------------------------------------------------------------------------------------- |E2E latency=13.80s |Gather latency=1.19s (8.59%) |Generate time=8.74s (63.32%) |Training time=2.37s (17.19%) |Others=2.69 (19.49%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 58|ppo_ep: 1|act_loss: 0.0274505615234375|cri_loss: 0.03302001953125|unsuper_loss: 0.0 average reward score: -3.16796875 ------------------------------------------------------------------------------------- |E2E latency=13.80s |Gather latency=0.73s (5.31%) |Generate time=8.87s (64.32%) |Training time=2.35s (17.03%) |Others=2.57 (18.66%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 [2023-05-17 00:15:15,640] [INFO] [logging.py:96:log_dist] [Rank 0] step=60, skipped=3, lr=[5.5005e-07, 5.5005e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 00:15:15,641] [INFO] [timer.py:199:stop] epoch=0/micro_step=60/global_step=60, RunningAvgSamplesPerSec=28.5834890078807, CurrSamplesPerSec=28.339748278302643, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 00:15:16,433] [INFO] [logging.py:96:log_dist] [Rank 0] step=60, skipped=3, lr=[2.85e-06, 2.85e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 59|ppo_ep: 1|act_loss: 0.0004177093505859375|cri_loss: 0.0518798828125|unsuper_loss: 0.0 average reward score: -3.13671875 ------------------------------------------------------------------------------------- |E2E latency=13.73s |Gather latency=0.91s (6.65%) |Generate time=8.65s (63.04%) |Training time=2.37s (17.30%) |Others=2.70 (19.66%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 60|ppo_ep: 1|act_loss: -0.025360107421875|cri_loss: 0.0677490234375|unsuper_loss: 0.0 average reward score: -3.224609375 ------------------------------------------------------------------------------------- |E2E latency=13.92s |Gather latency=0.77s (5.56%) |Generate time=8.71s (62.53%) |Training time=2.37s (17.03%) |Others=2.84 (20.43%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.30 epoch: 0|step: 61|ppo_ep: 1|act_loss: -0.032318115234375|cri_loss: 0.044219970703125|unsuper_loss: 0.0 average reward score: -2.990234375 ------------------------------------------------------------------------------------- |E2E latency=13.58s |Gather latency=0.89s (6.54%) |Generate time=8.60s (63.34%) |Training time=2.34s (17.23%) |Others=2.64 (19.44%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.30 epoch: 0|step: 62|ppo_ep: 1|act_loss: -0.0167694091796875|cri_loss: 0.041229248046875|unsuper_loss: 0.0 average reward score: -3.2734375 ------------------------------------------------------------------------------------- |E2E latency=12.78s |Gather latency=0.88s (6.85%) |Generate time=7.69s (60.20%) |Training time=2.46s (19.26%) |Others=2.63 (20.54%)|CurSamplesPerSec=2.50 |AvgSamplesPerSec=2.30 epoch: 0|step: 63|ppo_ep: 1|act_loss: 0.13037109375|cri_loss: 0.034942626953125|unsuper_loss: 0.0 average reward score: -3.31640625 ------------------------------------------------------------------------------------- |E2E latency=14.39s |Gather latency=0.81s (5.63%) |Generate time=9.15s (63.54%) |Training time=2.33s (16.22%) |Others=2.91 (20.25%)|CurSamplesPerSec=2.22 |AvgSamplesPerSec=2.30 epoch: 0|step: 64|ppo_ep: 1|act_loss: -0.034393310546875|cri_loss: 0.060150146484375|unsuper_loss: 0.0 average reward score: -1.86328125 ------------------------------------------------------------------------------------- |E2E latency=13.72s |Gather latency=0.83s (6.04%) |Generate time=8.63s (62.93%) |Training time=2.37s (17.28%) |Others=2.71 (19.79%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 65|ppo_ep: 1|act_loss: 0.0313720703125|cri_loss: 0.027740478515625|unsuper_loss: 0.0 average reward score: -2.77734375 ------------------------------------------------------------------------------------- |E2E latency=13.71s |Gather latency=0.81s (5.94%) |Generate time=8.64s (63.03%) |Training time=2.40s (17.47%) |Others=2.67 (19.50%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 66|ppo_ep: 1|act_loss: 0.0217742919921875|cri_loss: 0.055694580078125|unsuper_loss: 0.0 average reward score: -3.109375 ------------------------------------------------------------------------------------- |E2E latency=14.17s |Gather latency=0.68s (4.80%) |Generate time=9.17s (64.68%) |Training time=2.41s (17.02%) |Others=2.59 (18.30%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.30 epoch: 0|step: 67|ppo_ep: 1|act_loss: 0.0286102294921875|cri_loss: 0.031219482421875|unsuper_loss: 0.0 average reward score: -2.693359375 ------------------------------------------------------------------------------------- |E2E latency=14.38s |Gather latency=0.83s (5.79%) |Generate time=9.16s (63.71%) |Training time=2.43s (16.91%) |Others=2.79 (19.38%)|CurSamplesPerSec=2.23 |AvgSamplesPerSec=2.30 epoch: 0|step: 68|ppo_ep: 1|act_loss: 0.056884765625|cri_loss: 0.03564453125|unsuper_loss: 0.0 average reward score: -3.5078125 ------------------------------------------------------------------------------------- |E2E latency=13.93s |Gather latency=0.78s (5.60%) |Generate time=8.63s (61.97%) |Training time=2.40s (17.20%) |Others=2.90 (20.83%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.30 [2023-05-17 00:17:33,885] [INFO] [logging.py:96:log_dist] [Rank 0] step=70, skipped=3, lr=[6.465500000000001e-07, 6.465500000000001e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 00:17:33,885] [INFO] [timer.py:199:stop] epoch=0/micro_step=70/global_step=70, RunningAvgSamplesPerSec=28.50470516460453, CurrSamplesPerSec=27.150805839524732, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 00:17:34,596] [INFO] [logging.py:96:log_dist] [Rank 0] step=70, skipped=3, lr=[3.3500000000000005e-06, 3.3500000000000005e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 69|ppo_ep: 1|act_loss: -0.07122802734375|cri_loss: 0.069091796875|unsuper_loss: 0.0 average reward score: -2.91015625 ------------------------------------------------------------------------------------- |E2E latency=13.57s |Gather latency=0.69s (5.08%) |Generate time=8.67s (63.91%) |Training time=2.44s (17.98%) |Others=2.46 (18.11%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.30 epoch: 0|step: 70|ppo_ep: 1|act_loss: 0.01093292236328125|cri_loss: 0.0240631103515625|unsuper_loss: 0.0 average reward score: -2.75 ------------------------------------------------------------------------------------- |E2E latency=14.19s |Gather latency=0.84s (5.90%) |Generate time=9.07s (63.90%) |Training time=2.41s (17.00%) |Others=2.71 (19.09%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.30 epoch: 0|step: 71|ppo_ep: 1|act_loss: 0.01274871826171875|cri_loss: 0.0267486572265625|unsuper_loss: 0.0 average reward score: -2.91796875 ------------------------------------------------------------------------------------- |E2E latency=13.91s |Gather latency=0.89s (6.40%) |Generate time=8.93s (64.20%) |Training time=2.29s (16.46%) |Others=2.69 (19.34%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.30 epoch: 0|step: 72|ppo_ep: 1|act_loss: -0.10198974609375|cri_loss: 0.0307159423828125|unsuper_loss: 0.0 average reward score: -2.96875 ------------------------------------------------------------------------------------- |E2E latency=13.70s |Gather latency=0.76s (5.56%) |Generate time=8.65s (63.18%) |Training time=2.37s (17.33%) |Others=2.67 (19.49%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 73|ppo_ep: 1|act_loss: -0.031219482421875|cri_loss: 0.02191162109375|unsuper_loss: 0.0 average reward score: -2.388671875 ------------------------------------------------------------------------------------- |E2E latency=13.85s |Gather latency=0.96s (6.95%) |Generate time=8.66s (62.55%) |Training time=2.39s (17.23%) |Others=2.80 (20.21%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.30 epoch: 0|step: 74|ppo_ep: 1|act_loss: -0.11724853515625|cri_loss: 0.0279388427734375|unsuper_loss: 0.0 average reward score: -2.4375 ------------------------------------------------------------------------------------- |E2E latency=13.60s |Gather latency=0.66s (4.85%) |Generate time=8.68s (63.81%) |Training time=2.34s (17.19%) |Others=2.59 (19.00%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 75|ppo_ep: 1|act_loss: 0.028411865234375|cri_loss: 0.0175933837890625|unsuper_loss: 0.0 average reward score: -3.009765625 ------------------------------------------------------------------------------------- |E2E latency=13.66s |Gather latency=0.94s (6.89%) |Generate time=8.65s (63.32%) |Training time=2.39s (17.46%) |Others=2.63 (19.22%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 [2023-05-17 00:19:10,279] [INFO] [loss_scaler.py:181:update_scale] [deepspeed] OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 16384, reducing to 8192 epoch: 0|step: 76|ppo_ep: 1|act_loss: 0.056610107421875|cri_loss: 0.03179931640625|unsuper_loss: 0.0 average reward score: -2.1640625 ------------------------------------------------------------------------------------- |E2E latency=13.50s |Gather latency=0.76s (5.60%) |Generate time=8.64s (64.05%) |Training time=2.22s (16.47%) |Others=2.63 (19.49%)|CurSamplesPerSec=2.37 |AvgSamplesPerSec=2.30 epoch: 0|step: 77|ppo_ep: 1|act_loss: -0.00023698806762695312|cri_loss: 0.01910400390625|unsuper_loss: 0.0 average reward score: -2.974609375 ------------------------------------------------------------------------------------- |E2E latency=13.62s |Gather latency=1.06s (7.79%) |Generate time=8.62s (63.26%) |Training time=2.39s (17.53%) |Others=2.62 (19.21%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.31 epoch: 0|step: 78|ppo_ep: 1|act_loss: 0.0084381103515625|cri_loss: 0.0201416015625|unsuper_loss: 0.0 average reward score: -2.8828125 ------------------------------------------------------------------------------------- |E2E latency=13.88s |Gather latency=0.78s (5.61%) |Generate time=8.75s (63.02%) |Training time=2.43s (17.50%) |Others=2.70 (19.48%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.31 [2023-05-17 00:19:51,406] [INFO] [logging.py:96:log_dist] [Rank 0] step=80, skipped=4, lr=[7.334e-07, 7.334e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 00:19:51,406] [INFO] [timer.py:199:stop] epoch=0/micro_step=80/global_step=80, RunningAvgSamplesPerSec=28.561847823037557, CurrSamplesPerSec=28.987151230959718, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 00:19:52,126] [INFO] [logging.py:96:log_dist] [Rank 0] step=80, skipped=3, lr=[3.85e-06, 3.85e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 79|ppo_ep: 1|act_loss: 0.046295166015625|cri_loss: 0.034637451171875|unsuper_loss: 0.0 average reward score: -3.501953125 ------------------------------------------------------------------------------------- |E2E latency=13.62s |Gather latency=0.66s (4.85%) |Generate time=8.69s (63.81%) |Training time=2.31s (16.96%) |Others=2.62 (19.24%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.31 epoch: 0|step: 80|ppo_ep: 1|act_loss: -0.00962066650390625|cri_loss: 0.0190582275390625|unsuper_loss: 0.0 average reward score: -2.8671875 ------------------------------------------------------------------------------------- |E2E latency=13.86s |Gather latency=0.83s (5.98%) |Generate time=8.93s (64.42%) |Training time=2.35s (16.93%) |Others=2.59 (18.65%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.31 epoch: 0|step: 81|ppo_ep: 1|act_loss: 0.0120391845703125|cri_loss: 0.01227569580078125|unsuper_loss: 0.0 average reward score: -2.806640625 ------------------------------------------------------------------------------------- |E2E latency=13.91s |Gather latency=0.76s (5.50%) |Generate time=8.70s (62.51%) |Training time=2.34s (16.85%) |Others=2.87 (20.63%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.31 epoch: 0|step: 82|ppo_ep: 1|act_loss: 0.048736572265625|cri_loss: 0.0262451171875|unsuper_loss: 0.0 average reward score: -3.107421875 ------------------------------------------------------------------------------------- |E2E latency=13.54s |Gather latency=0.91s (6.71%) |Generate time=8.70s (64.20%) |Training time=2.35s (17.38%) |Others=2.49 (18.42%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.31 epoch: 0|step: 83|ppo_ep: 1|act_loss: 0.0019426345825195312|cri_loss: 0.023101806640625|unsuper_loss: 0.0 average reward score: -3.13671875 ------------------------------------------------------------------------------------- |E2E latency=13.63s |Gather latency=0.88s (6.48%) |Generate time=8.64s (63.40%) |Training time=2.37s (17.37%) |Others=2.62 (19.23%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.31 epoch: 0|step: 84|ppo_ep: 1|act_loss: 0.057830810546875|cri_loss: 0.099609375|unsuper_loss: 0.0 average reward score: -2.6484375 ------------------------------------------------------------------------------------- |E2E latency=14.03s |Gather latency=0.87s (6.18%) |Generate time=8.86s (63.13%) |Training time=2.44s (17.37%) |Others=2.73 (19.50%)|CurSamplesPerSec=2.28 |AvgSamplesPerSec=2.31 epoch: 0|step: 85|ppo_ep: 1|act_loss: -0.08447265625|cri_loss: 0.0428466796875|unsuper_loss: 0.0 average reward score: -2.7578125 ------------------------------------------------------------------------------------- |E2E latency=13.72s |Gather latency=0.99s (7.22%) |Generate time=8.75s (63.81%) |Training time=2.33s (16.95%) |Others=2.64 (19.24%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.31 epoch: 0|step: 86|ppo_ep: 1|act_loss: -0.045166015625|cri_loss: 0.044189453125|unsuper_loss: 0.0 average reward score: -2.96484375 ------------------------------------------------------------------------------------- |E2E latency=13.79s |Gather latency=1.14s (8.30%) |Generate time=8.75s (63.49%) |Training time=2.35s (17.04%) |Others=2.68 (19.47%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.31 epoch: 0|step: 87|ppo_ep: 1|act_loss: 0.0028553009033203125|cri_loss: 0.0166015625|unsuper_loss: 0.0 average reward score: -3.873046875 ------------------------------------------------------------------------------------- |E2E latency=13.54s |Gather latency=0.72s (5.35%) |Generate time=8.67s (64.08%) |Training time=2.33s (17.24%) |Others=2.53 (18.68%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.31 epoch: 0|step: 88|ppo_ep: 1|act_loss: -0.029266357421875|cri_loss: 0.0302581787109375|unsuper_loss: 0.0 average reward score: -3.05078125 ------------------------------------------------------------------------------------- |E2E latency=13.90s |Gather latency=1.25s (8.96%) |Generate time=8.77s (63.06%) |Training time=2.46s (17.70%) |Others=2.67 (19.23%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.31 [2023-05-17 00:22:09,270] [INFO] [logging.py:96:log_dist] [Rank 0] step=90, skipped=4, lr=[8.299000000000001e-07, 8.299000000000001e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 00:22:09,271] [INFO] [timer.py:199:stop] epoch=0/micro_step=90/global_step=90, RunningAvgSamplesPerSec=28.504641604602053, CurrSamplesPerSec=27.989503028806613, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 00:22:10,055] [INFO] [logging.py:96:log_dist] [Rank 0] step=90, skipped=3, lr=[4.350000000000001e-06, 4.350000000000001e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 89|ppo_ep: 1|act_loss: 0.049896240234375|cri_loss: 0.03021240234375|unsuper_loss: 0.0 average reward score: -2.921875 ------------------------------------------------------------------------------------- |E2E latency=14.00s |Gather latency=0.93s (6.61%) |Generate time=8.72s (62.27%) |Training time=2.41s (17.23%) |Others=2.87 (20.50%)|CurSamplesPerSec=2.28 |AvgSamplesPerSec=2.31 epoch: 0|step: 90|ppo_ep: 1|act_loss: 0.031524658203125|cri_loss: 0.015228271484375|unsuper_loss: 0.0 average reward score: -2.60546875 ------------------------------------------------------------------------------------- |E2E latency=13.79s |Gather latency=0.76s (5.55%) |Generate time=8.75s (63.47%) |Training time=2.43s (17.64%) |Others=2.61 (18.90%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.31 epoch: 0|step: 91|ppo_ep: 1|act_loss: 0.031280517578125|cri_loss: 0.0199127197265625|unsuper_loss: 0.0 average reward score: -2.626953125 ------------------------------------------------------------------------------------- |E2E latency=13.85s |Gather latency=1.11s (8.00%) |Generate time=8.78s (63.42%) |Training time=2.36s (17.04%) |Others=2.71 (19.54%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.31 epoch: 0|step: 92|ppo_ep: 1|act_loss: 0.0065460205078125|cri_loss: 0.0626220703125|unsuper_loss: 0.0 average reward score: -3.12109375 ------------------------------------------------------------------------------------- |E2E latency=14.52s |Gather latency=0.75s (5.17%) |Generate time=9.29s (63.97%) |Training time=2.54s (17.48%) |Others=2.69 (18.55%)|CurSamplesPerSec=2.20 |AvgSamplesPerSec=2.31 epoch: 0|step: 93|ppo_ep: 1|act_loss: 0.0992431640625|cri_loss: 0.03753662109375|unsuper_loss: 0.0 average reward score: -2.763671875 ------------------------------------------------------------------------------------- |E2E latency=13.83s |Gather latency=0.87s (6.29%) |Generate time=8.73s (63.12%) |Training time=2.40s (17.36%) |Others=2.70 (19.52%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.31 epoch: 0|step: 94|ppo_ep: 1|act_loss: 0.0163421630859375|cri_loss: 0.041229248046875|unsuper_loss: 0.0 average reward score: -3.5546875 ------------------------------------------------------------------------------------- |E2E latency=14.38s |Gather latency=0.81s (5.66%) |Generate time=9.17s (63.74%) |Training time=2.41s (16.77%) |Others=2.80 (19.48%)|CurSamplesPerSec=2.22 |AvgSamplesPerSec=2.31 epoch: 0|step: 95|ppo_ep: 1|act_loss: -0.00975799560546875|cri_loss: 0.04052734375|unsuper_loss: 0.0 average reward score: -3.24609375 ------------------------------------------------------------------------------------- |E2E latency=13.66s |Gather latency=1.05s (7.69%) |Generate time=8.62s (63.08%) |Training time=2.38s (17.42%) |Others=2.66 (19.51%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.31 epoch: 0|step: 96|ppo_ep: 1|act_loss: -0.1026611328125|cri_loss: 0.04351806640625|unsuper_loss: 0.0 average reward score: -2.763671875 ------------------------------------------------------------------------------------- |E2E latency=13.90s |Gather latency=0.93s (6.70%) |Generate time=8.74s (62.85%) |Training time=2.46s (17.73%) |Others=2.70 (19.41%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.31 epoch: 0|step: 97|ppo_ep: 1|act_loss: -0.1976318359375|cri_loss: 0.152587890625|unsuper_loss: 0.0 average reward score: -2.796875 ------------------------------------------------------------------------------------- |E2E latency=13.71s |Gather latency=0.61s (4.46%) |Generate time=8.84s (64.44%) |Training time=2.34s (17.08%) |Others=2.53 (18.48%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.31 epoch: 0|step: 98|ppo_ep: 1|act_loss: -0.10931396484375|cri_loss: 0.0552978515625|unsuper_loss: 0.0 average reward score: -3.30078125 ------------------------------------------------------------------------------------- |E2E latency=13.63s |Gather latency=1.15s (8.42%) |Generate time=8.71s (63.90%) |Training time=2.23s (16.40%) |Others=2.68 (19.70%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.31 [2023-05-17 00:24:28,161] [INFO] [logging.py:96:log_dist] [Rank 0] step=100, skipped=4, lr=[9.264e-07, 9.264e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 00:24:28,162] [INFO] [timer.py:199:stop] epoch=0/micro_step=100/global_step=100, RunningAvgSamplesPerSec=28.46031844066052, CurrSamplesPerSec=29.724201162412665, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 00:24:28,937] [INFO] [logging.py:96:log_dist] [Rank 0] step=100, skipped=3, lr=[4.85e-06, 4.85e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 99|ppo_ep: 1|act_loss: -0.021942138671875|cri_loss: 0.037628173828125|unsuper_loss: 0.0 average reward score: -2.537109375 ------------------------------------------------------------------------------------- |E2E latency=13.61s |Gather latency=0.47s (3.42%) |Generate time=8.64s (63.49%) |Training time=2.33s (17.15%) |Others=2.63 (19.36%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.31 epoch: 0|step: 100|ppo_ep: 1|act_loss: -0.08062744140625|cri_loss: 0.10595703125|unsuper_loss: 0.0 average reward score: -3.1015625 ------------------------------------------------------------------------------------- |E2E latency=14.32s |Gather latency=0.62s (4.35%) |Generate time=9.17s (64.00%) |Training time=2.39s (16.66%) |Others=2.77 (19.34%)|CurSamplesPerSec=2.23 |AvgSamplesPerSec=2.31 epoch: 0|step: 101|ppo_ep: 1|act_loss: -0.052032470703125|cri_loss: 0.042510986328125|unsuper_loss: 0.0 average reward score: -2.6015625 ------------------------------------------------------------------------------------- |E2E latency=13.86s |Gather latency=0.70s (5.07%) |Generate time=8.69s (62.69%) |Training time=2.46s (17.78%) |Others=2.71 (19.53%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.31 epoch: 0|step: 102|ppo_ep: 1|act_loss: -0.0160369873046875|cri_loss: 0.02655029296875|unsuper_loss: 0.0 average reward score: -3.46875 ------------------------------------------------------------------------------------- |E2E latency=14.95s |Gather latency=0.85s (5.71%) |Generate time=8.93s (59.73%) |Training time=3.30s (22.06%) |Others=2.72 (18.21%)|CurSamplesPerSec=2.14 |AvgSamplesPerSec=2.30 epoch: 0|step: 103|ppo_ep: 1|act_loss: -0.066650390625|cri_loss: 0.0227813720703125|unsuper_loss: 0.0 average reward score: -2.486328125 ------------------------------------------------------------------------------------- |E2E latency=14.66s |Gather latency=0.68s (4.63%) |Generate time=9.53s (64.97%) |Training time=2.41s (16.45%) |Others=2.72 (18.58%)|CurSamplesPerSec=2.18 |AvgSamplesPerSec=2.30 epoch: 0|step: 104|ppo_ep: 1|act_loss: -0.005970001220703125|cri_loss: 0.05401611328125|unsuper_loss: 0.0 average reward score: -3.08984375 ------------------------------------------------------------------------------------- |E2E latency=13.95s |Gather latency=0.94s (6.72%) |Generate time=8.73s (62.60%) |Training time=2.43s (17.39%) |Others=2.79 (20.01%)|CurSamplesPerSec=2.29 |AvgSamplesPerSec=2.30 epoch: 0|step: 105|ppo_ep: 1|act_loss: 0.047821044921875|cri_loss: 0.0253448486328125|unsuper_loss: 0.0 average reward score: -2.14453125 ------------------------------------------------------------------------------------- |E2E latency=13.63s |Gather latency=0.67s (4.91%) |Generate time=8.66s (63.53%) |Training time=2.37s (17.35%) |Others=2.61 (19.12%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 106|ppo_ep: 1|act_loss: 0.037109375|cri_loss: 0.060089111328125|unsuper_loss: 0.0 average reward score: -2.642578125 ------------------------------------------------------------------------------------- |E2E latency=14.16s |Gather latency=1.45s (10.25%) |Generate time=9.11s (64.32%) |Training time=2.40s (16.97%) |Others=2.65 (18.70%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.30 epoch: 0|step: 107|ppo_ep: 1|act_loss: -0.0391845703125|cri_loss: 0.061920166015625|unsuper_loss: 0.0 average reward score: -2.203125 ------------------------------------------------------------------------------------- |E2E latency=13.82s |Gather latency=1.33s (9.64%) |Generate time=8.64s (62.54%) |Training time=2.42s (17.48%) |Others=2.76 (19.99%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.30 epoch: 0|step: 108|ppo_ep: 1|act_loss: 0.0701904296875|cri_loss: 0.05535888671875|unsuper_loss: 0.0 average reward score: -2.1953125 ------------------------------------------------------------------------------------- |E2E latency=14.09s |Gather latency=0.62s (4.43%) |Generate time=9.13s (64.78%) |Training time=2.33s (16.55%) |Others=2.63 (18.67%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.30 [2023-05-17 00:26:49,502] [INFO] [logging.py:96:log_dist] [Rank 0] step=110, skipped=4, lr=[9.648824733938164e-07, 9.648824733938164e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 00:26:49,503] [INFO] [timer.py:199:stop] epoch=0/micro_step=110/global_step=110, RunningAvgSamplesPerSec=28.23216623643632, CurrSamplesPerSec=27.771129082484048, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 00:26:50,292] [INFO] [logging.py:96:log_dist] [Rank 0] step=110, skipped=3, lr=[4.999171168785783e-06, 4.999171168785783e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 109|ppo_ep: 1|act_loss: 0.0946044921875|cri_loss: 0.033782958984375|unsuper_loss: 0.0 average reward score: -2.994140625 ------------------------------------------------------------------------------------- |E2E latency=13.90s |Gather latency=0.88s (6.32%) |Generate time=8.71s (62.67%) |Training time=2.36s (17.00%) |Others=2.82 (20.32%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.30 epoch: 0|step: 110|ppo_ep: 1|act_loss: 0.060760498046875|cri_loss: 0.04107666015625|unsuper_loss: 0.0 average reward score: -3.130859375 ------------------------------------------------------------------------------------- |E2E latency=12.61s |Gather latency=0.62s (4.93%) |Generate time=7.74s (61.37%) |Training time=2.35s (18.66%) |Others=2.52 (19.98%)|CurSamplesPerSec=2.54 |AvgSamplesPerSec=2.30 [2023-05-17 00:27:16,602] [INFO] [loss_scaler.py:188:update_scale] [deepspeed] OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 32768, but hysteresis is 2. Reducing hysteresis to 1 epoch: 0|step: 111|ppo_ep: 1|act_loss: 0.033599853515625|cri_loss: 0.043701171875|unsuper_loss: 0.0 average reward score: -3.19921875 ------------------------------------------------------------------------------------- |E2E latency=13.70s |Gather latency=0.75s (5.46%) |Generate time=8.66s (63.22%) |Training time=2.34s (17.04%) |Others=2.70 (19.74%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.31 epoch: 0|step: 112|ppo_ep: 1|act_loss: 0.0860595703125|cri_loss: 0.05364990234375|unsuper_loss: 0.0 average reward score: -3.34765625 ------------------------------------------------------------------------------------- |E2E latency=14.04s |Gather latency=0.84s (5.95%) |Generate time=8.62s (61.45%) |Training time=2.52s (17.98%) |Others=2.89 (20.58%)|CurSamplesPerSec=2.28 |AvgSamplesPerSec=2.30 epoch: 0|step: 113|ppo_ep: 1|act_loss: 0.0267486572265625|cri_loss: 0.0164642333984375|unsuper_loss: 0.0 average reward score: -2.63671875 ------------------------------------------------------------------------------------- |E2E latency=14.08s |Gather latency=1.01s (7.18%) |Generate time=9.10s (64.68%) |Training time=2.38s (16.92%) |Others=2.59 (18.41%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.30 epoch: 0|step: 114|ppo_ep: 1|act_loss: -0.0474853515625|cri_loss: 0.0110321044921875|unsuper_loss: 0.0 average reward score: -2.734375 ------------------------------------------------------------------------------------- |E2E latency=13.99s |Gather latency=0.78s (5.61%) |Generate time=8.79s (62.83%) |Training time=2.45s (17.52%) |Others=2.75 (19.66%)|CurSamplesPerSec=2.29 |AvgSamplesPerSec=2.30 epoch: 0|step: 115|ppo_ep: 1|act_loss: -0.10687255859375|cri_loss: 0.07830810546875|unsuper_loss: 0.0 average reward score: -2.40625 ------------------------------------------------------------------------------------- |E2E latency=13.83s |Gather latency=0.74s (5.33%) |Generate time=8.68s (62.76%) |Training time=2.50s (18.11%) |Others=2.64 (19.13%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.30 epoch: 0|step: 116|ppo_ep: 1|act_loss: -0.056060791015625|cri_loss: 0.0197906494140625|unsuper_loss: 0.0 average reward score: -3.091796875 ------------------------------------------------------------------------------------- |E2E latency=13.60s |Gather latency=0.63s (4.67%) |Generate time=8.61s (63.29%) |Training time=2.36s (17.33%) |Others=2.64 (19.38%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.31 epoch: 0|step: 117|ppo_ep: 1|act_loss: -0.024505615234375|cri_loss: 0.048492431640625|unsuper_loss: 0.0 average reward score: -2.484375 ------------------------------------------------------------------------------------- |E2E latency=14.30s |Gather latency=0.97s (6.76%) |Generate time=9.16s (64.05%) |Training time=2.38s (16.62%) |Others=2.76 (19.33%)|CurSamplesPerSec=2.24 |AvgSamplesPerSec=2.30 epoch: 0|step: 118|ppo_ep: 1|act_loss: -0.01166534423828125|cri_loss: 0.018157958984375|unsuper_loss: 0.0 average reward score: -2.478515625 ------------------------------------------------------------------------------------- |E2E latency=13.79s |Gather latency=0.73s (5.30%) |Generate time=8.76s (63.48%) |Training time=2.40s (17.40%) |Others=2.64 (19.12%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 [2023-05-17 00:29:07,680] [INFO] [logging.py:96:log_dist] [Rank 0] step=120, skipped=4, lr=[9.641644625736652e-07, 9.641644625736652e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 00:29:07,681] [INFO] [timer.py:199:stop] epoch=0/micro_step=120/global_step=120, RunningAvgSamplesPerSec=28.229893316074133, CurrSamplesPerSec=29.8086343008089, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 00:29:08,435] [INFO] [logging.py:96:log_dist] [Rank 0] step=120, skipped=4, lr=[4.995670790537125e-06, 4.995670790537125e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 119|ppo_ep: 1|act_loss: -0.095703125|cri_loss: 0.04681396484375|unsuper_loss: 0.0 average reward score: -2.7578125 ------------------------------------------------------------------------------------- |E2E latency=14.22s |Gather latency=0.77s (5.43%) |Generate time=9.16s (64.45%) |Training time=2.35s (16.55%) |Others=2.70 (19.00%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.30 [2023-05-17 00:29:22,075] [INFO] [loss_scaler.py:181:update_scale] [deepspeed] OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 32768, reducing to 16384 epoch: 0|step: 120|ppo_ep: 1|act_loss: -0.0167694091796875|cri_loss: 0.0161590576171875|unsuper_loss: 0.0 average reward score: -2.5390625 ------------------------------------------------------------------------------------- |E2E latency=13.64s |Gather latency=0.60s (4.43%) |Generate time=8.68s (63.68%) |Training time=2.32s (16.99%) |Others=2.64 (19.33%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 121|ppo_ep: 1|act_loss: -0.0855712890625|cri_loss: 0.03173828125|unsuper_loss: 0.0 average reward score: -3.1328125 ------------------------------------------------------------------------------------- |E2E latency=14.28s |Gather latency=0.58s (4.10%) |Generate time=9.17s (64.18%) |Training time=2.55s (17.86%) |Others=2.56 (17.96%)|CurSamplesPerSec=2.24 |AvgSamplesPerSec=2.30 epoch: 0|step: 122|ppo_ep: 1|act_loss: -0.0225372314453125|cri_loss: 0.019256591796875|unsuper_loss: 0.0 average reward score: -3.51953125 ------------------------------------------------------------------------------------- |E2E latency=14.35s |Gather latency=0.95s (6.63%) |Generate time=9.12s (63.60%) |Training time=2.39s (16.63%) |Others=2.84 (19.78%)|CurSamplesPerSec=2.23 |AvgSamplesPerSec=2.30 epoch: 0|step: 123|ppo_ep: 1|act_loss: -0.024200439453125|cri_loss: 0.03704833984375|unsuper_loss: 0.0 average reward score: -3.3203125 ------------------------------------------------------------------------------------- |E2E latency=14.13s |Gather latency=0.66s (4.69%) |Generate time=9.21s (65.16%) |Training time=2.38s (16.84%) |Others=2.54 (18.00%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.30 epoch: 0|step: 124|ppo_ep: 1|act_loss: 0.06573486328125|cri_loss: 0.053619384765625|unsuper_loss: 0.0 average reward score: -3.14453125 ------------------------------------------------------------------------------------- |E2E latency=14.27s |Gather latency=0.94s (6.59%) |Generate time=9.13s (63.98%) |Training time=2.37s (16.60%) |Others=2.77 (19.42%)|CurSamplesPerSec=2.24 |AvgSamplesPerSec=2.30 epoch: 0|step: 125|ppo_ep: 1|act_loss: 0.048980712890625|cri_loss: 0.03424072265625|unsuper_loss: 0.0 average reward score: -2.669921875 ------------------------------------------------------------------------------------- |E2E latency=13.60s |Gather latency=0.58s (4.27%) |Generate time=8.74s (64.28%) |Training time=2.30s (16.91%) |Others=2.56 (18.82%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 126|ppo_ep: 1|act_loss: 0.0931396484375|cri_loss: 0.044708251953125|unsuper_loss: 0.0 average reward score: -3.03125 ------------------------------------------------------------------------------------- |E2E latency=13.63s |Gather latency=0.98s (7.23%) |Generate time=8.75s (64.22%) |Training time=2.29s (16.82%) |Others=2.58 (18.97%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 127|ppo_ep: 1|act_loss: 0.111328125|cri_loss: 0.02459716796875|unsuper_loss: 0.0 average reward score: -2.13671875 ------------------------------------------------------------------------------------- |E2E latency=14.33s |Gather latency=0.74s (5.16%) |Generate time=9.21s (64.28%) |Training time=2.38s (16.64%) |Others=2.74 (19.08%)|CurSamplesPerSec=2.23 |AvgSamplesPerSec=2.30 epoch: 0|step: 128|ppo_ep: 1|act_loss: 0.11962890625|cri_loss: 0.06402587890625|unsuper_loss: 0.0 average reward score: -3.1328125 ------------------------------------------------------------------------------------- |E2E latency=13.68s |Gather latency=0.80s (5.89%) |Generate time=8.76s (64.09%) |Training time=2.33s (17.02%) |Others=2.58 (18.89%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 [2023-05-17 00:31:27,820] [INFO] [logging.py:96:log_dist] [Rank 0] step=130, skipped=4, lr=[9.627947038661453e-07, 9.627947038661453e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 00:31:27,821] [INFO] [timer.py:199:stop] epoch=0/micro_step=130/global_step=130, RunningAvgSamplesPerSec=28.25373535562634, CurrSamplesPerSec=27.122834855069623, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 00:31:28,550] [INFO] [logging.py:96:log_dist] [Rank 0] step=130, skipped=5, lr=[4.989435038710013e-06, 4.989435038710013e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 129|ppo_ep: 1|act_loss: 0.1005859375|cri_loss: 0.04559326171875|unsuper_loss: 0.0 average reward score: -2.91796875 ------------------------------------------------------------------------------------- |E2E latency=14.22s |Gather latency=0.73s (5.14%) |Generate time=9.13s (64.21%) |Training time=2.44s (17.16%) |Others=2.65 (18.64%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.30 epoch: 0|step: 130|ppo_ep: 1|act_loss: 0.044677734375|cri_loss: 0.031951904296875|unsuper_loss: 0.0 average reward score: -3.08203125 ------------------------------------------------------------------------------------- |E2E latency=13.76s |Gather latency=0.76s (5.50%) |Generate time=8.74s (63.52%) |Training time=2.35s (17.05%) |Others=2.67 (19.43%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 131|ppo_ep: 1|act_loss: 0.041961669921875|cri_loss: 0.02716064453125|unsuper_loss: 0.0 average reward score: -3.04296875 ------------------------------------------------------------------------------------- |E2E latency=14.19s |Gather latency=0.93s (6.54%) |Generate time=9.16s (64.56%) |Training time=2.37s (16.67%) |Others=2.66 (18.76%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.30 epoch: 0|step: 132|ppo_ep: 1|act_loss: 0.0029850006103515625|cri_loss: 0.016143798828125|unsuper_loss: 0.0 average reward score: -2.5625 ------------------------------------------------------------------------------------- |E2E latency=14.33s |Gather latency=0.90s (6.30%) |Generate time=9.04s (63.09%) |Training time=2.37s (16.56%) |Others=2.92 (20.36%)|CurSamplesPerSec=2.23 |AvgSamplesPerSec=2.30 epoch: 0|step: 133|ppo_ep: 1|act_loss: -0.059173583984375|cri_loss: 0.0247955322265625|unsuper_loss: 0.0 average reward score: -2.541015625 ------------------------------------------------------------------------------------- |E2E latency=13.67s |Gather latency=0.61s (4.46%) |Generate time=8.73s (63.86%) |Training time=2.36s (17.29%) |Others=2.58 (18.86%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 134|ppo_ep: 1|act_loss: 0.0296478271484375|cri_loss: 0.01483917236328125|unsuper_loss: 0.0 average reward score: -2.275390625 ------------------------------------------------------------------------------------- |E2E latency=13.68s |Gather latency=0.63s (4.62%) |Generate time=8.59s (62.82%) |Training time=2.43s (17.79%) |Others=2.65 (19.39%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 135|ppo_ep: 1|act_loss: 0.042022705078125|cri_loss: 0.0236053466796875|unsuper_loss: 0.0 average reward score: -3.271484375 ------------------------------------------------------------------------------------- |E2E latency=13.62s |Gather latency=0.54s (3.95%) |Generate time=8.64s (63.42%) |Training time=2.39s (17.56%) |Others=2.59 (19.01%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 136|ppo_ep: 1|act_loss: 0.0345458984375|cri_loss: 0.01495361328125|unsuper_loss: 0.0 average reward score: -2.755859375 ------------------------------------------------------------------------------------- |E2E latency=12.70s |Gather latency=0.85s (6.66%) |Generate time=7.71s (60.73%) |Training time=2.27s (17.91%) |Others=2.71 (21.36%)|CurSamplesPerSec=2.52 |AvgSamplesPerSec=2.30 epoch: 0|step: 137|ppo_ep: 1|act_loss: 0.007564544677734375|cri_loss: 0.029541015625|unsuper_loss: 0.0 average reward score: -3.03125 ------------------------------------------------------------------------------------- |E2E latency=14.16s |Gather latency=0.67s (4.73%) |Generate time=9.12s (64.44%) |Training time=2.42s (17.06%) |Others=2.62 (18.50%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.30 epoch: 0|step: 138|ppo_ep: 1|act_loss: -0.021240234375|cri_loss: 0.023681640625|unsuper_loss: 0.0 average reward score: -2.623046875 ------------------------------------------------------------------------------------- |E2E latency=13.99s |Gather latency=0.94s (6.72%) |Generate time=9.00s (64.35%) |Training time=2.40s (17.17%) |Others=2.58 (18.48%)|CurSamplesPerSec=2.29 |AvgSamplesPerSec=2.30 [2023-05-17 00:33:46,115] [INFO] [logging.py:96:log_dist] [Rank 0] step=140, skipped=4, lr=[9.607750507136698e-07, 9.607750507136698e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 00:33:46,115] [INFO] [timer.py:199:stop] epoch=0/micro_step=140/global_step=140, RunningAvgSamplesPerSec=28.295818088245923, CurrSamplesPerSec=29.654862114267054, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 00:33:46,863] [INFO] [logging.py:96:log_dist] [Rank 0] step=140, skipped=5, lr=[4.9793066853402535e-06, 4.9793066853402535e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 139|ppo_ep: 1|act_loss: -0.039215087890625|cri_loss: 0.0374755859375|unsuper_loss: 0.0 average reward score: -2.453125 ------------------------------------------------------------------------------------- |E2E latency=14.21s |Gather latency=0.64s (4.47%) |Generate time=9.17s (64.50%) |Training time=2.34s (16.46%) |Others=2.71 (19.05%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.30 epoch: 0|step: 140|ppo_ep: 1|act_loss: -0.057708740234375|cri_loss: 0.030914306640625|unsuper_loss: 0.0 average reward score: -3.0703125 ------------------------------------------------------------------------------------- |E2E latency=13.89s |Gather latency=0.76s (5.44%) |Generate time=8.87s (63.86%) |Training time=2.33s (16.81%) |Others=2.68 (19.33%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.30 epoch: 0|step: 141|ppo_ep: 1|act_loss: -0.01110076904296875|cri_loss: 0.0322265625|unsuper_loss: 0.0 average reward score: -2.65234375 ------------------------------------------------------------------------------------- |E2E latency=13.53s |Gather latency=0.65s (4.81%) |Generate time=8.73s (64.54%) |Training time=2.26s (16.69%) |Others=2.54 (18.77%)|CurSamplesPerSec=2.37 |AvgSamplesPerSec=2.30 epoch: 0|step: 142|ppo_ep: 1|act_loss: 0.01113128662109375|cri_loss: 0.0133056640625|unsuper_loss: 0.0 average reward score: -2.6875 ------------------------------------------------------------------------------------- |E2E latency=13.70s |Gather latency=0.67s (4.92%) |Generate time=8.69s (63.44%) |Training time=2.36s (17.24%) |Others=2.65 (19.32%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 143|ppo_ep: 1|act_loss: 0.0150299072265625|cri_loss: 0.020599365234375|unsuper_loss: 0.0 average reward score: -3.236328125 ------------------------------------------------------------------------------------- |E2E latency=13.68s |Gather latency=0.70s (5.09%) |Generate time=8.75s (63.93%) |Training time=2.41s (17.62%) |Others=2.52 (18.45%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 144|ppo_ep: 1|act_loss: 0.052642822265625|cri_loss: 0.0174713134765625|unsuper_loss: 0.0 average reward score: -2.73046875 ------------------------------------------------------------------------------------- |E2E latency=14.22s |Gather latency=0.89s (6.26%) |Generate time=9.17s (64.47%) |Training time=2.43s (17.06%) |Others=2.63 (18.47%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.30 epoch: 0|step: 145|ppo_ep: 1|act_loss: 0.011566162109375|cri_loss: 0.0202178955078125|unsuper_loss: 0.0 average reward score: -2.482421875 ------------------------------------------------------------------------------------- |E2E latency=14.34s |Gather latency=0.65s (4.56%) |Generate time=9.15s (63.78%) |Training time=2.41s (16.83%) |Others=2.78 (19.38%)|CurSamplesPerSec=2.23 |AvgSamplesPerSec=2.30 epoch: 0|step: 146|ppo_ep: 1|act_loss: 0.0162506103515625|cri_loss: 0.033966064453125|unsuper_loss: 0.0 average reward score: -3.3359375 ------------------------------------------------------------------------------------- |E2E latency=15.22s |Gather latency=0.65s (4.29%) |Generate time=9.15s (60.11%) |Training time=2.36s (15.48%) |Others=3.72 (24.41%)|CurSamplesPerSec=2.10 |AvgSamplesPerSec=2.30 epoch: 0|step: 147|ppo_ep: 1|act_loss: -0.0389404296875|cri_loss: 0.0155181884765625|unsuper_loss: 0.0 average reward score: -2.376953125 ------------------------------------------------------------------------------------- |E2E latency=14.04s |Gather latency=0.93s (6.63%) |Generate time=8.94s (63.63%) |Training time=2.35s (16.72%) |Others=2.76 (19.64%)|CurSamplesPerSec=2.28 |AvgSamplesPerSec=2.30 epoch: 0|step: 148|ppo_ep: 1|act_loss: 0.0135040283203125|cri_loss: 0.010345458984375|unsuper_loss: 0.0 average reward score: -2.576171875 ------------------------------------------------------------------------------------- |E2E latency=14.14s |Gather latency=0.75s (5.32%) |Generate time=9.15s (64.72%) |Training time=2.37s (16.75%) |Others=2.62 (18.53%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.30 [2023-05-17 00:36:06,746] [INFO] [logging.py:96:log_dist] [Rank 0] step=150, skipped=4, lr=[9.581082359412187e-07, 9.581082359412187e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 00:36:06,746] [INFO] [timer.py:199:stop] epoch=0/micro_step=150/global_step=150, RunningAvgSamplesPerSec=28.29447205628164, CurrSamplesPerSec=28.107488776813504, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 00:36:07,518] [INFO] [logging.py:96:log_dist] [Rank 0] step=150, skipped=5, lr=[4.965823542444037e-06, 4.965823542444037e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 149|ppo_ep: 1|act_loss: 0.005706787109375|cri_loss: 0.01395416259765625|unsuper_loss: 0.0 average reward score: -3.486328125 ------------------------------------------------------------------------------------- |E2E latency=13.88s |Gather latency=0.66s (4.74%) |Generate time=8.81s (63.42%) |Training time=2.33s (16.77%) |Others=2.75 (19.81%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.30 epoch: 0|step: 150|ppo_ep: 1|act_loss: 0.002346038818359375|cri_loss: 0.0187225341796875|unsuper_loss: 0.0 average reward score: -2.9921875 ------------------------------------------------------------------------------------- |E2E latency=14.22s |Gather latency=0.75s (5.30%) |Generate time=9.16s (64.41%) |Training time=2.33s (16.41%) |Others=2.73 (19.18%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.30 epoch: 0|step: 151|ppo_ep: 1|act_loss: -0.0003383159637451172|cri_loss: 0.012847900390625|unsuper_loss: 0.0 average reward score: -2.74609375 ------------------------------------------------------------------------------------- |E2E latency=14.15s |Gather latency=0.63s (4.48%) |Generate time=9.16s (64.74%) |Training time=2.32s (16.42%) |Others=2.66 (18.84%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.30 epoch: 0|step: 152|ppo_ep: 1|act_loss: -0.033966064453125|cri_loss: 0.021820068359375|unsuper_loss: 0.0 average reward score: -2.3203125 ------------------------------------------------------------------------------------- |E2E latency=14.36s |Gather latency=0.90s (6.24%) |Generate time=9.11s (63.47%) |Training time=2.46s (17.15%) |Others=2.78 (19.39%)|CurSamplesPerSec=2.23 |AvgSamplesPerSec=2.30 epoch: 0|step: 153|ppo_ep: 1|act_loss: 0.04638671875|cri_loss: 0.040069580078125|unsuper_loss: 0.0 average reward score: -3.1875 ------------------------------------------------------------------------------------- |E2E latency=13.79s |Gather latency=0.83s (6.00%) |Generate time=8.77s (63.60%) |Training time=2.33s (16.91%) |Others=2.69 (19.49%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 154|ppo_ep: 1|act_loss: -0.0115966796875|cri_loss: 0.035980224609375|unsuper_loss: 0.0 average reward score: -3.23828125 ------------------------------------------------------------------------------------- |E2E latency=13.75s |Gather latency=1.02s (7.44%) |Generate time=8.69s (63.18%) |Training time=2.35s (17.06%) |Others=2.72 (19.76%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 155|ppo_ep: 1|act_loss: 0.01378631591796875|cri_loss: 0.0196533203125|unsuper_loss: 0.0 average reward score: -3.078125 ------------------------------------------------------------------------------------- |E2E latency=13.50s |Gather latency=0.66s (4.85%) |Generate time=8.63s (63.96%) |Training time=2.29s (16.94%) |Others=2.58 (19.10%)|CurSamplesPerSec=2.37 |AvgSamplesPerSec=2.30 epoch: 0|step: 156|ppo_ep: 1|act_loss: -0.00677490234375|cri_loss: 0.0299072265625|unsuper_loss: 0.0 average reward score: -3.240234375 ------------------------------------------------------------------------------------- |E2E latency=14.00s |Gather latency=0.91s (6.48%) |Generate time=9.07s (64.75%) |Training time=2.43s (17.35%) |Others=2.51 (17.90%)|CurSamplesPerSec=2.28 |AvgSamplesPerSec=2.30 epoch: 0|step: 157|ppo_ep: 1|act_loss: -0.006374359130859375|cri_loss: 0.018341064453125|unsuper_loss: 0.0 average reward score: -2.462890625 ------------------------------------------------------------------------------------- |E2E latency=13.86s |Gather latency=1.19s (8.60%) |Generate time=8.75s (63.10%) |Training time=2.41s (17.39%) |Others=2.70 (19.50%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.30 epoch: 0|step: 158|ppo_ep: 1|act_loss: 0.01422882080078125|cri_loss: 0.01003265380859375|unsuper_loss: 0.0 average reward score: -3.228515625 ------------------------------------------------------------------------------------- |E2E latency=14.09s |Gather latency=0.78s (5.54%) |Generate time=9.09s (64.52%) |Training time=2.44s (17.31%) |Others=2.56 (18.17%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.30 [2023-05-17 00:38:26,160] [INFO] [logging.py:96:log_dist] [Rank 0] step=160, skipped=4, lr=[9.547978680585101e-07, 9.547978680585101e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 00:38:26,161] [INFO] [timer.py:199:stop] epoch=0/micro_step=160/global_step=160, RunningAvgSamplesPerSec=28.300108366173355, CurrSamplesPerSec=28.899875802931465, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 00:38:26,914] [INFO] [logging.py:96:log_dist] [Rank 0] step=160, skipped=5, lr=[4.949003854277644e-06, 4.949003854277644e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 159|ppo_ep: 1|act_loss: 0.024505615234375|cri_loss: 0.01100921630859375|unsuper_loss: 0.0 average reward score: -2.943359375 ------------------------------------------------------------------------------------- |E2E latency=13.67s |Gather latency=0.65s (4.79%) |Generate time=8.74s (63.95%) |Training time=2.26s (16.55%) |Others=2.67 (19.50%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 160|ppo_ep: 1|act_loss: -0.074462890625|cri_loss: 0.0494384765625|unsuper_loss: 0.0 average reward score: -2.453125 ------------------------------------------------------------------------------------- |E2E latency=13.82s |Gather latency=0.76s (5.47%) |Generate time=8.73s (63.15%) |Training time=2.34s (16.91%) |Others=2.76 (19.94%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 161|ppo_ep: 1|act_loss: -0.002475738525390625|cri_loss: 0.039337158203125|unsuper_loss: 0.0 average reward score: -2.5390625 ------------------------------------------------------------------------------------- |E2E latency=14.04s |Gather latency=0.76s (5.44%) |Generate time=9.08s (64.70%) |Training time=2.42s (17.21%) |Others=2.54 (18.09%)|CurSamplesPerSec=2.28 |AvgSamplesPerSec=2.30 epoch: 0|step: 162|ppo_ep: 1|act_loss: 0.10040283203125|cri_loss: 0.029937744140625|unsuper_loss: 0.0 average reward score: -3.341796875 ------------------------------------------------------------------------------------- |E2E latency=13.91s |Gather latency=0.73s (5.28%) |Generate time=8.68s (62.38%) |Training time=2.43s (17.45%) |Others=2.81 (20.16%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.30 epoch: 0|step: 163|ppo_ep: 1|act_loss: 0.01117706298828125|cri_loss: 0.020904541015625|unsuper_loss: 0.0 average reward score: -2.078125 ------------------------------------------------------------------------------------- |E2E latency=13.71s |Gather latency=0.90s (6.55%) |Generate time=8.66s (63.19%) |Training time=2.31s (16.89%) |Others=2.73 (19.92%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 164|ppo_ep: 1|act_loss: 0.0350341796875|cri_loss: 0.0238189697265625|unsuper_loss: 0.0 average reward score: -3.24609375 ------------------------------------------------------------------------------------- |E2E latency=13.68s |Gather latency=0.63s (4.60%) |Generate time=8.68s (63.50%) |Training time=2.38s (17.39%) |Others=2.61 (19.11%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 165|ppo_ep: 1|act_loss: 0.029693603515625|cri_loss: 0.05670166015625|unsuper_loss: 0.0 average reward score: -2.7578125 ------------------------------------------------------------------------------------- |E2E latency=14.30s |Gather latency=0.87s (6.08%) |Generate time=9.16s (64.07%) |Training time=2.38s (16.67%) |Others=2.76 (19.27%)|CurSamplesPerSec=2.24 |AvgSamplesPerSec=2.30 epoch: 0|step: 166|ppo_ep: 1|act_loss: 0.0163726806640625|cri_loss: 0.02178955078125|unsuper_loss: 0.0 average reward score: -3.0625 ------------------------------------------------------------------------------------- |E2E latency=13.62s |Gather latency=0.78s (5.74%) |Generate time=8.68s (63.73%) |Training time=2.35s (17.23%) |Others=2.59 (19.04%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 167|ppo_ep: 1|act_loss: 0.00830078125|cri_loss: 0.0189666748046875|unsuper_loss: 0.0 average reward score: -3.0234375 ------------------------------------------------------------------------------------- |E2E latency=13.79s |Gather latency=1.12s (8.13%) |Generate time=8.74s (63.38%) |Training time=2.32s (16.83%) |Others=2.73 (19.79%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 168|ppo_ep: 1|act_loss: -0.030548095703125|cri_loss: 0.0184173583984375|unsuper_loss: 0.0 average reward score: -3.3203125 ------------------------------------------------------------------------------------- |E2E latency=13.67s |Gather latency=0.65s (4.74%) |Generate time=8.61s (62.97%) |Training time=2.32s (16.93%) |Others=2.75 (20.09%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 [2023-05-17 00:40:44,268] [INFO] [logging.py:96:log_dist] [Rank 0] step=170, skipped=4, lr=[9.508484263772676e-07, 9.508484263772676e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 00:40:44,269] [INFO] [timer.py:199:stop] epoch=0/micro_step=170/global_step=170, RunningAvgSamplesPerSec=28.326673539456642, CurrSamplesPerSec=29.58818727510051, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 00:40:45,083] [INFO] [logging.py:96:log_dist] [Rank 0] step=170, skipped=5, lr=[4.928870379830124e-06, 4.928870379830124e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 169|ppo_ep: 1|act_loss: -0.0963134765625|cri_loss: 0.03289794921875|unsuper_loss: 0.0 average reward score: -2.974609375 ------------------------------------------------------------------------------------- |E2E latency=13.64s |Gather latency=1.13s (8.30%) |Generate time=8.73s (63.98%) |Training time=2.31s (16.94%) |Others=2.60 (19.08%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 170|ppo_ep: 1|act_loss: -0.097900390625|cri_loss: 0.041290283203125|unsuper_loss: 0.0 average reward score: -2.216796875 ------------------------------------------------------------------------------------- |E2E latency=13.71s |Gather latency=0.79s (5.80%) |Generate time=8.81s (64.24%) |Training time=2.29s (16.73%) |Others=2.61 (19.03%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 171|ppo_ep: 1|act_loss: -0.1041259765625|cri_loss: 0.06549072265625|unsuper_loss: 0.0 average reward score: -2.3671875 ------------------------------------------------------------------------------------- |E2E latency=13.64s |Gather latency=0.82s (6.02%) |Generate time=8.71s (63.88%) |Training time=2.37s (17.35%) |Others=2.56 (18.77%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 172|ppo_ep: 1|act_loss: -0.06329345703125|cri_loss: 0.040924072265625|unsuper_loss: 0.0 average reward score: -3.337890625 ------------------------------------------------------------------------------------- |E2E latency=13.55s |Gather latency=1.04s (7.69%) |Generate time=8.65s (63.87%) |Training time=2.34s (17.25%) |Others=2.56 (18.88%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.30 epoch: 0|step: 173|ppo_ep: 1|act_loss: -0.0648193359375|cri_loss: 0.026824951171875|unsuper_loss: 0.0 average reward score: -2.7265625 ------------------------------------------------------------------------------------- |E2E latency=13.91s |Gather latency=0.90s (6.47%) |Generate time=8.69s (62.51%) |Training time=2.47s (17.75%) |Others=2.75 (19.74%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.30 epoch: 0|step: 174|ppo_ep: 1|act_loss: -0.04010009765625|cri_loss: 0.044036865234375|unsuper_loss: 0.0 average reward score: -3.185546875 ------------------------------------------------------------------------------------- |E2E latency=13.65s |Gather latency=0.77s (5.67%) |Generate time=8.73s (63.97%) |Training time=2.35s (17.24%) |Others=2.56 (18.79%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 175|ppo_ep: 1|act_loss: -0.0289306640625|cri_loss: 0.0246734619140625|unsuper_loss: 0.0 average reward score: -2.91015625 ------------------------------------------------------------------------------------- |E2E latency=13.12s |Gather latency=0.84s (6.44%) |Generate time=8.13s (62.00%) |Training time=2.36s (18.00%) |Others=2.62 (20.00%)|CurSamplesPerSec=2.44 |AvgSamplesPerSec=2.30 epoch: 0|step: 176|ppo_ep: 1|act_loss: -0.041351318359375|cri_loss: 0.03875732421875|unsuper_loss: 0.0 average reward score: -2.583984375 ------------------------------------------------------------------------------------- |E2E latency=13.86s |Gather latency=0.70s (5.03%) |Generate time=8.75s (63.17%) |Training time=2.36s (17.00%) |Others=2.75 (19.83%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.30 epoch: 0|step: 177|ppo_ep: 1|act_loss: -0.04327392578125|cri_loss: 0.04864501953125|unsuper_loss: 0.0 average reward score: -2.697265625 ------------------------------------------------------------------------------------- |E2E latency=13.70s |Gather latency=0.83s (6.03%) |Generate time=8.73s (63.69%) |Training time=2.31s (16.86%) |Others=2.66 (19.44%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 178|ppo_ep: 1|act_loss: 0.037506103515625|cri_loss: 0.036102294921875|unsuper_loss: 0.0 average reward score: -2.37890625 ------------------------------------------------------------------------------------- |E2E latency=13.66s |Gather latency=0.77s (5.62%) |Generate time=8.62s (63.10%) |Training time=2.32s (16.96%) |Others=2.72 (19.94%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 [2023-05-17 00:43:00,859] [INFO] [logging.py:96:log_dist] [Rank 0] step=180, skipped=4, lr=[9.462652549501915e-07, 9.462652549501915e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 00:43:00,860] [INFO] [timer.py:199:stop] epoch=0/micro_step=180/global_step=180, RunningAvgSamplesPerSec=28.343826670317956, CurrSamplesPerSec=27.026939601358983, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 00:43:01,626] [INFO] [logging.py:96:log_dist] [Rank 0] step=180, skipped=5, lr=[4.905450362027739e-06, 4.905450362027739e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 179|ppo_ep: 1|act_loss: 0.0283050537109375|cri_loss: 0.0382080078125|unsuper_loss: 0.0 average reward score: -2.115234375 ------------------------------------------------------------------------------------- |E2E latency=13.75s |Gather latency=0.38s (2.76%) |Generate time=8.76s (63.76%) |Training time=2.46s (17.90%) |Others=2.52 (18.35%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 180|ppo_ep: 1|act_loss: 0.0394287109375|cri_loss: 0.030487060546875|unsuper_loss: 0.0 average reward score: -2.759765625 ------------------------------------------------------------------------------------- |E2E latency=13.88s |Gather latency=0.59s (4.22%) |Generate time=8.72s (62.87%) |Training time=2.41s (17.40%) |Others=2.74 (19.73%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.30 epoch: 0|step: 181|ppo_ep: 1|act_loss: 0.1317138671875|cri_loss: 0.08074951171875|unsuper_loss: 0.0 average reward score: -2.55859375 ------------------------------------------------------------------------------------- |E2E latency=13.86s |Gather latency=0.74s (5.34%) |Generate time=8.71s (62.88%) |Training time=2.42s (17.50%) |Others=2.72 (19.62%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.30 epoch: 0|step: 182|ppo_ep: 1|act_loss: 0.10650634765625|cri_loss: 0.0830078125|unsuper_loss: 0.0 average reward score: -3.330078125 ------------------------------------------------------------------------------------- |E2E latency=13.72s |Gather latency=0.74s (5.40%) |Generate time=8.66s (63.16%) |Training time=2.45s (17.83%) |Others=2.61 (19.01%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 183|ppo_ep: 1|act_loss: 0.0980224609375|cri_loss: 0.059844970703125|unsuper_loss: 0.0 average reward score: -2.60546875 ------------------------------------------------------------------------------------- |E2E latency=13.76s |Gather latency=0.72s (5.24%) |Generate time=8.67s (63.01%) |Training time=2.47s (17.93%) |Others=2.62 (19.06%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 184|ppo_ep: 1|act_loss: 0.07525634765625|cri_loss: 0.0289459228515625|unsuper_loss: 0.0 average reward score: -3.291015625 ------------------------------------------------------------------------------------- |E2E latency=13.91s |Gather latency=0.82s (5.91%) |Generate time=8.76s (62.96%) |Training time=2.39s (17.17%) |Others=2.76 (19.87%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.30 epoch: 0|step: 185|ppo_ep: 1|act_loss: 0.1090087890625|cri_loss: 0.0357666015625|unsuper_loss: 0.0 average reward score: -2.630859375 ------------------------------------------------------------------------------------- |E2E latency=13.62s |Gather latency=0.60s (4.39%) |Generate time=8.76s (64.31%) |Training time=2.31s (16.99%) |Others=2.55 (18.69%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 186|ppo_ep: 1|act_loss: -0.0053558349609375|cri_loss: 0.056182861328125|unsuper_loss: 0.0 average reward score: -2.693359375 ------------------------------------------------------------------------------------- |E2E latency=13.74s |Gather latency=1.03s (7.50%) |Generate time=8.71s (63.41%) |Training time=2.35s (17.11%) |Others=2.68 (19.47%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 187|ppo_ep: 1|act_loss: 0.053436279296875|cri_loss: 0.0255126953125|unsuper_loss: 0.0 average reward score: -2.439453125 ------------------------------------------------------------------------------------- |E2E latency=13.63s |Gather latency=0.79s (5.77%) |Generate time=8.69s (63.76%) |Training time=2.37s (17.36%) |Others=2.57 (18.88%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.31 epoch: 0|step: 188|ppo_ep: 1|act_loss: 0.0280914306640625|cri_loss: 0.0293731689453125|unsuper_loss: 0.0 average reward score: -3.48828125 ------------------------------------------------------------------------------------- |E2E latency=13.88s |Gather latency=0.65s (4.70%) |Generate time=8.94s (64.38%) |Training time=2.33s (16.77%) |Others=2.62 (18.84%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.31 [2023-05-17 00:45:18,693] [INFO] [logging.py:96:log_dist] [Rank 0] step=190, skipped=4, lr=[9.410545553398367e-07, 9.410545553398367e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 00:45:18,694] [INFO] [timer.py:199:stop] epoch=0/micro_step=190/global_step=190, RunningAvgSamplesPerSec=28.32191484552592, CurrSamplesPerSec=29.29595703710753, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 00:45:19,413] [INFO] [logging.py:96:log_dist] [Rank 0] step=190, skipped=5, lr=[4.878775490871121e-06, 4.878775490871121e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 189|ppo_ep: 1|act_loss: -0.0256195068359375|cri_loss: 0.06512451171875|unsuper_loss: 0.0 average reward score: -2.537109375 ------------------------------------------------------------------------------------- |E2E latency=13.80s |Gather latency=0.85s (6.18%) |Generate time=8.82s (63.95%) |Training time=2.39s (17.29%) |Others=2.59 (18.76%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.31 epoch: 0|step: 190|ppo_ep: 1|act_loss: 0.0029926300048828125|cri_loss: 0.03753662109375|unsuper_loss: 0.0 average reward score: -3.49609375 ------------------------------------------------------------------------------------- |E2E latency=12.93s |Gather latency=0.73s (5.65%) |Generate time=7.77s (60.10%) |Training time=2.40s (18.60%) |Others=2.75 (21.30%)|CurSamplesPerSec=2.48 |AvgSamplesPerSec=2.31 epoch: 0|step: 191|ppo_ep: 1|act_loss: -0.044158935546875|cri_loss: 0.040985107421875|unsuper_loss: 0.0 average reward score: -3.24609375 ------------------------------------------------------------------------------------- |E2E latency=13.78s |Gather latency=0.67s (4.84%) |Generate time=8.76s (63.53%) |Training time=2.42s (17.56%) |Others=2.61 (18.91%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.31 epoch: 0|step: 192|ppo_ep: 1|act_loss: -0.0689697265625|cri_loss: 0.043853759765625|unsuper_loss: 0.0 average reward score: -2.9375 ------------------------------------------------------------------------------------- |E2E latency=13.96s |Gather latency=0.92s (6.57%) |Generate time=8.73s (62.50%) |Training time=2.44s (17.45%) |Others=2.80 (20.05%)|CurSamplesPerSec=2.29 |AvgSamplesPerSec=2.31 epoch: 0|step: 193|ppo_ep: 1|act_loss: -0.07098388671875|cri_loss: 0.059539794921875|unsuper_loss: 0.0 average reward score: -2.89453125 ------------------------------------------------------------------------------------- |E2E latency=13.71s |Gather latency=1.22s (8.86%) |Generate time=8.72s (63.60%) |Training time=2.37s (17.29%) |Others=2.62 (19.11%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.31 epoch: 0|step: 194|ppo_ep: 1|act_loss: -0.0257110595703125|cri_loss: 0.0211639404296875|unsuper_loss: 0.0 average reward score: -2.4921875 ------------------------------------------------------------------------------------- |E2E latency=13.68s |Gather latency=1.14s (8.35%) |Generate time=8.64s (63.17%) |Training time=2.36s (17.24%) |Others=2.68 (19.60%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.31 epoch: 0|step: 195|ppo_ep: 1|act_loss: -0.07598876953125|cri_loss: 0.0518798828125|unsuper_loss: 0.0 average reward score: -3.453125 ------------------------------------------------------------------------------------- |E2E latency=13.67s |Gather latency=0.74s (5.40%) |Generate time=8.62s (63.08%) |Training time=2.37s (17.36%) |Others=2.67 (19.56%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.31 epoch: 0|step: 196|ppo_ep: 1|act_loss: 0.0302734375|cri_loss: 0.024169921875|unsuper_loss: 0.0 average reward score: -3.173828125 ------------------------------------------------------------------------------------- |E2E latency=13.62s |Gather latency=0.64s (4.67%) |Generate time=8.70s (63.86%) |Training time=2.38s (17.45%) |Others=2.55 (18.68%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.31 epoch: 0|step: 197|ppo_ep: 1|act_loss: -0.032318115234375|cri_loss: 0.0225677490234375|unsuper_loss: 0.0 average reward score: -2.42578125 ------------------------------------------------------------------------------------- |E2E latency=13.85s |Gather latency=0.89s (6.40%) |Generate time=8.73s (62.99%) |Training time=2.42s (17.48%) |Others=2.71 (19.53%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.31 epoch: 0|step: 198|ppo_ep: 1|act_loss: 0.041900634765625|cri_loss: 0.0225677490234375|unsuper_loss: 0.0 average reward score: -3.470703125 ------------------------------------------------------------------------------------- |E2E latency=13.71s |Gather latency=0.89s (6.46%) |Generate time=8.65s (63.09%) |Training time=2.35s (17.14%) |Others=2.71 (19.76%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.31 [2023-05-17 00:47:35,223] [INFO] [logging.py:96:log_dist] [Rank 0] step=200, skipped=4, lr=[9.352233782271784e-07, 9.352233782271784e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 00:47:35,223] [INFO] [timer.py:199:stop] epoch=0/micro_step=200/global_step=200, RunningAvgSamplesPerSec=28.31052394393893, CurrSamplesPerSec=28.537078525070296, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 00:47:35,941] [INFO] [logging.py:96:log_dist] [Rank 0] step=200, skipped=5, lr=[4.848881860555035e-06, 4.848881860555035e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 199|ppo_ep: 1|act_loss: 0.0161895751953125|cri_loss: 0.01479339599609375|unsuper_loss: 0.0 average reward score: -3.404296875 ------------------------------------------------------------------------------------- |E2E latency=13.60s |Gather latency=0.63s (4.62%) |Generate time=8.71s (64.09%) |Training time=2.35s (17.30%) |Others=2.53 (18.61%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.31 epoch: 0|step: 200|ppo_ep: 1|act_loss: 0.065673828125|cri_loss: 0.01485443115234375|unsuper_loss: 0.0 average reward score: -2.1796875 ------------------------------------------------------------------------------------- |E2E latency=13.95s |Gather latency=0.83s (5.92%) |Generate time=8.73s (62.63%) |Training time=2.36s (16.90%) |Others=2.85 (20.47%)|CurSamplesPerSec=2.29 |AvgSamplesPerSec=2.31 epoch: 0|step: 201|ppo_ep: 1|act_loss: -0.0293426513671875|cri_loss: 0.01021575927734375|unsuper_loss: 0.0 average reward score: -2.943359375 ------------------------------------------------------------------------------------- |E2E latency=13.86s |Gather latency=0.89s (6.40%) |Generate time=8.84s (63.81%) |Training time=2.42s (17.45%) |Others=2.60 (18.74%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.31 epoch: 0|step: 202|ppo_ep: 1|act_loss: 0.0226898193359375|cri_loss: 0.00911712646484375|unsuper_loss: 0.0 average reward score: -3.32421875 ------------------------------------------------------------------------------------- |E2E latency=13.82s |Gather latency=0.75s (5.40%) |Generate time=8.79s (63.59%) |Training time=2.33s (16.88%) |Others=2.70 (19.53%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.31 epoch: 0|step: 203|ppo_ep: 1|act_loss: -0.020111083984375|cri_loss: 0.0209808349609375|unsuper_loss: 0.0 average reward score: -2.908203125 ------------------------------------------------------------------------------------- |E2E latency=13.73s |Gather latency=0.84s (6.12%) |Generate time=8.60s (62.65%) |Training time=2.47s (17.97%) |Others=2.66 (19.38%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.31 epoch: 0|step: 204|ppo_ep: 1|act_loss: 0.09368896484375|cri_loss: 0.0179290771484375|unsuper_loss: 0.0 average reward score: -3.130859375 ------------------------------------------------------------------------------------- |E2E latency=13.76s |Gather latency=0.77s (5.63%) |Generate time=8.70s (63.22%) |Training time=2.40s (17.47%) |Others=2.66 (19.30%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.31 epoch: 0|step: 205|ppo_ep: 1|act_loss: 0.005893707275390625|cri_loss: 0.016510009765625|unsuper_loss: 0.0 average reward score: -3.419921875 ------------------------------------------------------------------------------------- |E2E latency=13.95s |Gather latency=0.99s (7.10%) |Generate time=8.81s (63.13%) |Training time=2.42s (17.32%) |Others=2.73 (19.55%)|CurSamplesPerSec=2.29 |AvgSamplesPerSec=2.31 epoch: 0|step: 206|ppo_ep: 1|act_loss: -0.12384033203125|cri_loss: 0.06787109375|unsuper_loss: 0.0 average reward score: -2.59375 ------------------------------------------------------------------------------------- |E2E latency=13.79s |Gather latency=0.98s (7.10%) |Generate time=8.74s (63.36%) |Training time=2.46s (17.84%) |Others=2.59 (18.81%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.31 epoch: 0|step: 207|ppo_ep: 1|act_loss: 0.04522705078125|cri_loss: 0.0258636474609375|unsuper_loss: 0.0 average reward score: -2.66015625 ------------------------------------------------------------------------------------- |E2E latency=13.82s |Gather latency=0.92s (6.68%) |Generate time=8.76s (63.42%) |Training time=2.41s (17.46%) |Others=2.64 (19.12%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.31 epoch: 0|step: 208|ppo_ep: 1|act_loss: 0.022430419921875|cri_loss: 0.018951416015625|unsuper_loss: 0.0 average reward score: -2.978515625 ------------------------------------------------------------------------------------- |E2E latency=13.91s |Gather latency=1.09s (7.87%) |Generate time=8.73s (62.79%) |Training time=2.34s (16.83%) |Others=2.83 (20.38%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.31 [2023-05-17 00:49:53,377] [INFO] [logging.py:96:log_dist] [Rank 0] step=210, skipped=4, lr=[9.287796138712235e-07, 9.287796138712235e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 00:49:53,378] [INFO] [timer.py:199:stop] epoch=0/micro_step=210/global_step=210, RunningAvgSamplesPerSec=28.27265178435404, CurrSamplesPerSec=28.29309716425515, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 00:49:54,100] [INFO] [logging.py:96:log_dist] [Rank 0] step=210, skipped=5, lr=[4.815809920628738e-06, 4.815809920628738e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 209|ppo_ep: 1|act_loss: -0.036834716796875|cri_loss: 0.0133209228515625|unsuper_loss: 0.0 average reward score: -2.146484375 ------------------------------------------------------------------------------------- |E2E latency=13.58s |Gather latency=0.69s (5.11%) |Generate time=8.74s (64.41%) |Training time=2.33s (17.19%) |Others=2.50 (18.40%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.31 epoch: 0|step: 210|ppo_ep: 1|act_loss: -0.003955841064453125|cri_loss: 0.0362548828125|unsuper_loss: 0.0 average reward score: -2.255859375 ------------------------------------------------------------------------------------- |E2E latency=13.64s |Gather latency=0.72s (5.26%) |Generate time=8.74s (64.06%) |Training time=2.24s (16.43%) |Others=2.66 (19.50%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.31 epoch: 0|step: 211|ppo_ep: 1|act_loss: -0.04425048828125|cri_loss: 0.0207061767578125|unsuper_loss: 0.0 average reward score: -2.513671875 ------------------------------------------------------------------------------------- |E2E latency=13.71s |Gather latency=0.76s (5.55%) |Generate time=8.65s (63.08%) |Training time=2.41s (17.57%) |Others=2.65 (19.34%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.31 epoch: 0|step: 212|ppo_ep: 1|act_loss: -0.0167694091796875|cri_loss: 0.0203857421875|unsuper_loss: 0.0 average reward score: -2.451171875 ------------------------------------------------------------------------------------- |E2E latency=13.75s |Gather latency=1.28s (9.31%) |Generate time=8.72s (63.45%) |Training time=2.41s (17.53%) |Others=2.62 (19.02%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.31 epoch: 0|step: 213|ppo_ep: 1|act_loss: 0.002269744873046875|cri_loss: 0.014129638671875|unsuper_loss: 0.0 average reward score: -3.73828125 ------------------------------------------------------------------------------------- |E2E latency=13.76s |Gather latency=1.13s (8.22%) |Generate time=8.73s (63.44%) |Training time=2.36s (17.16%) |Others=2.67 (19.40%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.31 epoch: 0|step: 214|ppo_ep: 1|act_loss: -0.006938934326171875|cri_loss: 0.0177001953125|unsuper_loss: 0.0 average reward score: -3.1328125 ------------------------------------------------------------------------------------- |E2E latency=13.71s |Gather latency=1.13s (8.27%) |Generate time=8.83s (64.43%) |Training time=2.32s (16.92%) |Others=2.56 (18.65%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.31 epoch: 0|step: 215|ppo_ep: 1|act_loss: 0.01557159423828125|cri_loss: 0.0220794677734375|unsuper_loss: 0.0 average reward score: -3.41796875 ------------------------------------------------------------------------------------- |E2E latency=13.78s |Gather latency=0.48s (3.46%) |Generate time=8.69s (63.08%) |Training time=2.42s (17.54%) |Others=2.67 (19.38%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.31 epoch: 0|step: 216|ppo_ep: 1|act_loss: -0.00572967529296875|cri_loss: 0.0192718505859375|unsuper_loss: 0.0 average reward score: -2.23828125 ------------------------------------------------------------------------------------- |E2E latency=13.79s |Gather latency=0.61s (4.40%) |Generate time=8.64s (62.66%) |Training time=2.38s (17.27%) |Others=2.77 (20.07%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.31 epoch: 0|step: 217|ppo_ep: 1|act_loss: 0.06256103515625|cri_loss: 0.0244903564453125|unsuper_loss: 0.0 average reward score: -2.818359375 ------------------------------------------------------------------------------------- |E2E latency=13.72s |Gather latency=0.72s (5.28%) |Generate time=8.71s (63.50%) |Training time=2.36s (17.22%) |Others=2.65 (19.28%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.31 epoch: 0|step: 218|ppo_ep: 1|act_loss: 0.08013916015625|cri_loss: 0.0239105224609375|unsuper_loss: 0.0 average reward score: -3.01953125 ------------------------------------------------------------------------------------- |E2E latency=13.91s |Gather latency=0.68s (4.90%) |Generate time=8.77s (63.03%) |Training time=2.43s (17.47%) |Others=2.71 (19.49%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.31 [2023-05-17 00:52:10,871] [INFO] [logging.py:96:log_dist] [Rank 0] step=220, skipped=4, lr=[9.217319814325746e-07, 9.217319814325746e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 00:52:10,871] [INFO] [timer.py:199:stop] epoch=0/micro_step=220/global_step=220, RunningAvgSamplesPerSec=28.26261418055086, CurrSamplesPerSec=26.87175556899633, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 00:52:11,632] [INFO] [logging.py:96:log_dist] [Rank 0] step=220, skipped=5, lr=[4.779604421263048e-06, 4.779604421263048e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 219|ppo_ep: 1|act_loss: -0.0147247314453125|cri_loss: 0.01551055908203125|unsuper_loss: 0.0 average reward score: -2.83984375 ------------------------------------------------------------------------------------- |E2E latency=13.77s |Gather latency=0.88s (6.38%) |Generate time=8.72s (63.38%) |Training time=2.43s (17.66%) |Others=2.61 (18.95%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.31 epoch: 0|step: 220|ppo_ep: 1|act_loss: -0.00026226043701171875|cri_loss: 0.02264404296875|unsuper_loss: 0.0 average reward score: -3.1171875 ------------------------------------------------------------------------------------- |E2E latency=13.78s |Gather latency=0.99s (7.22%) |Generate time=8.67s (62.95%) |Training time=2.37s (17.22%) |Others=2.73 (19.83%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.31 epoch: 0|step: 221|ppo_ep: 1|act_loss: -0.041961669921875|cri_loss: 0.049407958984375|unsuper_loss: 0.0 average reward score: -2.921875 ------------------------------------------------------------------------------------- |E2E latency=13.65s |Gather latency=1.14s (8.33%) |Generate time=8.70s (63.73%) |Training time=2.34s (17.13%) |Others=2.61 (19.14%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.31 epoch: 0|step: 222|ppo_ep: 1|act_loss: 0.0174713134765625|cri_loss: 0.0200347900390625|unsuper_loss: 0.0 average reward score: -3.23046875 ------------------------------------------------------------------------------------- |E2E latency=13.62s |Gather latency=0.55s (4.05%) |Generate time=8.68s (63.72%) |Training time=2.29s (16.79%) |Others=2.65 (19.49%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.31 epoch: 0|step: 223|ppo_ep: 1|act_loss: -0.099609375|cri_loss: 0.1502685546875|unsuper_loss: 0.0 average reward score: -2.794921875 ------------------------------------------------------------------------------------- |E2E latency=14.33s |Gather latency=0.56s (3.94%) |Generate time=9.37s (65.41%) |Training time=2.40s (16.73%) |Others=2.56 (17.86%)|CurSamplesPerSec=2.23 |AvgSamplesPerSec=2.31 epoch: 0|step: 224|ppo_ep: 1|act_loss: 0.0826416015625|cri_loss: 0.0247650146484375|unsuper_loss: 0.0 average reward score: -2.921875 ------------------------------------------------------------------------------------- |E2E latency=14.38s |Gather latency=0.83s (5.74%) |Generate time=9.21s (64.03%) |Training time=2.40s (16.70%) |Others=2.77 (19.27%)|CurSamplesPerSec=2.23 |AvgSamplesPerSec=2.31 epoch: 0|step: 225|ppo_ep: 1|act_loss: -0.060455322265625|cri_loss: 0.0275421142578125|unsuper_loss: 0.0 average reward score: -3.01953125 ------------------------------------------------------------------------------------- |E2E latency=13.77s |Gather latency=0.74s (5.39%) |Generate time=8.71s (63.25%) |Training time=2.38s (17.25%) |Others=2.68 (19.50%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.31 epoch: 0|step: 226|ppo_ep: 1|act_loss: -0.0195770263671875|cri_loss: 0.0198516845703125|unsuper_loss: 0.0 average reward score: -2.755859375 ------------------------------------------------------------------------------------- |E2E latency=13.71s |Gather latency=1.03s (7.53%) |Generate time=8.70s (63.45%) |Training time=2.38s (17.38%) |Others=2.63 (19.17%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.31 epoch: 0|step: 227|ppo_ep: 1|act_loss: -0.0733642578125|cri_loss: 0.021484375|unsuper_loss: 0.0 average reward score: -2.818359375 ------------------------------------------------------------------------------------- |E2E latency=13.69s |Gather latency=0.70s (5.14%) |Generate time=8.77s (64.10%) |Training time=2.28s (16.65%) |Others=2.64 (19.26%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.31 epoch: 0|step: 228|ppo_ep: 1|act_loss: -0.007007598876953125|cri_loss: 0.0193023681640625|unsuper_loss: 0.0 average reward score: -2.666015625 ------------------------------------------------------------------------------------- |E2E latency=13.62s |Gather latency=0.90s (6.62%) |Generate time=8.66s (63.61%) |Training time=2.36s (17.33%) |Others=2.60 (19.06%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.31 [2023-05-17 00:54:29,703] [INFO] [logging.py:96:log_dist] [Rank 0] step=230, skipped=4, lr=[9.140900171753939e-07, 9.140900171753939e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 00:54:29,704] [INFO] [timer.py:199:stop] epoch=0/micro_step=230/global_step=230, RunningAvgSamplesPerSec=28.279282333021474, CurrSamplesPerSec=27.476837240076737, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 00:54:30,457] [INFO] [logging.py:96:log_dist] [Rank 0] step=230, skipped=5, lr=[4.74031435269818e-06, 4.74031435269818e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 229|ppo_ep: 1|act_loss: -0.055938720703125|cri_loss: 0.021392822265625|unsuper_loss: 0.0 average reward score: -2.15625 ------------------------------------------------------------------------------------- |E2E latency=14.28s |Gather latency=0.99s (6.92%) |Generate time=9.15s (64.05%) |Training time=2.40s (16.83%) |Others=2.73 (19.13%)|CurSamplesPerSec=2.24 |AvgSamplesPerSec=2.31 epoch: 0|step: 230|ppo_ep: 1|act_loss: -0.02032470703125|cri_loss: 0.023101806640625|unsuper_loss: 0.0 average reward score: -1.875 ------------------------------------------------------------------------------------- |E2E latency=13.74s |Gather latency=1.13s (8.22%) |Generate time=8.77s (63.82%) |Training time=2.37s (17.21%) |Others=2.61 (18.97%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.31 epoch: 0|step: 231|ppo_ep: 1|act_loss: 0.066162109375|cri_loss: 0.07318115234375|unsuper_loss: 0.0 average reward score: -2.71484375 ------------------------------------------------------------------------------------- |E2E latency=13.74s |Gather latency=0.65s (4.70%) |Generate time=8.66s (63.05%) |Training time=2.32s (16.87%) |Others=2.76 (20.08%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.31 epoch: 0|step: 232|ppo_ep: 1|act_loss: -0.0129547119140625|cri_loss: 0.013671875|unsuper_loss: 0.0 average reward score: -2.328125 ------------------------------------------------------------------------------------- |E2E latency=15.86s |Gather latency=1.13s (7.12%) |Generate time=9.15s (57.72%) |Training time=3.30s (20.81%) |Others=3.40 (21.47%)|CurSamplesPerSec=2.02 |AvgSamplesPerSec=2.31 epoch: 0|step: 233|ppo_ep: 1|act_loss: 0.0313720703125|cri_loss: 0.0184173583984375|unsuper_loss: 0.0 average reward score: -1.3984375 ------------------------------------------------------------------------------------- |E2E latency=14.17s |Gather latency=0.71s (5.03%) |Generate time=9.12s (64.41%) |Training time=2.35s (16.62%) |Others=2.69 (18.97%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.31 epoch: 0|step: 234|ppo_ep: 1|act_loss: 0.004375457763671875|cri_loss: 0.0302886962890625|unsuper_loss: 0.0 average reward score: -2.71875 ------------------------------------------------------------------------------------- |E2E latency=13.56s |Gather latency=0.99s (7.33%) |Generate time=8.63s (63.61%) |Training time=2.30s (16.97%) |Others=2.63 (19.42%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.31 epoch: 0|step: 235|ppo_ep: 1|act_loss: 0.128173828125|cri_loss: 0.06121826171875|unsuper_loss: 0.0 average reward score: -1.5234375 ------------------------------------------------------------------------------------- |E2E latency=13.56s |Gather latency=0.69s (5.09%) |Generate time=8.67s (63.91%) |Training time=2.31s (17.03%) |Others=2.58 (19.06%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.31 epoch: 0|step: 236|ppo_ep: 1|act_loss: 0.12255859375|cri_loss: 0.038238525390625|unsuper_loss: 0.0 average reward score: -3.013671875 ------------------------------------------------------------------------------------- |E2E latency=14.30s |Gather latency=1.07s (7.51%) |Generate time=9.16s (64.02%) |Training time=2.47s (17.25%) |Others=2.68 (18.73%)|CurSamplesPerSec=2.24 |AvgSamplesPerSec=2.31 epoch: 0|step: 237|ppo_ep: 1|act_loss: -0.0362548828125|cri_loss: 0.0170440673828125|unsuper_loss: 0.0 average reward score: -2.78125 ------------------------------------------------------------------------------------- |E2E latency=13.73s |Gather latency=1.00s (7.31%) |Generate time=8.67s (63.14%) |Training time=2.36s (17.17%) |Others=2.70 (19.70%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.31 epoch: 0|step: 238|ppo_ep: 1|act_loss: -0.00675201416015625|cri_loss: 0.01201629638671875|unsuper_loss: 0.0 average reward score: -3.126953125 ------------------------------------------------------------------------------------- |E2E latency=14.20s |Gather latency=0.97s (6.86%) |Generate time=9.15s (64.41%) |Training time=2.41s (16.94%) |Others=2.65 (18.65%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.31 [2023-05-17 00:56:50,679] [INFO] [logging.py:96:log_dist] [Rank 0] step=240, skipped=4, lr=[9.05864061563733e-07, 9.05864061563733e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 00:56:50,680] [INFO] [timer.py:199:stop] epoch=0/micro_step=240/global_step=240, RunningAvgSamplesPerSec=28.288271333055608, CurrSamplesPerSec=29.736068099451572, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 00:56:51,437] [INFO] [logging.py:96:log_dist] [Rank 0] step=240, skipped=5, lr=[4.697992878954255e-06, 4.697992878954255e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 239|ppo_ep: 1|act_loss: -0.0012369155883789062|cri_loss: 0.01296234130859375|unsuper_loss: 0.0 average reward score: -2.173828125 ------------------------------------------------------------------------------------- |E2E latency=14.12s |Gather latency=0.73s (5.18%) |Generate time=9.12s (64.62%) |Training time=2.30s (16.31%) |Others=2.69 (19.08%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.31 epoch: 0|step: 240|ppo_ep: 1|act_loss: -0.03582763671875|cri_loss: 0.01308441162109375|unsuper_loss: 0.0 average reward score: -1.830078125 ------------------------------------------------------------------------------------- |E2E latency=13.72s |Gather latency=0.81s (5.91%) |Generate time=8.71s (63.52%) |Training time=2.35s (17.11%) |Others=2.66 (19.36%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.31 epoch: 0|step: 241|ppo_ep: 1|act_loss: -0.0210418701171875|cri_loss: 0.023468017578125|unsuper_loss: 0.0 average reward score: -2.779296875 ------------------------------------------------------------------------------------- |E2E latency=13.71s |Gather latency=0.68s (4.93%) |Generate time=8.68s (63.30%) |Training time=2.38s (17.38%) |Others=2.65 (19.33%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.31 epoch: 0|step: 242|ppo_ep: 1|act_loss: -0.030364990234375|cri_loss: 0.0167694091796875|unsuper_loss: 0.0 average reward score: -1.517578125 ------------------------------------------------------------------------------------- |E2E latency=14.05s |Gather latency=0.72s (5.14%) |Generate time=9.15s (65.09%) |Training time=2.37s (16.85%) |Others=2.54 (18.05%)|CurSamplesPerSec=2.28 |AvgSamplesPerSec=2.31 epoch: 0|step: 243|ppo_ep: 1|act_loss: -0.0225830078125|cri_loss: 0.031585693359375|unsuper_loss: 0.0 average reward score: -3.068359375 ------------------------------------------------------------------------------------- |E2E latency=13.59s |Gather latency=0.88s (6.50%) |Generate time=8.62s (63.40%) |Training time=2.28s (16.75%) |Others=2.70 (19.85%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.31 epoch: 0|step: 244|ppo_ep: 1|act_loss: -0.0287322998046875|cri_loss: 0.0171661376953125|unsuper_loss: 0.0 average reward score: -2.548828125 ------------------------------------------------------------------------------------- |E2E latency=13.95s |Gather latency=0.76s (5.46%) |Generate time=8.72s (62.48%) |Training time=2.44s (17.51%) |Others=2.79 (20.01%)|CurSamplesPerSec=2.29 |AvgSamplesPerSec=2.31 epoch: 0|step: 245|ppo_ep: 1|act_loss: -0.04986572265625|cri_loss: 0.01486968994140625|unsuper_loss: 0.0 average reward score: -3.103515625 ------------------------------------------------------------------------------------- |E2E latency=13.55s |Gather latency=0.59s (4.33%) |Generate time=8.64s (63.73%) |Training time=2.33s (17.20%) |Others=2.58 (19.07%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.31 epoch: 0|step: 246|ppo_ep: 1|act_loss: -0.05255126953125|cri_loss: 0.0140838623046875|unsuper_loss: 0.0 average reward score: -2.69921875 ------------------------------------------------------------------------------------- |E2E latency=14.03s |Gather latency=0.63s (4.52%) |Generate time=9.13s (65.09%) |Training time=2.31s (16.46%) |Others=2.59 (18.44%)|CurSamplesPerSec=2.28 |AvgSamplesPerSec=2.31 epoch: 0|step: 247|ppo_ep: 1|act_loss: 0.004611968994140625|cri_loss: 0.02001953125|unsuper_loss: 0.0 average reward score: -2.626953125 ------------------------------------------------------------------------------------- |E2E latency=13.56s |Gather latency=0.67s (4.97%) |Generate time=8.64s (63.67%) |Training time=2.33s (17.18%) |Others=2.60 (19.15%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.31 epoch: 0|step: 248|ppo_ep: 1|act_loss: -0.0055084228515625|cri_loss: 0.044677734375|unsuper_loss: 0.0 average reward score: -2.572265625 ------------------------------------------------------------------------------------- |E2E latency=14.11s |Gather latency=0.66s (4.68%) |Generate time=9.14s (64.82%) |Training time=2.43s (17.25%) |Others=2.53 (17.93%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.31 [2023-05-17 00:59:09,276] [INFO] [logging.py:96:log_dist] [Rank 0] step=250, skipped=4, lr=[8.97065245269684e-07, 8.97065245269684e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 00:59:09,277] [INFO] [timer.py:199:stop] epoch=0/micro_step=250/global_step=250, RunningAvgSamplesPerSec=28.29216870580982, CurrSamplesPerSec=27.98231967808321, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 00:59:10,029] [INFO] [logging.py:96:log_dist] [Rank 0] step=250, skipped=5, lr=[4.652697265894228e-06, 4.652697265894228e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 249|ppo_ep: 1|act_loss: -0.02923583984375|cri_loss: 0.0261688232421875|unsuper_loss: 0.0 average reward score: -3.494140625 ------------------------------------------------------------------------------------- |E2E latency=14.30s |Gather latency=1.22s (8.56%) |Generate time=9.16s (64.02%) |Training time=2.39s (16.70%) |Others=2.76 (19.28%)|CurSamplesPerSec=2.24 |AvgSamplesPerSec=2.31 epoch: 0|step: 250|ppo_ep: 1|act_loss: 0.05792236328125|cri_loss: 0.0259246826171875|unsuper_loss: 0.0 average reward score: -2.8046875 ------------------------------------------------------------------------------------- |E2E latency=14.28s |Gather latency=1.04s (7.25%) |Generate time=9.15s (64.07%) |Training time=2.41s (16.84%) |Others=2.73 (19.09%)|CurSamplesPerSec=2.24 |AvgSamplesPerSec=2.31 epoch: 0|step: 251|ppo_ep: 1|act_loss: 0.00450897216796875|cri_loss: 0.03936767578125|unsuper_loss: 0.0 average reward score: -2.462890625 ------------------------------------------------------------------------------------- |E2E latency=13.91s |Gather latency=1.11s (8.00%) |Generate time=8.73s (62.77%) |Training time=2.40s (17.23%) |Others=2.78 (20.00%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.31 epoch: 0|step: 252|ppo_ep: 1|act_loss: 0.0938720703125|cri_loss: 0.025115966796875|unsuper_loss: 0.0 average reward score: -2.1875 ------------------------------------------------------------------------------------- |E2E latency=13.58s |Gather latency=0.81s (5.93%) |Generate time=8.64s (63.59%) |Training time=2.41s (17.71%) |Others=2.54 (18.70%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.31 epoch: 0|step: 253|ppo_ep: 1|act_loss: 0.1312255859375|cri_loss: 0.05450439453125|unsuper_loss: 0.0 average reward score: -2.66015625 ------------------------------------------------------------------------------------- |E2E latency=13.75s |Gather latency=0.77s (5.62%) |Generate time=8.80s (64.04%) |Training time=2.36s (17.14%) |Others=2.59 (18.82%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.31 epoch: 0|step: 254|ppo_ep: 1|act_loss: 0.024810791015625|cri_loss: 0.01302337646484375|unsuper_loss: 0.0 average reward score: -3.1640625 ------------------------------------------------------------------------------------- |E2E latency=14.11s |Gather latency=0.66s (4.67%) |Generate time=9.13s (64.68%) |Training time=2.34s (16.55%) |Others=2.65 (18.77%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.31 epoch: 0|step: 255|ppo_ep: 1|act_loss: 0.02935791015625|cri_loss: 0.0159149169921875|unsuper_loss: 0.0 average reward score: -2.720703125 ------------------------------------------------------------------------------------- |E2E latency=14.11s |Gather latency=0.79s (5.62%) |Generate time=9.11s (64.59%) |Training time=2.41s (17.05%) |Others=2.59 (18.36%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.31 epoch: 0|step: 256|ppo_ep: 1|act_loss: -0.01507568359375|cri_loss: 0.01560211181640625|unsuper_loss: 0.0 average reward score: -1.8369140625 ------------------------------------------------------------------------------------- |E2E latency=13.66s |Gather latency=1.19s (8.73%) |Generate time=8.61s (63.04%) |Training time=2.36s (17.28%) |Others=2.69 (19.68%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.31 epoch: 0|step: 257|ppo_ep: 1|act_loss: 0.056365966796875|cri_loss: 0.032684326171875|unsuper_loss: 0.0 average reward score: -2.6640625 ------------------------------------------------------------------------------------- |E2E latency=14.25s |Gather latency=1.18s (8.25%) |Generate time=9.20s (64.58%) |Training time=2.41s (16.91%) |Others=2.64 (18.51%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.31 epoch: 0|step: 258|ppo_ep: 1|act_loss: -0.01354217529296875|cri_loss: 0.0214385986328125|unsuper_loss: 0.0 average reward score: -2.1875 ------------------------------------------------------------------------------------- |E2E latency=13.70s |Gather latency=0.96s (7.04%) |Generate time=8.62s (62.97%) |Training time=2.40s (17.49%) |Others=2.68 (19.54%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.31 [2023-05-17 01:01:28,827] [INFO] [logging.py:96:log_dist] [Rank 0] step=260, skipped=4, lr=[8.877054741122905e-07, 8.877054741122905e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 01:01:28,827] [INFO] [timer.py:199:stop] epoch=0/micro_step=260/global_step=260, RunningAvgSamplesPerSec=28.286026217650896, CurrSamplesPerSec=28.042414922778633, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 01:01:29,582] [INFO] [logging.py:96:log_dist] [Rank 0] step=260, skipped=5, lr=[4.604488803736523e-06, 4.604488803736523e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 259|ppo_ep: 1|act_loss: -0.023956298828125|cri_loss: 0.02880859375|unsuper_loss: 0.0 average reward score: -1.3681640625 ------------------------------------------------------------------------------------- |E2E latency=14.20s |Gather latency=1.22s (8.57%) |Generate time=9.05s (63.75%) |Training time=2.40s (16.91%) |Others=2.75 (19.34%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.31 epoch: 0|step: 260|ppo_ep: 1|act_loss: 0.01500701904296875|cri_loss: 0.0194244384765625|unsuper_loss: 0.0 average reward score: -2.51953125 ------------------------------------------------------------------------------------- |E2E latency=13.69s |Gather latency=0.40s (2.95%) |Generate time=8.66s (63.27%) |Training time=2.36s (17.21%) |Others=2.67 (19.52%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.31 epoch: 0|step: 261|ppo_ep: 1|act_loss: -0.0517578125|cri_loss: 0.0565185546875|unsuper_loss: 0.0 average reward score: -2.59765625 ------------------------------------------------------------------------------------- |E2E latency=14.16s |Gather latency=0.63s (4.48%) |Generate time=8.92s (62.96%) |Training time=2.60s (18.37%) |Others=2.64 (18.67%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.31 epoch: 0|step: 262|ppo_ep: 1|act_loss: -0.07452392578125|cri_loss: 0.03955078125|unsuper_loss: 0.0 average reward score: -3.2265625 ------------------------------------------------------------------------------------- |E2E latency=14.24s |Gather latency=0.84s (5.90%) |Generate time=9.10s (63.91%) |Training time=2.40s (16.84%) |Others=2.74 (19.25%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.31 epoch: 0|step: 263|ppo_ep: 1|act_loss: -0.076416015625|cri_loss: 0.05548095703125|unsuper_loss: 0.0 average reward score: -2.642578125 ------------------------------------------------------------------------------------- |E2E latency=14.17s |Gather latency=0.93s (6.57%) |Generate time=9.11s (64.30%) |Training time=2.39s (16.87%) |Others=2.67 (18.83%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.31 epoch: 0|step: 264|ppo_ep: 1|act_loss: -0.09344482421875|cri_loss: 0.04766845703125|unsuper_loss: 0.0 average reward score: -2.59765625 ------------------------------------------------------------------------------------- |E2E latency=13.89s |Gather latency=0.67s (4.79%) |Generate time=8.61s (61.98%) |Training time=2.37s (17.03%) |Others=2.92 (20.99%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.31 epoch: 0|step: 265|ppo_ep: 1|act_loss: -0.034027099609375|cri_loss: 0.0394287109375|unsuper_loss: 0.0 average reward score: -3.091796875 ------------------------------------------------------------------------------------- |E2E latency=13.52s |Gather latency=0.67s (4.92%) |Generate time=8.67s (64.12%) |Training time=2.38s (17.57%) |Others=2.48 (18.31%)|CurSamplesPerSec=2.37 |AvgSamplesPerSec=2.31 epoch: 0|step: 266|ppo_ep: 1|act_loss: -0.005626678466796875|cri_loss: 0.03436279296875|unsuper_loss: 0.0 average reward score: -2.3828125 ------------------------------------------------------------------------------------- |E2E latency=14.15s |Gather latency=0.91s (6.40%) |Generate time=9.16s (64.72%) |Training time=2.38s (16.80%) |Others=2.61 (18.48%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.31 epoch: 0|step: 267|ppo_ep: 1|act_loss: 0.0254974365234375|cri_loss: 0.0311279296875|unsuper_loss: 0.0 average reward score: -1.78515625 ------------------------------------------------------------------------------------- |E2E latency=13.66s |Gather latency=0.73s (5.36%) |Generate time=8.63s (63.15%) |Training time=2.32s (16.97%) |Others=2.72 (19.88%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.31 epoch: 0|step: 268|ppo_ep: 1|act_loss: 0.0161590576171875|cri_loss: 0.028167724609375|unsuper_loss: 0.0 average reward score: -2.55859375 ------------------------------------------------------------------------------------- |E2E latency=13.59s |Gather latency=0.60s (4.42%) |Generate time=8.64s (63.56%) |Training time=2.36s (17.37%) |Others=2.59 (19.07%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.31 [2023-05-17 01:03:48,106] [INFO] [logging.py:96:log_dist] [Rank 0] step=270, skipped=4, lr=[8.777974129475923e-07, 8.777974129475923e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 01:03:48,106] [INFO] [timer.py:199:stop] epoch=0/micro_step=270/global_step=270, RunningAvgSamplesPerSec=28.262197857513897, CurrSamplesPerSec=27.372011552644597, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 01:03:48,877] [INFO] [logging.py:96:log_dist] [Rank 0] step=270, skipped=5, lr=[4.553432724122265e-06, 4.553432724122265e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 269|ppo_ep: 1|act_loss: 0.0021457672119140625|cri_loss: 0.031219482421875|unsuper_loss: 0.0 average reward score: -2.5 ------------------------------------------------------------------------------------- |E2E latency=14.21s |Gather latency=1.14s (8.05%) |Generate time=9.11s (64.09%) |Training time=2.46s (17.34%) |Others=2.64 (18.57%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.31 epoch: 0|step: 270|ppo_ep: 1|act_loss: 0.055419921875|cri_loss: 0.022247314453125|unsuper_loss: 0.0 average reward score: -3.255859375 ------------------------------------------------------------------------------------- |E2E latency=13.96s |Gather latency=0.83s (5.98%) |Generate time=8.91s (63.82%) |Training time=2.38s (17.05%) |Others=2.67 (19.13%)|CurSamplesPerSec=2.29 |AvgSamplesPerSec=2.31 epoch: 0|step: 271|ppo_ep: 1|act_loss: -0.047149658203125|cri_loss: 0.01532745361328125|unsuper_loss: 0.0 average reward score: -2.525390625 ------------------------------------------------------------------------------------- |E2E latency=13.65s |Gather latency=0.76s (5.59%) |Generate time=8.67s (63.46%) |Training time=2.27s (16.63%) |Others=2.72 (19.91%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.31 epoch: 0|step: 272|ppo_ep: 1|act_loss: 0.038848876953125|cri_loss: 0.0153656005859375|unsuper_loss: 0.0 average reward score: -2.796875 ------------------------------------------------------------------------------------- |E2E latency=14.28s |Gather latency=0.82s (5.72%) |Generate time=9.15s (64.06%) |Training time=2.35s (16.48%) |Others=2.78 (19.46%)|CurSamplesPerSec=2.24 |AvgSamplesPerSec=2.31 epoch: 0|step: 273|ppo_ep: 1|act_loss: 0.0631103515625|cri_loss: 0.014312744140625|unsuper_loss: 0.0 average reward score: -1.9404296875 ------------------------------------------------------------------------------------- |E2E latency=14.08s |Gather latency=0.80s (5.66%) |Generate time=9.12s (64.75%) |Training time=2.36s (16.74%) |Others=2.61 (18.51%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.31 epoch: 0|step: 274|ppo_ep: 1|act_loss: 0.049072265625|cri_loss: 0.039459228515625|unsuper_loss: 0.0 average reward score: -2.615234375 ------------------------------------------------------------------------------------- |E2E latency=14.01s |Gather latency=0.87s (6.24%) |Generate time=8.95s (63.88%) |Training time=2.33s (16.64%) |Others=2.73 (19.48%)|CurSamplesPerSec=2.28 |AvgSamplesPerSec=2.31 epoch: 0|step: 275|ppo_ep: 1|act_loss: 0.046478271484375|cri_loss: 0.0088653564453125|unsuper_loss: 0.0 average reward score: -1.7060546875 ------------------------------------------------------------------------------------- |E2E latency=15.90s |Gather latency=0.73s (4.59%) |Generate time=9.10s (57.26%) |Training time=4.20s (26.44%) |Others=2.59 (16.30%)|CurSamplesPerSec=2.01 |AvgSamplesPerSec=2.30 epoch: 0|step: 276|ppo_ep: 1|act_loss: 0.01026153564453125|cri_loss: 0.01360321044921875|unsuper_loss: 0.0 average reward score: -2.76953125 ------------------------------------------------------------------------------------- |E2E latency=14.40s |Gather latency=0.62s (4.31%) |Generate time=9.12s (63.36%) |Training time=2.46s (17.07%) |Others=2.82 (19.57%)|CurSamplesPerSec=2.22 |AvgSamplesPerSec=2.30 epoch: 0|step: 277|ppo_ep: 1|act_loss: -0.048797607421875|cri_loss: 0.0158538818359375|unsuper_loss: 0.0 average reward score: -2.408203125 ------------------------------------------------------------------------------------- |E2E latency=14.16s |Gather latency=1.03s (7.28%) |Generate time=9.11s (64.36%) |Training time=2.39s (16.91%) |Others=2.65 (18.73%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.30 epoch: 0|step: 278|ppo_ep: 1|act_loss: -0.045745849609375|cri_loss: 0.06573486328125|unsuper_loss: 0.0 average reward score: -2.53515625 ------------------------------------------------------------------------------------- |E2E latency=14.34s |Gather latency=0.97s (6.80%) |Generate time=9.21s (64.27%) |Training time=2.42s (16.88%) |Others=2.70 (18.85%)|CurSamplesPerSec=2.23 |AvgSamplesPerSec=2.30 [2023-05-17 01:06:10,600] [INFO] [logging.py:96:log_dist] [Rank 0] step=280, skipped=4, lr=[8.673544685316068e-07, 8.673544685316068e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 01:06:10,601] [INFO] [timer.py:199:stop] epoch=0/micro_step=280/global_step=280, RunningAvgSamplesPerSec=28.190958950069984, CurrSamplesPerSec=29.384409623239605, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 01:06:11,333] [INFO] [logging.py:96:log_dist] [Rank 0] step=280, skipped=5, lr=[4.499598111849299e-06, 4.499598111849299e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 279|ppo_ep: 1|act_loss: 0.00530242919921875|cri_loss: 0.0341796875|unsuper_loss: 0.0 average reward score: -2.123046875 ------------------------------------------------------------------------------------- |E2E latency=13.68s |Gather latency=1.01s (7.35%) |Generate time=8.64s (63.14%) |Training time=2.34s (17.12%) |Others=2.70 (19.74%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 280|ppo_ep: 1|act_loss: 0.04107666015625|cri_loss: 0.012603759765625|unsuper_loss: 0.0 average reward score: -3.26953125 ------------------------------------------------------------------------------------- |E2E latency=13.60s |Gather latency=0.98s (7.19%) |Generate time=8.67s (63.74%) |Training time=2.28s (16.76%) |Others=2.65 (19.49%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 281|ppo_ep: 1|act_loss: -0.06317138671875|cri_loss: 0.03155517578125|unsuper_loss: 0.0 average reward score: -1.9912109375 ------------------------------------------------------------------------------------- |E2E latency=13.62s |Gather latency=1.00s (7.33%) |Generate time=8.60s (63.17%) |Training time=2.37s (17.43%) |Others=2.64 (19.41%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 282|ppo_ep: 1|act_loss: 0.0214691162109375|cri_loss: 0.01163482666015625|unsuper_loss: 0.0 average reward score: -2.16796875 ------------------------------------------------------------------------------------- |E2E latency=13.50s |Gather latency=0.79s (5.82%) |Generate time=8.58s (63.57%) |Training time=2.39s (17.68%) |Others=2.53 (18.75%)|CurSamplesPerSec=2.37 |AvgSamplesPerSec=2.30 epoch: 0|step: 283|ppo_ep: 1|act_loss: -0.005859375|cri_loss: 0.0175933837890625|unsuper_loss: 0.0 average reward score: -2.19140625 ------------------------------------------------------------------------------------- |E2E latency=13.68s |Gather latency=1.05s (7.67%) |Generate time=8.72s (63.79%) |Training time=2.38s (17.39%) |Others=2.57 (18.82%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 284|ppo_ep: 1|act_loss: 0.0035190582275390625|cri_loss: 0.0171661376953125|unsuper_loss: 0.0 average reward score: -2.0625 ------------------------------------------------------------------------------------- |E2E latency=14.31s |Gather latency=1.18s (8.24%) |Generate time=9.20s (64.33%) |Training time=2.38s (16.62%) |Others=2.73 (19.05%)|CurSamplesPerSec=2.24 |AvgSamplesPerSec=2.30 epoch: 0|step: 285|ppo_ep: 1|act_loss: -0.01220703125|cri_loss: 0.0197296142578125|unsuper_loss: 0.0 average reward score: -1.90625 ------------------------------------------------------------------------------------- |E2E latency=14.08s |Gather latency=0.68s (4.84%) |Generate time=9.08s (64.53%) |Training time=2.32s (16.51%) |Others=2.67 (18.96%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.30 epoch: 0|step: 286|ppo_ep: 1|act_loss: -0.02667236328125|cri_loss: 0.0113983154296875|unsuper_loss: 0.0 average reward score: -2.28125 ------------------------------------------------------------------------------------- |E2E latency=14.18s |Gather latency=0.71s (4.99%) |Generate time=9.15s (64.53%) |Training time=2.38s (16.75%) |Others=2.66 (18.72%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.30 epoch: 0|step: 287|ppo_ep: 1|act_loss: -0.003406524658203125|cri_loss: 0.04693603515625|unsuper_loss: 0.0 average reward score: -2.16015625 ------------------------------------------------------------------------------------- |E2E latency=13.60s |Gather latency=1.14s (8.38%) |Generate time=8.72s (64.07%) |Training time=2.31s (16.95%) |Others=2.58 (18.98%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 288|ppo_ep: 1|act_loss: -0.062286376953125|cri_loss: 0.0316162109375|unsuper_loss: 0.0 average reward score: -3.01171875 ------------------------------------------------------------------------------------- |E2E latency=13.66s |Gather latency=0.94s (6.89%) |Generate time=8.69s (63.63%) |Training time=2.33s (17.08%) |Others=2.64 (19.30%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 [2023-05-17 01:08:28,980] [INFO] [logging.py:96:log_dist] [Rank 0] step=290, skipped=4, lr=[8.563907713794332e-07, 8.563907713794332e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 01:08:28,980] [INFO] [timer.py:199:stop] epoch=0/micro_step=290/global_step=290, RunningAvgSamplesPerSec=28.217939866304157, CurrSamplesPerSec=30.061530198552354, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 01:08:29,735] [INFO] [logging.py:96:log_dist] [Rank 0] step=290, skipped=5, lr=[4.443057811392445e-06, 4.443057811392445e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 289|ppo_ep: 1|act_loss: 0.021881103515625|cri_loss: 0.0171051025390625|unsuper_loss: 0.0 average reward score: -2.38671875 ------------------------------------------------------------------------------------- |E2E latency=14.17s |Gather latency=0.79s (5.60%) |Generate time=9.15s (64.59%) |Training time=2.36s (16.65%) |Others=2.66 (18.76%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.30 epoch: 0|step: 290|ppo_ep: 1|act_loss: 0.00943756103515625|cri_loss: 0.033966064453125|unsuper_loss: 0.0 average reward score: -2.3203125 ------------------------------------------------------------------------------------- |E2E latency=14.11s |Gather latency=0.89s (6.29%) |Generate time=9.15s (64.89%) |Training time=2.36s (16.75%) |Others=2.59 (18.36%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.30 epoch: 0|step: 291|ppo_ep: 1|act_loss: -0.0068511962890625|cri_loss: 0.03704833984375|unsuper_loss: 0.0 average reward score: -2.572265625 ------------------------------------------------------------------------------------- |E2E latency=13.65s |Gather latency=0.47s (3.43%) |Generate time=8.68s (63.64%) |Training time=2.34s (17.13%) |Others=2.62 (19.23%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 292|ppo_ep: 1|act_loss: 0.020782470703125|cri_loss: 0.070556640625|unsuper_loss: 0.0 average reward score: -2.1875 ------------------------------------------------------------------------------------- |E2E latency=14.15s |Gather latency=0.64s (4.53%) |Generate time=9.10s (64.31%) |Training time=2.40s (16.94%) |Others=2.65 (18.75%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.30 epoch: 0|step: 293|ppo_ep: 1|act_loss: -0.0029811859130859375|cri_loss: 0.0343017578125|unsuper_loss: 0.0 average reward score: -2.474609375 ------------------------------------------------------------------------------------- |E2E latency=14.09s |Gather latency=0.78s (5.53%) |Generate time=9.06s (64.30%) |Training time=2.36s (16.77%) |Others=2.67 (18.93%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.30 epoch: 0|step: 294|ppo_ep: 1|act_loss: 0.019439697265625|cri_loss: 0.016143798828125|unsuper_loss: 0.0 average reward score: -2.4296875 ------------------------------------------------------------------------------------- |E2E latency=14.27s |Gather latency=0.96s (6.72%) |Generate time=9.08s (63.61%) |Training time=2.40s (16.79%) |Others=2.80 (19.61%)|CurSamplesPerSec=2.24 |AvgSamplesPerSec=2.30 epoch: 0|step: 295|ppo_ep: 1|act_loss: -0.022735595703125|cri_loss: 0.0245513916015625|unsuper_loss: 0.0 average reward score: -2.3828125 ------------------------------------------------------------------------------------- |E2E latency=13.73s |Gather latency=1.03s (7.51%) |Generate time=8.70s (63.36%) |Training time=2.31s (16.82%) |Others=2.72 (19.82%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 296|ppo_ep: 1|act_loss: 0.021636962890625|cri_loss: 0.023101806640625|unsuper_loss: 0.0 average reward score: -1.99609375 ------------------------------------------------------------------------------------- |E2E latency=13.66s |Gather latency=0.56s (4.10%) |Generate time=8.67s (63.51%) |Training time=2.32s (17.00%) |Others=2.66 (19.49%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 297|ppo_ep: 1|act_loss: 0.0188446044921875|cri_loss: 0.0248565673828125|unsuper_loss: 0.0 average reward score: -2.369140625 ------------------------------------------------------------------------------------- |E2E latency=13.90s |Gather latency=0.82s (5.88%) |Generate time=8.85s (63.65%) |Training time=2.30s (16.57%) |Others=2.75 (19.77%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.30 epoch: 0|step: 298|ppo_ep: 1|act_loss: 0.03497314453125|cri_loss: 0.01293182373046875|unsuper_loss: 0.0 average reward score: -2.302734375 ------------------------------------------------------------------------------------- |E2E latency=14.08s |Gather latency=1.00s (7.08%) |Generate time=9.14s (64.95%) |Training time=2.33s (16.56%) |Others=2.60 (18.49%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.30 [2023-05-17 01:10:48,936] [INFO] [logging.py:96:log_dist] [Rank 0] step=300, skipped=4, lr=[8.449211566450259e-07, 8.449211566450259e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 01:10:48,936] [INFO] [timer.py:199:stop] epoch=0/micro_step=300/global_step=300, RunningAvgSamplesPerSec=28.234601623697724, CurrSamplesPerSec=28.412958069325658, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 01:10:49,670] [INFO] [logging.py:96:log_dist] [Rank 0] step=300, skipped=5, lr=[4.383888328336477e-06, 4.383888328336477e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 299|ppo_ep: 1|act_loss: 0.03790283203125|cri_loss: 0.0225067138671875|unsuper_loss: 0.0 average reward score: -2.77734375 ------------------------------------------------------------------------------------- |E2E latency=14.30s |Gather latency=0.77s (5.35%) |Generate time=9.17s (64.14%) |Training time=2.36s (16.48%) |Others=2.77 (19.38%)|CurSamplesPerSec=2.24 |AvgSamplesPerSec=2.30 epoch: 0|step: 300|ppo_ep: 1|act_loss: -0.08428955078125|cri_loss: 0.036468505859375|unsuper_loss: 0.0 average reward score: -1.78125 ------------------------------------------------------------------------------------- |E2E latency=13.76s |Gather latency=0.67s (4.90%) |Generate time=8.78s (63.78%) |Training time=2.29s (16.64%) |Others=2.69 (19.58%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 301|ppo_ep: 1|act_loss: -0.01904296875|cri_loss: 0.033660888671875|unsuper_loss: 0.0 average reward score: -2.58203125 ------------------------------------------------------------------------------------- |E2E latency=13.68s |Gather latency=0.82s (6.01%) |Generate time=8.68s (63.47%) |Training time=2.37s (17.31%) |Others=2.63 (19.22%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 302|ppo_ep: 1|act_loss: -0.029022216796875|cri_loss: 0.024749755859375|unsuper_loss: 0.0 average reward score: -2.25 ------------------------------------------------------------------------------------- |E2E latency=14.27s |Gather latency=1.24s (8.66%) |Generate time=9.14s (64.03%) |Training time=2.40s (16.82%) |Others=2.73 (19.15%)|CurSamplesPerSec=2.24 |AvgSamplesPerSec=2.30 epoch: 0|step: 303|ppo_ep: 1|act_loss: -0.053131103515625|cri_loss: 0.042755126953125|unsuper_loss: 0.0 average reward score: -2.703125 ------------------------------------------------------------------------------------- |E2E latency=13.67s |Gather latency=0.74s (5.44%) |Generate time=8.64s (63.21%) |Training time=2.33s (17.02%) |Others=2.70 (19.77%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 304|ppo_ep: 1|act_loss: 0.020050048828125|cri_loss: 0.04290771484375|unsuper_loss: 0.0 average reward score: -2.607421875 ------------------------------------------------------------------------------------- |E2E latency=13.73s |Gather latency=0.83s (6.03%) |Generate time=8.71s (63.45%) |Training time=2.38s (17.30%) |Others=2.64 (19.25%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 305|ppo_ep: 1|act_loss: -0.11114501953125|cri_loss: 0.2139892578125|unsuper_loss: 0.0 average reward score: -1.1708984375 ------------------------------------------------------------------------------------- |E2E latency=14.02s |Gather latency=0.71s (5.03%) |Generate time=9.14s (65.20%) |Training time=2.35s (16.76%) |Others=2.53 (18.04%)|CurSamplesPerSec=2.28 |AvgSamplesPerSec=2.30 epoch: 0|step: 306|ppo_ep: 1|act_loss: -0.040740966796875|cri_loss: 0.06494140625|unsuper_loss: 0.0 average reward score: -2.7578125 ------------------------------------------------------------------------------------- |E2E latency=14.23s |Gather latency=0.98s (6.91%) |Generate time=9.14s (64.23%) |Training time=2.40s (16.87%) |Others=2.69 (18.90%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.30 epoch: 0|step: 307|ppo_ep: 1|act_loss: -0.049713134765625|cri_loss: 0.02880859375|unsuper_loss: 0.0 average reward score: -2.57421875 ------------------------------------------------------------------------------------- |E2E latency=13.69s |Gather latency=0.58s (4.27%) |Generate time=8.63s (63.05%) |Training time=2.37s (17.28%) |Others=2.69 (19.67%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 308|ppo_ep: 1|act_loss: -0.07025146484375|cri_loss: 0.044830322265625|unsuper_loss: 0.0 average reward score: -2.3828125 ------------------------------------------------------------------------------------- |E2E latency=13.58s |Gather latency=0.72s (5.34%) |Generate time=8.64s (63.61%) |Training time=2.34s (17.25%) |Others=2.60 (19.14%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.30 [2023-05-17 01:13:07,155] [INFO] [logging.py:96:log_dist] [Rank 0] step=310, skipped=4, lr=[8.329611440475108e-07, 8.329611440475108e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 01:13:07,156] [INFO] [timer.py:199:stop] epoch=0/micro_step=310/global_step=310, RunningAvgSamplesPerSec=28.252965187570425, CurrSamplesPerSec=29.674091947908728, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 01:13:07,908] [INFO] [logging.py:96:log_dist] [Rank 0] step=310, skipped=5, lr=[4.322169725855191e-06, 4.322169725855191e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 309|ppo_ep: 1|act_loss: 0.01102447509765625|cri_loss: 0.0182037353515625|unsuper_loss: 0.0 average reward score: -2.12109375 ------------------------------------------------------------------------------------- |E2E latency=13.61s |Gather latency=1.28s (9.38%) |Generate time=8.65s (63.56%) |Training time=2.27s (16.67%) |Others=2.69 (19.76%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 310|ppo_ep: 1|act_loss: -0.0282440185546875|cri_loss: 0.031219482421875|unsuper_loss: 0.0 average reward score: -1.953125 ------------------------------------------------------------------------------------- |E2E latency=13.64s |Gather latency=0.51s (3.76%) |Generate time=8.60s (63.10%) |Training time=2.35s (17.21%) |Others=2.69 (19.69%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 311|ppo_ep: 1|act_loss: 0.044464111328125|cri_loss: 0.026123046875|unsuper_loss: 0.0 average reward score: -2.66015625 ------------------------------------------------------------------------------------- |E2E latency=14.06s |Gather latency=0.85s (6.02%) |Generate time=9.18s (65.27%) |Training time=2.36s (16.76%) |Others=2.53 (17.98%)|CurSamplesPerSec=2.28 |AvgSamplesPerSec=2.30 epoch: 0|step: 312|ppo_ep: 1|act_loss: 0.059967041015625|cri_loss: 0.0216827392578125|unsuper_loss: 0.0 average reward score: -2.19921875 ------------------------------------------------------------------------------------- |E2E latency=13.71s |Gather latency=0.86s (6.25%) |Generate time=8.62s (62.85%) |Training time=2.39s (17.44%) |Others=2.70 (19.71%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 313|ppo_ep: 1|act_loss: 0.10357666015625|cri_loss: 0.0160675048828125|unsuper_loss: 0.0 average reward score: -1.861328125 ------------------------------------------------------------------------------------- |E2E latency=13.75s |Gather latency=0.71s (5.19%) |Generate time=8.79s (63.93%) |Training time=2.32s (16.87%) |Others=2.64 (19.20%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 314|ppo_ep: 1|act_loss: 0.0450439453125|cri_loss: 0.06878662109375|unsuper_loss: 0.0 average reward score: -2.841796875 ------------------------------------------------------------------------------------- |E2E latency=13.67s |Gather latency=0.85s (6.26%) |Generate time=8.65s (63.30%) |Training time=2.39s (17.47%) |Others=2.63 (19.24%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 315|ppo_ep: 1|act_loss: -0.00799560546875|cri_loss: 0.01035308837890625|unsuper_loss: 0.0 average reward score: -2.2109375 ------------------------------------------------------------------------------------- |E2E latency=13.73s |Gather latency=1.19s (8.69%) |Generate time=8.66s (63.09%) |Training time=2.35s (17.10%) |Others=2.72 (19.81%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 316|ppo_ep: 1|act_loss: 0.039520263671875|cri_loss: 0.0243988037109375|unsuper_loss: 0.0 average reward score: -2.166015625 ------------------------------------------------------------------------------------- |E2E latency=14.00s |Gather latency=0.85s (6.07%) |Generate time=9.13s (65.24%) |Training time=2.39s (17.09%) |Others=2.48 (17.68%)|CurSamplesPerSec=2.29 |AvgSamplesPerSec=2.30 epoch: 0|step: 317|ppo_ep: 1|act_loss: 0.041595458984375|cri_loss: 0.0130615234375|unsuper_loss: 0.0 average reward score: -1.724609375 ------------------------------------------------------------------------------------- |E2E latency=13.76s |Gather latency=0.93s (6.79%) |Generate time=8.70s (63.21%) |Training time=2.34s (17.01%) |Others=2.72 (19.78%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 318|ppo_ep: 1|act_loss: -0.005039215087890625|cri_loss: 0.0199737548828125|unsuper_loss: 0.0 average reward score: -2.337890625 ------------------------------------------------------------------------------------- |E2E latency=14.25s |Gather latency=0.60s (4.23%) |Generate time=8.63s (60.61%) |Training time=3.07s (21.56%) |Others=2.54 (17.83%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.30 [2023-05-17 01:15:26,837] [INFO] [logging.py:96:log_dist] [Rank 0] step=320, skipped=4, lr=[8.20526916871205e-07, 8.20526916871205e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 01:15:26,838] [INFO] [timer.py:199:stop] epoch=0/micro_step=320/global_step=320, RunningAvgSamplesPerSec=28.21890563489276, CurrSamplesPerSec=28.07438855803927, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 01:15:27,603] [INFO] [logging.py:96:log_dist] [Rank 0] step=320, skipped=5, lr=[4.257985516376644e-06, 4.257985516376644e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 319|ppo_ep: 1|act_loss: -0.0284881591796875|cri_loss: 0.01456451416015625|unsuper_loss: 0.0 average reward score: -2.66015625 ------------------------------------------------------------------------------------- |E2E latency=15.12s |Gather latency=1.69s (11.16%) |Generate time=8.95s (59.19%) |Training time=2.39s (15.82%) |Others=3.78 (24.98%)|CurSamplesPerSec=2.12 |AvgSamplesPerSec=2.30 epoch: 0|step: 320|ppo_ep: 1|act_loss: -0.034271240234375|cri_loss: 0.01384735107421875|unsuper_loss: 0.0 average reward score: -2.6015625 ------------------------------------------------------------------------------------- |E2E latency=13.62s |Gather latency=1.10s (8.05%) |Generate time=8.65s (63.46%) |Training time=2.34s (17.16%) |Others=2.64 (19.38%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 [2023-05-17 01:15:55,067] [INFO] [loss_scaler.py:188:update_scale] [deepspeed] OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 65536, but hysteresis is 2. Reducing hysteresis to 1 epoch: 0|step: 321|ppo_ep: 1|act_loss: 0.00334930419921875|cri_loss: 0.0094451904296875|unsuper_loss: 0.0 average reward score: -3.0703125 ------------------------------------------------------------------------------------- |E2E latency=13.84s |Gather latency=1.16s (8.37%) |Generate time=8.88s (64.15%) |Training time=2.36s (17.02%) |Others=2.61 (18.83%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.30 [2023-05-17 01:16:09,276] [INFO] [loss_scaler.py:181:update_scale] [deepspeed] OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 65536, reducing to 32768 epoch: 0|step: 322|ppo_ep: 1|act_loss: 0.0274658203125|cri_loss: 0.0179290771484375|unsuper_loss: 0.0 average reward score: -2.0703125 ------------------------------------------------------------------------------------- |E2E latency=14.21s |Gather latency=1.28s (8.98%) |Generate time=9.04s (63.61%) |Training time=2.52s (17.71%) |Others=2.65 (18.68%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.30 epoch: 0|step: 323|ppo_ep: 1|act_loss: -0.048614501953125|cri_loss: 0.0272674560546875|unsuper_loss: 0.0 average reward score: -2.04296875 ------------------------------------------------------------------------------------- |E2E latency=14.39s |Gather latency=0.42s (2.92%) |Generate time=9.16s (63.66%) |Training time=2.52s (17.51%) |Others=2.71 (18.83%)|CurSamplesPerSec=2.22 |AvgSamplesPerSec=2.30 epoch: 0|step: 324|ppo_ep: 1|act_loss: 0.006839752197265625|cri_loss: 0.02655029296875|unsuper_loss: 0.0 average reward score: -2.27734375 ------------------------------------------------------------------------------------- |E2E latency=12.57s |Gather latency=0.68s (5.40%) |Generate time=7.64s (60.78%) |Training time=2.35s (18.73%) |Others=2.58 (20.50%)|CurSamplesPerSec=2.55 |AvgSamplesPerSec=2.30 epoch: 0|step: 325|ppo_ep: 1|act_loss: -0.04766845703125|cri_loss: 0.0302276611328125|unsuper_loss: 0.0 average reward score: -2.375 ------------------------------------------------------------------------------------- |E2E latency=14.28s |Gather latency=0.70s (4.88%) |Generate time=9.07s (63.50%) |Training time=2.38s (16.64%) |Others=2.83 (19.85%)|CurSamplesPerSec=2.24 |AvgSamplesPerSec=2.30 epoch: 0|step: 326|ppo_ep: 1|act_loss: -0.028564453125|cri_loss: 0.031982421875|unsuper_loss: 0.0 average reward score: -2.767578125 ------------------------------------------------------------------------------------- |E2E latency=13.85s |Gather latency=0.82s (5.91%) |Generate time=8.79s (63.49%) |Training time=2.35s (16.94%) |Others=2.71 (19.58%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.30 epoch: 0|step: 327|ppo_ep: 1|act_loss: -0.1295166015625|cri_loss: 0.0556640625|unsuper_loss: 0.0 average reward score: -1.9990234375 ------------------------------------------------------------------------------------- |E2E latency=14.03s |Gather latency=0.79s (5.62%) |Generate time=8.76s (62.44%) |Training time=2.45s (17.49%) |Others=2.82 (20.07%)|CurSamplesPerSec=2.28 |AvgSamplesPerSec=2.30 epoch: 0|step: 328|ppo_ep: 1|act_loss: -0.06475830078125|cri_loss: 0.0220184326171875|unsuper_loss: 0.0 average reward score: -2.078125 ------------------------------------------------------------------------------------- |E2E latency=13.69s |Gather latency=0.52s (3.78%) |Generate time=8.68s (63.37%) |Training time=2.43s (17.78%) |Others=2.58 (18.85%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 [2023-05-17 01:17:45,633] [INFO] [logging.py:96:log_dist] [Rank 0] step=330, skipped=4, lr=[8.076353000677563e-07, 8.076353000677563e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 01:17:45,633] [INFO] [timer.py:199:stop] epoch=0/micro_step=330/global_step=330, RunningAvgSamplesPerSec=28.225026998539036, CurrSamplesPerSec=27.563102439669525, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 01:17:46,385] [INFO] [logging.py:96:log_dist] [Rank 0] step=330, skipped=7, lr=[4.204921164949269e-06, 4.204921164949269e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 329|ppo_ep: 1|act_loss: -0.01465606689453125|cri_loss: 0.018341064453125|unsuper_loss: 0.0 average reward score: -2.33984375 ------------------------------------------------------------------------------------- |E2E latency=14.30s |Gather latency=0.88s (6.16%) |Generate time=9.26s (64.75%) |Training time=2.43s (16.96%) |Others=2.62 (18.29%)|CurSamplesPerSec=2.24 |AvgSamplesPerSec=2.30 epoch: 0|step: 330|ppo_ep: 1|act_loss: -0.04083251953125|cri_loss: 0.017242431640625|unsuper_loss: 0.0 average reward score: -1.96484375 ------------------------------------------------------------------------------------- |E2E latency=14.02s |Gather latency=0.75s (5.37%) |Generate time=8.75s (62.42%) |Training time=2.44s (17.39%) |Others=2.83 (20.19%)|CurSamplesPerSec=2.28 |AvgSamplesPerSec=2.30 epoch: 0|step: 331|ppo_ep: 1|act_loss: -0.014434814453125|cri_loss: 0.0217437744140625|unsuper_loss: 0.0 average reward score: -2.0390625 ------------------------------------------------------------------------------------- |E2E latency=14.24s |Gather latency=0.90s (6.34%) |Generate time=9.14s (64.18%) |Training time=2.42s (17.00%) |Others=2.68 (18.82%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.30 epoch: 0|step: 332|ppo_ep: 1|act_loss: -0.049224853515625|cri_loss: 0.0198822021484375|unsuper_loss: 0.0 average reward score: -2.052734375 ------------------------------------------------------------------------------------- |E2E latency=13.64s |Gather latency=0.59s (4.36%) |Generate time=8.70s (63.78%) |Training time=2.34s (17.16%) |Others=2.60 (19.06%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 333|ppo_ep: 1|act_loss: 0.0193939208984375|cri_loss: 0.0312347412109375|unsuper_loss: 0.0 average reward score: -2.248046875 ------------------------------------------------------------------------------------- |E2E latency=13.44s |Gather latency=0.75s (5.62%) |Generate time=8.58s (63.85%) |Training time=2.37s (17.61%) |Others=2.49 (18.54%)|CurSamplesPerSec=2.38 |AvgSamplesPerSec=2.30 epoch: 0|step: 334|ppo_ep: 1|act_loss: 0.0105743408203125|cri_loss: 0.031341552734375|unsuper_loss: 0.0 average reward score: -1.4326171875 ------------------------------------------------------------------------------------- |E2E latency=14.06s |Gather latency=1.04s (7.39%) |Generate time=9.03s (64.25%) |Training time=2.36s (16.78%) |Others=2.67 (18.97%)|CurSamplesPerSec=2.28 |AvgSamplesPerSec=2.30 epoch: 0|step: 335|ppo_ep: 1|act_loss: 0.00231170654296875|cri_loss: 0.049560546875|unsuper_loss: 0.0 average reward score: -1.8671875 ------------------------------------------------------------------------------------- |E2E latency=13.72s |Gather latency=0.90s (6.56%) |Generate time=8.62s (62.82%) |Training time=2.33s (16.96%) |Others=2.78 (20.23%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 336|ppo_ep: 1|act_loss: 0.021942138671875|cri_loss: 0.0377197265625|unsuper_loss: 0.0 average reward score: -2.169921875 ------------------------------------------------------------------------------------- |E2E latency=13.61s |Gather latency=0.41s (3.00%) |Generate time=8.65s (63.57%) |Training time=2.34s (17.19%) |Others=2.62 (19.24%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 337|ppo_ep: 1|act_loss: 0.036224365234375|cri_loss: 0.04754638671875|unsuper_loss: 0.0 average reward score: -1.880859375 ------------------------------------------------------------------------------------- |E2E latency=13.95s |Gather latency=0.52s (3.70%) |Generate time=9.02s (64.65%) |Training time=2.38s (17.06%) |Others=2.55 (18.29%)|CurSamplesPerSec=2.29 |AvgSamplesPerSec=2.30 epoch: 0|step: 338|ppo_ep: 1|act_loss: 0.0650634765625|cri_loss: 0.0615234375|unsuper_loss: 0.0 average reward score: -2.6484375 ------------------------------------------------------------------------------------- |E2E latency=13.64s |Gather latency=0.79s (5.80%) |Generate time=8.60s (63.02%) |Training time=2.32s (16.98%) |Others=2.73 (20.01%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 [2023-05-17 01:20:04,050] [INFO] [logging.py:96:log_dist] [Rank 0] step=340, skipped=4, lr=[7.943037374900319e-07, 7.943037374900319e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 01:20:04,051] [INFO] [timer.py:199:stop] epoch=0/micro_step=340/global_step=340, RunningAvgSamplesPerSec=28.23423757328371, CurrSamplesPerSec=28.353499895536856, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 01:20:04,767] [INFO] [logging.py:96:log_dist] [Rank 0] step=340, skipped=7, lr=[4.136519888601191e-06, 4.136519888601191e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 339|ppo_ep: 1|act_loss: 0.04241943359375|cri_loss: 0.028289794921875|unsuper_loss: 0.0 average reward score: -2.931640625 ------------------------------------------------------------------------------------- |E2E latency=14.06s |Gather latency=0.61s (4.35%) |Generate time=9.24s (65.74%) |Training time=2.36s (16.78%) |Others=2.46 (17.48%)|CurSamplesPerSec=2.28 |AvgSamplesPerSec=2.30 epoch: 0|step: 340|ppo_ep: 1|act_loss: 0.06463623046875|cri_loss: 0.02996826171875|unsuper_loss: 0.0 average reward score: -1.34765625 ------------------------------------------------------------------------------------- |E2E latency=14.13s |Gather latency=0.86s (6.05%) |Generate time=9.15s (64.76%) |Training time=2.35s (16.64%) |Others=2.63 (18.60%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.30 epoch: 0|step: 341|ppo_ep: 1|act_loss: 0.0657958984375|cri_loss: 0.02459716796875|unsuper_loss: 0.0 average reward score: -2.44140625 ------------------------------------------------------------------------------------- |E2E latency=13.60s |Gather latency=0.58s (4.27%) |Generate time=8.64s (63.51%) |Training time=2.40s (17.61%) |Others=2.57 (18.88%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 342|ppo_ep: 1|act_loss: -0.005939483642578125|cri_loss: 0.017120361328125|unsuper_loss: 0.0 average reward score: -1.7548828125 ------------------------------------------------------------------------------------- |E2E latency=14.12s |Gather latency=0.65s (4.62%) |Generate time=9.13s (64.71%) |Training time=2.37s (16.82%) |Others=2.61 (18.47%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.30 epoch: 0|step: 343|ppo_ep: 1|act_loss: -0.01041412353515625|cri_loss: 0.015106201171875|unsuper_loss: 0.0 average reward score: -1.341796875 ------------------------------------------------------------------------------------- |E2E latency=14.37s |Gather latency=0.79s (5.48%) |Generate time=9.16s (63.75%) |Training time=2.37s (16.48%) |Others=2.84 (19.77%)|CurSamplesPerSec=2.23 |AvgSamplesPerSec=2.30 epoch: 0|step: 344|ppo_ep: 1|act_loss: 0.0218353271484375|cri_loss: 0.026947021484375|unsuper_loss: 0.0 average reward score: -2.4453125 ------------------------------------------------------------------------------------- |E2E latency=13.59s |Gather latency=1.18s (8.70%) |Generate time=8.63s (63.50%) |Training time=2.34s (17.20%) |Others=2.62 (19.30%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 345|ppo_ep: 1|act_loss: -0.0214385986328125|cri_loss: 0.01111602783203125|unsuper_loss: 0.0 average reward score: -2.552734375 ------------------------------------------------------------------------------------- |E2E latency=14.12s |Gather latency=0.62s (4.40%) |Generate time=9.13s (64.65%) |Training time=2.38s (16.89%) |Others=2.61 (18.46%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.30 epoch: 0|step: 346|ppo_ep: 1|act_loss: 0.02392578125|cri_loss: 0.04608154296875|unsuper_loss: 0.0 average reward score: -3.3203125 ------------------------------------------------------------------------------------- |E2E latency=14.19s |Gather latency=0.74s (5.20%) |Generate time=9.16s (64.57%) |Training time=2.39s (16.82%) |Others=2.64 (18.61%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.30 epoch: 0|step: 347|ppo_ep: 1|act_loss: 0.044036865234375|cri_loss: 0.0285186767578125|unsuper_loss: 0.0 average reward score: -2.53125 ------------------------------------------------------------------------------------- |E2E latency=13.91s |Gather latency=0.51s (3.67%) |Generate time=8.85s (63.62%) |Training time=2.40s (17.26%) |Others=2.66 (19.12%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.30 epoch: 0|step: 348|ppo_ep: 1|act_loss: 0.0107269287109375|cri_loss: 0.01708984375|unsuper_loss: 0.0 average reward score: -2.01171875 ------------------------------------------------------------------------------------- |E2E latency=13.95s |Gather latency=0.67s (4.83%) |Generate time=8.73s (62.56%) |Training time=2.35s (16.82%) |Others=2.88 (20.63%)|CurSamplesPerSec=2.29 |AvgSamplesPerSec=2.30 [2023-05-17 01:22:23,730] [INFO] [logging.py:96:log_dist] [Rank 0] step=350, skipped=4, lr=[7.805502682885617e-07, 7.805502682885617e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 01:22:23,730] [INFO] [timer.py:199:stop] epoch=0/micro_step=350/global_step=350, RunningAvgSamplesPerSec=28.23457339448935, CurrSamplesPerSec=27.227822753651385, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 01:22:24,490] [INFO] [logging.py:96:log_dist] [Rank 0] step=350, skipped=7, lr=[4.0659042110196635e-06, 4.0659042110196635e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 349|ppo_ep: 1|act_loss: -0.0242462158203125|cri_loss: 0.02581787109375|unsuper_loss: 0.0 average reward score: -2.37890625 ------------------------------------------------------------------------------------- |E2E latency=13.75s |Gather latency=1.08s (7.88%) |Generate time=8.76s (63.71%) |Training time=2.44s (17.77%) |Others=2.55 (18.52%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 350|ppo_ep: 1|act_loss: -0.0740966796875|cri_loss: 0.02655029296875|unsuper_loss: 0.0 average reward score: -2.796875 ------------------------------------------------------------------------------------- |E2E latency=14.14s |Gather latency=0.78s (5.50%) |Generate time=9.12s (64.48%) |Training time=2.35s (16.64%) |Others=2.67 (18.88%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.30 epoch: 0|step: 351|ppo_ep: 1|act_loss: -0.025726318359375|cri_loss: 0.03076171875|unsuper_loss: 0.0 average reward score: -2.591796875 ------------------------------------------------------------------------------------- |E2E latency=13.72s |Gather latency=1.23s (8.94%) |Generate time=8.62s (62.83%) |Training time=2.45s (17.84%) |Others=2.65 (19.33%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 352|ppo_ep: 1|act_loss: -0.0697021484375|cri_loss: 0.017974853515625|unsuper_loss: 0.0 average reward score: -2.234375 ------------------------------------------------------------------------------------- |E2E latency=13.73s |Gather latency=1.18s (8.61%) |Generate time=8.69s (63.30%) |Training time=2.39s (17.39%) |Others=2.65 (19.30%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 353|ppo_ep: 1|act_loss: -0.08123779296875|cri_loss: 0.0487060546875|unsuper_loss: 0.0 average reward score: -2.46484375 ------------------------------------------------------------------------------------- |E2E latency=14.25s |Gather latency=0.98s (6.86%) |Generate time=9.13s (64.07%) |Training time=2.35s (16.51%) |Others=2.77 (19.42%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.30 epoch: 0|step: 354|ppo_ep: 1|act_loss: 0.020538330078125|cri_loss: 0.0113525390625|unsuper_loss: 0.0 average reward score: -2.400390625 ------------------------------------------------------------------------------------- |E2E latency=13.56s |Gather latency=0.73s (5.41%) |Generate time=8.61s (63.49%) |Training time=2.37s (17.45%) |Others=2.58 (19.06%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.30 epoch: 0|step: 355|ppo_ep: 1|act_loss: -0.01012420654296875|cri_loss: 0.0194854736328125|unsuper_loss: 0.0 average reward score: -2.6171875 ------------------------------------------------------------------------------------- |E2E latency=14.22s |Gather latency=0.73s (5.16%) |Generate time=9.10s (63.96%) |Training time=2.46s (17.27%) |Others=2.67 (18.77%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.30 epoch: 0|step: 356|ppo_ep: 1|act_loss: 0.1015625|cri_loss: 0.0467529296875|unsuper_loss: 0.0 average reward score: -2.208984375 ------------------------------------------------------------------------------------- |E2E latency=14.08s |Gather latency=0.67s (4.78%) |Generate time=9.07s (64.44%) |Training time=2.40s (17.03%) |Others=2.61 (18.53%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.30 epoch: 0|step: 357|ppo_ep: 1|act_loss: 0.07183837890625|cri_loss: 0.09112548828125|unsuper_loss: 0.0 average reward score: -2.76171875 ------------------------------------------------------------------------------------- |E2E latency=13.84s |Gather latency=0.77s (5.56%) |Generate time=8.71s (62.97%) |Training time=2.41s (17.39%) |Others=2.72 (19.64%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.30 epoch: 0|step: 358|ppo_ep: 1|act_loss: -0.0294189453125|cri_loss: 0.014862060546875|unsuper_loss: 0.0 average reward score: -2.0703125 ------------------------------------------------------------------------------------- |E2E latency=13.90s |Gather latency=0.92s (6.59%) |Generate time=8.71s (62.68%) |Training time=2.32s (16.67%) |Others=2.87 (20.65%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.30 [2023-05-17 01:24:43,298] [INFO] [logging.py:96:log_dist] [Rank 0] step=360, skipped=4, lr=[7.663935025024756e-07, 7.663935025024756e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 01:24:43,298] [INFO] [timer.py:199:stop] epoch=0/micro_step=360/global_step=360, RunningAvgSamplesPerSec=28.23351695526317, CurrSamplesPerSec=28.331002561274058, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 01:24:44,047] [INFO] [logging.py:96:log_dist] [Rank 0] step=360, skipped=7, lr=[3.993169683407347e-06, 3.993169683407347e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 359|ppo_ep: 1|act_loss: -0.034576416015625|cri_loss: 0.0552978515625|unsuper_loss: 0.0 average reward score: -2.611328125 ------------------------------------------------------------------------------------- |E2E latency=14.11s |Gather latency=0.65s (4.61%) |Generate time=9.15s (64.84%) |Training time=2.39s (16.91%) |Others=2.57 (18.24%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.30 epoch: 0|step: 360|ppo_ep: 1|act_loss: 0.02996826171875|cri_loss: 0.0194549560546875|unsuper_loss: 0.0 average reward score: -2.5625 ------------------------------------------------------------------------------------- |E2E latency=13.95s |Gather latency=0.74s (5.33%) |Generate time=8.68s (62.19%) |Training time=2.49s (17.86%) |Others=2.78 (19.95%)|CurSamplesPerSec=2.29 |AvgSamplesPerSec=2.30 epoch: 0|step: 361|ppo_ep: 1|act_loss: 0.067138671875|cri_loss: 0.033355712890625|unsuper_loss: 0.0 average reward score: -1.6806640625 ------------------------------------------------------------------------------------- |E2E latency=15.95s |Gather latency=0.72s (4.52%) |Generate time=9.16s (57.43%) |Training time=3.31s (20.75%) |Others=3.48 (21.82%)|CurSamplesPerSec=2.01 |AvgSamplesPerSec=2.30 epoch: 0|step: 362|ppo_ep: 1|act_loss: -0.0292510986328125|cri_loss: 0.0232086181640625|unsuper_loss: 0.0 average reward score: -2.416015625 ------------------------------------------------------------------------------------- |E2E latency=14.09s |Gather latency=0.93s (6.63%) |Generate time=9.05s (64.22%) |Training time=2.42s (17.19%) |Others=2.62 (18.59%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.30 epoch: 0|step: 363|ppo_ep: 1|act_loss: 0.0139007568359375|cri_loss: 0.0190887451171875|unsuper_loss: 0.0 average reward score: -2.548828125 ------------------------------------------------------------------------------------- |E2E latency=14.05s |Gather latency=1.08s (7.67%) |Generate time=9.14s (65.04%) |Training time=2.34s (16.62%) |Others=2.58 (18.35%)|CurSamplesPerSec=2.28 |AvgSamplesPerSec=2.30 epoch: 0|step: 364|ppo_ep: 1|act_loss: 0.0157318115234375|cri_loss: 0.048980712890625|unsuper_loss: 0.0 average reward score: -1.2939453125 ------------------------------------------------------------------------------------- |E2E latency=14.24s |Gather latency=0.88s (6.18%) |Generate time=9.16s (64.30%) |Training time=2.42s (16.98%) |Others=2.67 (18.72%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.30 epoch: 0|step: 365|ppo_ep: 1|act_loss: 0.0236663818359375|cri_loss: 0.042236328125|unsuper_loss: 0.0 average reward score: -2.615234375 ------------------------------------------------------------------------------------- |E2E latency=14.31s |Gather latency=0.45s (3.16%) |Generate time=9.06s (63.31%) |Training time=2.43s (17.01%) |Others=2.82 (19.67%)|CurSamplesPerSec=2.24 |AvgSamplesPerSec=2.30 epoch: 0|step: 366|ppo_ep: 1|act_loss: -0.03607177734375|cri_loss: 0.0302581787109375|unsuper_loss: 0.0 average reward score: -2.771484375 ------------------------------------------------------------------------------------- |E2E latency=14.12s |Gather latency=0.53s (3.79%) |Generate time=9.16s (64.84%) |Training time=2.41s (17.08%) |Others=2.55 (18.08%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.30 epoch: 0|step: 367|ppo_ep: 1|act_loss: -0.0906982421875|cri_loss: 0.0518798828125|unsuper_loss: 0.0 average reward score: -1.923828125 ------------------------------------------------------------------------------------- |E2E latency=13.88s |Gather latency=0.92s (6.59%) |Generate time=8.71s (62.77%) |Training time=2.42s (17.40%) |Others=2.75 (19.82%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.30 epoch: 0|step: 368|ppo_ep: 1|act_loss: -0.01399993896484375|cri_loss: 0.031646728515625|unsuper_loss: 0.0 average reward score: -2.015625 ------------------------------------------------------------------------------------- |E2E latency=13.63s |Gather latency=0.87s (6.38%) |Generate time=8.63s (63.30%) |Training time=2.38s (17.46%) |Others=2.62 (19.24%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 [2023-05-17 01:27:05,730] [INFO] [logging.py:96:log_dist] [Rank 0] step=370, skipped=4, lr=[7.518525958779622e-07, 7.518525958779622e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 01:27:05,731] [INFO] [timer.py:199:stop] epoch=0/micro_step=370/global_step=370, RunningAvgSamplesPerSec=28.22745299957415, CurrSamplesPerSec=29.608780689683165, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 01:27:06,453] [INFO] [logging.py:96:log_dist] [Rank 0] step=370, skipped=7, lr=[3.918414724016767e-06, 3.918414724016767e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 369|ppo_ep: 1|act_loss: 0.0034503936767578125|cri_loss: 0.024810791015625|unsuper_loss: 0.0 average reward score: -3.080078125 ------------------------------------------------------------------------------------- |E2E latency=14.18s |Gather latency=0.93s (6.56%) |Generate time=9.19s (64.83%) |Training time=2.33s (16.46%) |Others=2.65 (18.71%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.30 epoch: 0|step: 370|ppo_ep: 1|act_loss: -0.01873779296875|cri_loss: 0.07208251953125|unsuper_loss: 0.0 average reward score: -2.2578125 ------------------------------------------------------------------------------------- |E2E latency=14.42s |Gather latency=0.78s (5.40%) |Generate time=9.11s (63.22%) |Training time=2.40s (16.62%) |Others=2.90 (20.15%)|CurSamplesPerSec=2.22 |AvgSamplesPerSec=2.30 epoch: 0|step: 371|ppo_ep: 1|act_loss: -0.0239105224609375|cri_loss: 0.01885986328125|unsuper_loss: 0.0 average reward score: -1.9921875 ------------------------------------------------------------------------------------- |E2E latency=13.68s |Gather latency=0.71s (5.17%) |Generate time=8.64s (63.17%) |Training time=2.44s (17.85%) |Others=2.60 (18.98%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 372|ppo_ep: 1|act_loss: -0.0169677734375|cri_loss: 0.01551055908203125|unsuper_loss: 0.0 average reward score: -1.984375 ------------------------------------------------------------------------------------- |E2E latency=13.84s |Gather latency=1.04s (7.54%) |Generate time=8.73s (63.07%) |Training time=2.34s (16.90%) |Others=2.77 (20.03%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.30 epoch: 0|step: 373|ppo_ep: 1|act_loss: -0.00711822509765625|cri_loss: 0.04022216796875|unsuper_loss: 0.0 average reward score: -2.080078125 ------------------------------------------------------------------------------------- |E2E latency=13.74s |Gather latency=0.62s (4.49%) |Generate time=8.84s (64.32%) |Training time=2.32s (16.86%) |Others=2.59 (18.82%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 374|ppo_ep: 1|act_loss: -0.058013916015625|cri_loss: 0.01812744140625|unsuper_loss: 0.0 average reward score: -1.8583984375 ------------------------------------------------------------------------------------- |E2E latency=13.67s |Gather latency=1.09s (8.00%) |Generate time=8.68s (63.48%) |Training time=2.35s (17.20%) |Others=2.64 (19.32%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 375|ppo_ep: 1|act_loss: -0.01189422607421875|cri_loss: 0.01202392578125|unsuper_loss: 0.0 average reward score: -2.634765625 ------------------------------------------------------------------------------------- |E2E latency=14.29s |Gather latency=0.74s (5.17%) |Generate time=9.16s (64.14%) |Training time=2.42s (16.91%) |Others=2.71 (18.95%)|CurSamplesPerSec=2.24 |AvgSamplesPerSec=2.30 epoch: 0|step: 376|ppo_ep: 1|act_loss: -0.01441192626953125|cri_loss: 0.0111236572265625|unsuper_loss: 0.0 average reward score: -2.328125 ------------------------------------------------------------------------------------- |E2E latency=13.79s |Gather latency=0.80s (5.80%) |Generate time=8.68s (62.97%) |Training time=2.39s (17.33%) |Others=2.72 (19.70%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 [2023-05-17 01:28:56,818] [INFO] [loss_scaler.py:188:update_scale] [deepspeed] OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 65536, but hysteresis is 2. Reducing hysteresis to 1 epoch: 0|step: 377|ppo_ep: 1|act_loss: 0.125732421875|cri_loss: 0.061859130859375|unsuper_loss: 0.0 average reward score: -2.1015625 ------------------------------------------------------------------------------------- |E2E latency=13.72s |Gather latency=0.84s (6.10%) |Generate time=8.72s (63.55%) |Training time=2.30s (16.73%) |Others=2.71 (19.71%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 [2023-05-17 01:29:11,036] [INFO] [loss_scaler.py:181:update_scale] [deepspeed] OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 65536, reducing to 32768 epoch: 0|step: 378|ppo_ep: 1|act_loss: 0.0082855224609375|cri_loss: 0.0131683349609375|unsuper_loss: 0.0 average reward score: -2.236328125 ------------------------------------------------------------------------------------- |E2E latency=14.15s |Gather latency=0.46s (3.24%) |Generate time=9.15s (64.63%) |Training time=2.42s (17.09%) |Others=2.59 (18.28%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.30 [2023-05-17 01:29:25,342] [INFO] [logging.py:96:log_dist] [Rank 0] step=380, skipped=6, lr=[7.399564943446002e-07, 7.399564943446002e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 01:29:25,343] [INFO] [timer.py:199:stop] epoch=0/micro_step=380/global_step=380, RunningAvgSamplesPerSec=28.23025573201384, CurrSamplesPerSec=27.471651955991398, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 01:29:26,070] [INFO] [logging.py:96:log_dist] [Rank 0] step=380, skipped=7, lr=[3.841740484979002e-06, 3.841740484979002e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 379|ppo_ep: 1|act_loss: 0.043060302734375|cri_loss: 0.017852783203125|unsuper_loss: 0.0 average reward score: -2.17578125 ------------------------------------------------------------------------------------- |E2E latency=14.33s |Gather latency=0.80s (5.61%) |Generate time=9.16s (63.97%) |Training time=2.55s (17.82%) |Others=2.61 (18.21%)|CurSamplesPerSec=2.23 |AvgSamplesPerSec=2.30 epoch: 0|step: 380|ppo_ep: 1|act_loss: 0.06591796875|cri_loss: 0.01146697998046875|unsuper_loss: 0.0 average reward score: -1.9794921875 ------------------------------------------------------------------------------------- |E2E latency=14.04s |Gather latency=0.59s (4.23%) |Generate time=8.72s (62.09%) |Training time=2.42s (17.24%) |Others=2.90 (20.67%)|CurSamplesPerSec=2.28 |AvgSamplesPerSec=2.30 epoch: 0|step: 381|ppo_ep: 1|act_loss: 0.08551025390625|cri_loss: 0.0127105712890625|unsuper_loss: 0.0 average reward score: -2.921875 ------------------------------------------------------------------------------------- |E2E latency=13.66s |Gather latency=0.59s (4.30%) |Generate time=8.68s (63.52%) |Training time=2.38s (17.42%) |Others=2.60 (19.06%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 382|ppo_ep: 1|act_loss: 0.06005859375|cri_loss: 0.0137176513671875|unsuper_loss: 0.0 average reward score: -2.12890625 ------------------------------------------------------------------------------------- |E2E latency=13.54s |Gather latency=0.58s (4.31%) |Generate time=8.65s (63.88%) |Training time=2.30s (17.00%) |Others=2.59 (19.13%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.30 epoch: 0|step: 383|ppo_ep: 1|act_loss: -0.030059814453125|cri_loss: 0.02569580078125|unsuper_loss: 0.0 average reward score: -2.16015625 ------------------------------------------------------------------------------------- |E2E latency=13.77s |Gather latency=0.63s (4.55%) |Generate time=8.68s (63.03%) |Training time=2.42s (17.57%) |Others=2.67 (19.40%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 384|ppo_ep: 1|act_loss: -0.0027446746826171875|cri_loss: 0.00439453125|unsuper_loss: 0.0 average reward score: -2.177734375 ------------------------------------------------------------------------------------- |E2E latency=13.62s |Gather latency=0.64s (4.70%) |Generate time=8.66s (63.58%) |Training time=2.33s (17.10%) |Others=2.63 (19.32%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 385|ppo_ep: 1|act_loss: 0.0264892578125|cri_loss: 0.038787841796875|unsuper_loss: 0.0 average reward score: -2.53125 ------------------------------------------------------------------------------------- |E2E latency=13.78s |Gather latency=0.76s (5.50%) |Generate time=8.72s (63.31%) |Training time=2.33s (16.94%) |Others=2.72 (19.75%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 386|ppo_ep: 1|act_loss: -0.01117706298828125|cri_loss: 0.01090240478515625|unsuper_loss: 0.0 average reward score: -2.466796875 ------------------------------------------------------------------------------------- |E2E latency=14.12s |Gather latency=0.44s (3.11%) |Generate time=9.10s (64.47%) |Training time=2.42s (17.15%) |Others=2.60 (18.38%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.30 epoch: 0|step: 387|ppo_ep: 1|act_loss: 0.023681640625|cri_loss: 0.0180511474609375|unsuper_loss: 0.0 average reward score: -2.0625 ------------------------------------------------------------------------------------- |E2E latency=13.64s |Gather latency=0.57s (4.21%) |Generate time=8.63s (63.29%) |Training time=2.31s (16.96%) |Others=2.69 (19.75%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 388|ppo_ep: 1|act_loss: 0.05224609375|cri_loss: 0.04449462890625|unsuper_loss: 0.0 average reward score: -1.439453125 ------------------------------------------------------------------------------------- |E2E latency=13.77s |Gather latency=0.43s (3.14%) |Generate time=8.62s (62.57%) |Training time=2.34s (16.98%) |Others=2.81 (20.44%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 [2023-05-17 01:31:42,770] [INFO] [logging.py:96:log_dist] [Rank 0] step=390, skipped=6, lr=[7.247740488727002e-07, 7.247740488727002e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 01:31:42,770] [INFO] [timer.py:199:stop] epoch=0/micro_step=390/global_step=390, RunningAvgSamplesPerSec=28.231743426769423, CurrSamplesPerSec=28.67449221660867, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 01:31:43,514] [INFO] [logging.py:96:log_dist] [Rank 0] step=390, skipped=7, lr=[3.763250715433111e-06, 3.763250715433111e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 389|ppo_ep: 1|act_loss: 0.006824493408203125|cri_loss: 0.0174102783203125|unsuper_loss: 0.0 average reward score: -1.8330078125 ------------------------------------------------------------------------------------- |E2E latency=13.51s |Gather latency=0.52s (3.81%) |Generate time=8.64s (63.91%) |Training time=2.37s (17.57%) |Others=2.50 (18.52%)|CurSamplesPerSec=2.37 |AvgSamplesPerSec=2.30 epoch: 0|step: 390|ppo_ep: 1|act_loss: -0.004055023193359375|cri_loss: 0.01287841796875|unsuper_loss: 0.0 average reward score: -1.869140625 ------------------------------------------------------------------------------------- |E2E latency=13.68s |Gather latency=0.70s (5.15%) |Generate time=8.73s (63.78%) |Training time=2.36s (17.22%) |Others=2.60 (18.99%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 391|ppo_ep: 1|act_loss: 0.04522705078125|cri_loss: 0.012786865234375|unsuper_loss: 0.0 average reward score: -2.171875 ------------------------------------------------------------------------------------- |E2E latency=13.77s |Gather latency=0.76s (5.51%) |Generate time=8.63s (62.64%) |Training time=2.39s (17.38%) |Others=2.75 (19.99%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 392|ppo_ep: 1|act_loss: 0.0131683349609375|cri_loss: 0.006885528564453125|unsuper_loss: 0.0 average reward score: -1.779296875 ------------------------------------------------------------------------------------- |E2E latency=14.20s |Gather latency=0.61s (4.30%) |Generate time=9.17s (64.56%) |Training time=2.43s (17.08%) |Others=2.61 (18.36%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.30 epoch: 0|step: 393|ppo_ep: 1|act_loss: 0.0226898193359375|cri_loss: 0.015045166015625|unsuper_loss: 0.0 average reward score: -1.84765625 ------------------------------------------------------------------------------------- |E2E latency=13.80s |Gather latency=0.86s (6.24%) |Generate time=8.62s (62.49%) |Training time=2.34s (16.95%) |Others=2.84 (20.56%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 394|ppo_ep: 1|act_loss: 0.0282745361328125|cri_loss: 0.01403045654296875|unsuper_loss: 0.0 average reward score: -2.1328125 ------------------------------------------------------------------------------------- |E2E latency=13.61s |Gather latency=0.78s (5.72%) |Generate time=8.69s (63.87%) |Training time=2.32s (17.06%) |Others=2.60 (19.07%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 395|ppo_ep: 1|act_loss: -0.00091552734375|cri_loss: 0.01495361328125|unsuper_loss: 0.0 average reward score: -2.64453125 ------------------------------------------------------------------------------------- |E2E latency=13.05s |Gather latency=0.74s (5.68%) |Generate time=8.17s (62.56%) |Training time=2.26s (17.29%) |Others=2.63 (20.16%)|CurSamplesPerSec=2.45 |AvgSamplesPerSec=2.30 epoch: 0|step: 396|ppo_ep: 1|act_loss: -0.00687408447265625|cri_loss: 0.0245361328125|unsuper_loss: 0.0 average reward score: -2.7109375 ------------------------------------------------------------------------------------- |E2E latency=13.75s |Gather latency=0.72s (5.22%) |Generate time=8.62s (62.72%) |Training time=2.41s (17.50%) |Others=2.72 (19.78%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 397|ppo_ep: 1|act_loss: 0.0357666015625|cri_loss: 0.030517578125|unsuper_loss: 0.0 average reward score: -1.3798828125 ------------------------------------------------------------------------------------- |E2E latency=14.26s |Gather latency=0.39s (2.75%) |Generate time=9.18s (64.36%) |Training time=2.39s (16.79%) |Others=2.69 (18.85%)|CurSamplesPerSec=2.24 |AvgSamplesPerSec=2.30 epoch: 0|step: 398|ppo_ep: 1|act_loss: -0.029083251953125|cri_loss: 0.0255126953125|unsuper_loss: 0.0 average reward score: -1.701171875 ------------------------------------------------------------------------------------- |E2E latency=14.23s |Gather latency=0.55s (3.84%) |Generate time=9.14s (64.24%) |Training time=2.38s (16.76%) |Others=2.70 (19.00%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.30 [2023-05-17 01:34:01,293] [INFO] [logging.py:96:log_dist] [Rank 0] step=400, skipped=6, lr=[7.09263778510682e-07, 7.09263778510682e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 01:34:01,293] [INFO] [timer.py:199:stop] epoch=0/micro_step=400/global_step=400, RunningAvgSamplesPerSec=28.243072066420073, CurrSamplesPerSec=28.033219363341487, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 01:34:02,011] [INFO] [logging.py:96:log_dist] [Rank 0] step=400, skipped=7, lr=[3.6830516211415224e-06, 3.6830516211415224e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 399|ppo_ep: 1|act_loss: 0.023284912109375|cri_loss: 0.01727294921875|unsuper_loss: 0.0 average reward score: -2.712890625 ------------------------------------------------------------------------------------- |E2E latency=14.14s |Gather latency=0.64s (4.53%) |Generate time=9.05s (63.99%) |Training time=2.44s (17.23%) |Others=2.66 (18.78%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.30 epoch: 0|step: 400|ppo_ep: 1|act_loss: 0.02392578125|cri_loss: 0.0190277099609375|unsuper_loss: 0.0 average reward score: -2.455078125 ------------------------------------------------------------------------------------- |E2E latency=13.82s |Gather latency=0.75s (5.40%) |Generate time=8.74s (63.25%) |Training time=2.34s (16.97%) |Others=2.73 (19.78%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 401|ppo_ep: 1|act_loss: 0.0271759033203125|cri_loss: 0.015869140625|unsuper_loss: 0.0 average reward score: -2.298828125 ------------------------------------------------------------------------------------- |E2E latency=13.82s |Gather latency=0.86s (6.22%) |Generate time=8.61s (62.35%) |Training time=2.39s (17.27%) |Others=2.82 (20.38%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 402|ppo_ep: 1|act_loss: -0.07000732421875|cri_loss: 0.01458740234375|unsuper_loss: 0.0 average reward score: -2.4375 ------------------------------------------------------------------------------------- |E2E latency=13.53s |Gather latency=0.59s (4.37%) |Generate time=8.63s (63.77%) |Training time=2.33s (17.21%) |Others=2.57 (19.02%)|CurSamplesPerSec=2.37 |AvgSamplesPerSec=2.30 epoch: 0|step: 403|ppo_ep: 1|act_loss: -0.0299835205078125|cri_loss: 0.033966064453125|unsuper_loss: 0.0 average reward score: -1.607421875 ------------------------------------------------------------------------------------- |E2E latency=13.60s |Gather latency=0.93s (6.82%) |Generate time=8.65s (63.65%) |Training time=2.36s (17.38%) |Others=2.58 (18.97%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 404|ppo_ep: 1|act_loss: -0.0284423828125|cri_loss: 0.01534271240234375|unsuper_loss: 0.0 average reward score: -2.380859375 ------------------------------------------------------------------------------------- |E2E latency=14.55s |Gather latency=0.71s (4.90%) |Generate time=8.70s (59.76%) |Training time=2.40s (16.51%) |Others=3.45 (23.73%)|CurSamplesPerSec=2.20 |AvgSamplesPerSec=2.30 epoch: 0|step: 405|ppo_ep: 1|act_loss: 0.031463623046875|cri_loss: 0.0177154541015625|unsuper_loss: 0.0 average reward score: -2.255859375 ------------------------------------------------------------------------------------- |E2E latency=14.30s |Gather latency=0.81s (5.65%) |Generate time=9.12s (63.79%) |Training time=2.30s (16.09%) |Others=2.88 (20.12%)|CurSamplesPerSec=2.24 |AvgSamplesPerSec=2.30 epoch: 0|step: 406|ppo_ep: 1|act_loss: -0.03167724609375|cri_loss: 0.013427734375|unsuper_loss: 0.0 average reward score: -1.71875 ------------------------------------------------------------------------------------- |E2E latency=14.34s |Gather latency=0.82s (5.72%) |Generate time=9.18s (64.00%) |Training time=2.31s (16.09%) |Others=2.85 (19.90%)|CurSamplesPerSec=2.23 |AvgSamplesPerSec=2.30 epoch: 0|step: 407|ppo_ep: 1|act_loss: 0.00881195068359375|cri_loss: 0.010711669921875|unsuper_loss: 0.0 average reward score: -2.71484375 ------------------------------------------------------------------------------------- |E2E latency=13.63s |Gather latency=0.67s (4.95%) |Generate time=8.62s (63.22%) |Training time=2.34s (17.19%) |Others=2.67 (19.59%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 408|ppo_ep: 1|act_loss: -0.0482177734375|cri_loss: 0.0158233642578125|unsuper_loss: 0.0 average reward score: -1.6865234375 ------------------------------------------------------------------------------------- |E2E latency=14.13s |Gather latency=0.77s (5.42%) |Generate time=9.03s (63.89%) |Training time=2.38s (16.85%) |Others=2.72 (19.26%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.30 [2023-05-17 01:36:20,758] [INFO] [logging.py:96:log_dist] [Rank 0] step=410, skipped=6, lr=[6.934466704534219e-07, 6.934466704534219e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 01:36:20,759] [INFO] [timer.py:199:stop] epoch=0/micro_step=410/global_step=410, RunningAvgSamplesPerSec=28.252354885277775, CurrSamplesPerSec=28.221968636539593, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 01:36:21,501] [INFO] [logging.py:96:log_dist] [Rank 0] step=410, skipped=7, lr=[3.6012517207813124e-06, 3.6012517207813124e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 409|ppo_ep: 1|act_loss: 0.0114898681640625|cri_loss: 0.01030731201171875|unsuper_loss: 0.0 average reward score: -2.71875 ------------------------------------------------------------------------------------- |E2E latency=13.78s |Gather latency=0.69s (5.02%) |Generate time=8.65s (62.80%) |Training time=2.35s (17.08%) |Others=2.77 (20.12%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 410|ppo_ep: 1|act_loss: 0.0528564453125|cri_loss: 0.00921630859375|unsuper_loss: 0.0 average reward score: -2.412109375 ------------------------------------------------------------------------------------- |E2E latency=13.77s |Gather latency=0.77s (5.58%) |Generate time=8.64s (62.75%) |Training time=2.35s (17.06%) |Others=2.78 (20.19%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 411|ppo_ep: 1|act_loss: -0.062042236328125|cri_loss: 0.017333984375|unsuper_loss: 0.0 average reward score: -1.6435546875 ------------------------------------------------------------------------------------- |E2E latency=13.63s |Gather latency=0.62s (4.58%) |Generate time=8.61s (63.15%) |Training time=2.40s (17.58%) |Others=2.63 (19.27%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 412|ppo_ep: 1|act_loss: 0.08551025390625|cri_loss: 0.0100250244140625|unsuper_loss: 0.0 average reward score: -2.3046875 ------------------------------------------------------------------------------------- |E2E latency=13.97s |Gather latency=0.58s (4.15%) |Generate time=9.12s (65.29%) |Training time=2.35s (16.80%) |Others=2.50 (17.91%)|CurSamplesPerSec=2.29 |AvgSamplesPerSec=2.30 epoch: 0|step: 413|ppo_ep: 1|act_loss: 0.026611328125|cri_loss: 0.0116729736328125|unsuper_loss: 0.0 average reward score: -1.9521484375 ------------------------------------------------------------------------------------- |E2E latency=14.10s |Gather latency=0.73s (5.15%) |Generate time=9.13s (64.75%) |Training time=2.34s (16.58%) |Others=2.63 (18.67%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.30 epoch: 0|step: 414|ppo_ep: 1|act_loss: 0.0174560546875|cri_loss: 0.0148773193359375|unsuper_loss: 0.0 average reward score: -1.5966796875 ------------------------------------------------------------------------------------- |E2E latency=14.23s |Gather latency=1.10s (7.75%) |Generate time=9.10s (63.94%) |Training time=2.33s (16.40%) |Others=2.80 (19.66%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.30 epoch: 0|step: 415|ppo_ep: 1|act_loss: -0.0645751953125|cri_loss: 0.031524658203125|unsuper_loss: 0.0 average reward score: -1.689453125 ------------------------------------------------------------------------------------- |E2E latency=13.57s |Gather latency=0.73s (5.38%) |Generate time=8.60s (63.40%) |Training time=2.30s (16.94%) |Others=2.67 (19.66%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.30 epoch: 0|step: 416|ppo_ep: 1|act_loss: -0.0187530517578125|cri_loss: 0.005458831787109375|unsuper_loss: 0.0 average reward score: -2.123046875 ------------------------------------------------------------------------------------- |E2E latency=14.09s |Gather latency=0.78s (5.51%) |Generate time=9.02s (64.05%) |Training time=2.47s (17.51%) |Others=2.60 (18.44%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.30 epoch: 0|step: 417|ppo_ep: 1|act_loss: -9.500980377197266e-05|cri_loss: 0.0183258056640625|unsuper_loss: 0.0 average reward score: -1.7998046875 ------------------------------------------------------------------------------------- |E2E latency=13.65s |Gather latency=0.95s (6.94%) |Generate time=8.61s (63.09%) |Training time=2.39s (17.48%) |Others=2.65 (19.44%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 418|ppo_ep: 1|act_loss: -0.0012350082397460938|cri_loss: 0.0141143798828125|unsuper_loss: 0.0 average reward score: -1.6513671875 ------------------------------------------------------------------------------------- |E2E latency=13.75s |Gather latency=0.78s (5.69%) |Generate time=8.73s (63.53%) |Training time=2.43s (17.65%) |Others=2.59 (18.83%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 [2023-05-17 01:38:39,274] [INFO] [logging.py:96:log_dist] [Rank 0] step=420, skipped=6, lr=[6.773441270827885e-07, 6.773441270827885e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 01:38:39,274] [INFO] [timer.py:199:stop] epoch=0/micro_step=420/global_step=420, RunningAvgSamplesPerSec=28.259664626473022, CurrSamplesPerSec=27.540677255226086, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 01:38:40,061] [INFO] [logging.py:96:log_dist] [Rank 0] step=420, skipped=7, lr=[3.5179616991058513e-06, 3.5179616991058513e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 419|ppo_ep: 1|act_loss: 0.01910400390625|cri_loss: 0.00595855712890625|unsuper_loss: 0.0 average reward score: -1.6806640625 ------------------------------------------------------------------------------------- |E2E latency=13.80s |Gather latency=0.78s (5.67%) |Generate time=8.60s (62.31%) |Training time=2.45s (17.73%) |Others=2.75 (19.95%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 420|ppo_ep: 1|act_loss: -0.025360107421875|cri_loss: 0.014495849609375|unsuper_loss: 0.0 average reward score: -1.541015625 ------------------------------------------------------------------------------------- |E2E latency=14.00s |Gather latency=0.83s (5.96%) |Generate time=9.05s (64.62%) |Training time=2.45s (17.49%) |Others=2.51 (17.89%)|CurSamplesPerSec=2.29 |AvgSamplesPerSec=2.30 epoch: 0|step: 421|ppo_ep: 1|act_loss: -0.0195770263671875|cri_loss: 0.01324462890625|unsuper_loss: 0.0 average reward score: -1.8876953125 ------------------------------------------------------------------------------------- |E2E latency=14.26s |Gather latency=1.16s (8.12%) |Generate time=9.13s (64.02%) |Training time=2.42s (16.99%) |Others=2.71 (18.99%)|CurSamplesPerSec=2.24 |AvgSamplesPerSec=2.30 epoch: 0|step: 422|ppo_ep: 1|act_loss: -0.027557373046875|cri_loss: 0.021087646484375|unsuper_loss: 0.0 average reward score: -1.70703125 ------------------------------------------------------------------------------------- |E2E latency=14.22s |Gather latency=0.74s (5.19%) |Generate time=9.12s (64.09%) |Training time=2.39s (16.77%) |Others=2.72 (19.14%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.30 [2023-05-17 01:39:36,712] [INFO] [loss_scaler.py:188:update_scale] [deepspeed] OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 65536, but hysteresis is 2. Reducing hysteresis to 1 epoch: 0|step: 423|ppo_ep: 1|act_loss: -0.00959014892578125|cri_loss: 0.00838470458984375|unsuper_loss: 0.0 average reward score: -2.01953125 ------------------------------------------------------------------------------------- |E2E latency=14.16s |Gather latency=0.75s (5.33%) |Generate time=9.15s (64.58%) |Training time=2.36s (16.65%) |Others=2.66 (18.77%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.30 [2023-05-17 01:39:50,647] [INFO] [loss_scaler.py:181:update_scale] [deepspeed] OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 65536, reducing to 32768 epoch: 0|step: 424|ppo_ep: 1|act_loss: -0.060577392578125|cri_loss: 0.03466796875|unsuper_loss: 0.0 average reward score: -2.140625 ------------------------------------------------------------------------------------- |E2E latency=13.93s |Gather latency=0.82s (5.88%) |Generate time=8.62s (61.84%) |Training time=2.51s (18.01%) |Others=2.81 (20.16%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.30 epoch: 0|step: 425|ppo_ep: 1|act_loss: -0.01459503173828125|cri_loss: 0.007450103759765625|unsuper_loss: 0.0 average reward score: -1.9453125 ------------------------------------------------------------------------------------- |E2E latency=14.37s |Gather latency=0.84s (5.84%) |Generate time=9.19s (63.95%) |Training time=2.61s (18.14%) |Others=2.57 (17.90%)|CurSamplesPerSec=2.23 |AvgSamplesPerSec=2.30 epoch: 0|step: 426|ppo_ep: 1|act_loss: -0.05657958984375|cri_loss: 0.0238189697265625|unsuper_loss: 0.0 average reward score: -2.087890625 ------------------------------------------------------------------------------------- |E2E latency=12.75s |Gather latency=0.89s (6.97%) |Generate time=7.64s (59.93%) |Training time=2.46s (19.26%) |Others=2.65 (20.81%)|CurSamplesPerSec=2.51 |AvgSamplesPerSec=2.30 epoch: 0|step: 427|ppo_ep: 1|act_loss: -0.01287078857421875|cri_loss: 0.0197601318359375|unsuper_loss: 0.0 average reward score: -1.3486328125 ------------------------------------------------------------------------------------- |E2E latency=13.66s |Gather latency=0.79s (5.80%) |Generate time=8.71s (63.74%) |Training time=2.34s (17.13%) |Others=2.61 (19.13%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 428|ppo_ep: 1|act_loss: 0.0182342529296875|cri_loss: 0.026611328125|unsuper_loss: 0.0 average reward score: -2.265625 ------------------------------------------------------------------------------------- |E2E latency=14.03s |Gather latency=1.10s (7.82%) |Generate time=9.04s (64.47%) |Training time=2.29s (16.34%) |Others=2.69 (19.19%)|CurSamplesPerSec=2.28 |AvgSamplesPerSec=2.30 [2023-05-17 01:40:58,508] [INFO] [logging.py:96:log_dist] [Rank 0] step=430, skipped=6, lr=[6.60977937007738e-07, 6.60977937007738e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 01:40:58,509] [INFO] [timer.py:199:stop] epoch=0/micro_step=430/global_step=430, RunningAvgSamplesPerSec=28.257025696053958, CurrSamplesPerSec=29.151593895046403, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 01:40:59,270] [INFO] [logging.py:96:log_dist] [Rank 0] step=430, skipped=9, lr=[3.4503324656641074e-06, 3.4503324656641074e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 429|ppo_ep: 1|act_loss: 0.0182342529296875|cri_loss: 0.0173492431640625|unsuper_loss: 0.0 average reward score: -2.3125 ------------------------------------------------------------------------------------- |E2E latency=13.82s |Gather latency=0.65s (4.69%) |Generate time=8.73s (63.18%) |Training time=2.35s (17.01%) |Others=2.74 (19.82%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.30 epoch: 0|step: 430|ppo_ep: 1|act_loss: -0.068603515625|cri_loss: 0.0295257568359375|unsuper_loss: 0.0 average reward score: -1.33984375 ------------------------------------------------------------------------------------- |E2E latency=14.16s |Gather latency=0.95s (6.71%) |Generate time=9.17s (64.71%) |Training time=2.42s (17.12%) |Others=2.57 (18.17%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.30 epoch: 0|step: 431|ppo_ep: 1|act_loss: -0.006565093994140625|cri_loss: 0.0156707763671875|unsuper_loss: 0.0 average reward score: -2.2109375 ------------------------------------------------------------------------------------- |E2E latency=13.79s |Gather latency=0.52s (3.79%) |Generate time=8.75s (63.45%) |Training time=2.35s (17.01%) |Others=2.69 (19.54%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 432|ppo_ep: 1|act_loss: 0.02239990234375|cri_loss: 0.00981903076171875|unsuper_loss: 0.0 average reward score: -3.037109375 ------------------------------------------------------------------------------------- |E2E latency=14.18s |Gather latency=0.76s (5.33%) |Generate time=9.16s (64.59%) |Training time=2.35s (16.55%) |Others=2.67 (18.86%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.30 epoch: 0|step: 433|ppo_ep: 1|act_loss: -0.039337158203125|cri_loss: 0.01324462890625|unsuper_loss: 0.0 average reward score: -1.98828125 ------------------------------------------------------------------------------------- |E2E latency=13.68s |Gather latency=0.51s (3.75%) |Generate time=8.70s (63.58%) |Training time=2.36s (17.23%) |Others=2.63 (19.20%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 434|ppo_ep: 1|act_loss: -0.026092529296875|cri_loss: 0.01450347900390625|unsuper_loss: 0.0 average reward score: -2.365234375 ------------------------------------------------------------------------------------- |E2E latency=13.76s |Gather latency=1.04s (7.56%) |Generate time=8.70s (63.24%) |Training time=2.42s (17.59%) |Others=2.64 (19.17%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 435|ppo_ep: 1|act_loss: -0.0382080078125|cri_loss: 0.01031494140625|unsuper_loss: 0.0 average reward score: -2.22265625 ------------------------------------------------------------------------------------- |E2E latency=14.22s |Gather latency=0.44s (3.08%) |Generate time=9.16s (64.41%) |Training time=2.34s (16.42%) |Others=2.73 (19.16%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.30 epoch: 0|step: 436|ppo_ep: 1|act_loss: -0.0097503662109375|cri_loss: 0.0266265869140625|unsuper_loss: 0.0 average reward score: -1.8251953125 ------------------------------------------------------------------------------------- |E2E latency=13.79s |Gather latency=0.60s (4.33%) |Generate time=8.70s (63.13%) |Training time=2.34s (17.01%) |Others=2.74 (19.86%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 437|ppo_ep: 1|act_loss: 0.0809326171875|cri_loss: 0.01271820068359375|unsuper_loss: 0.0 average reward score: -2.318359375 ------------------------------------------------------------------------------------- |E2E latency=13.84s |Gather latency=0.78s (5.65%) |Generate time=8.64s (62.47%) |Training time=2.38s (17.22%) |Others=2.81 (20.30%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.30 epoch: 0|step: 438|ppo_ep: 1|act_loss: 0.02484130859375|cri_loss: 0.014373779296875|unsuper_loss: 0.0 average reward score: -2.0234375 ------------------------------------------------------------------------------------- |E2E latency=14.08s |Gather latency=0.93s (6.60%) |Generate time=9.12s (64.77%) |Training time=2.43s (17.27%) |Others=2.53 (17.96%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.30 [2023-05-17 01:43:17,731] [INFO] [logging.py:96:log_dist] [Rank 0] step=440, skipped=6, lr=[6.443702455817985e-07, 6.443702455817985e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 01:43:17,731] [INFO] [timer.py:199:stop] epoch=0/micro_step=440/global_step=440, RunningAvgSamplesPerSec=28.253096384400113, CurrSamplesPerSec=28.365112645555413, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 01:43:18,441] [INFO] [logging.py:96:log_dist] [Rank 0] step=440, skipped=9, lr=[3.364645489962566e-06, 3.364645489962566e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 439|ppo_ep: 1|act_loss: 0.020721435546875|cri_loss: 0.0146331787109375|unsuper_loss: 0.0 average reward score: -1.5810546875 ------------------------------------------------------------------------------------- |E2E latency=13.66s |Gather latency=0.81s (5.89%) |Generate time=8.65s (63.28%) |Training time=2.33s (17.04%) |Others=2.69 (19.68%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 440|ppo_ep: 1|act_loss: -0.035797119140625|cri_loss: 0.0185089111328125|unsuper_loss: 0.0 average reward score: -1.80078125 ------------------------------------------------------------------------------------- |E2E latency=13.66s |Gather latency=0.63s (4.61%) |Generate time=8.67s (63.50%) |Training time=2.37s (17.33%) |Others=2.62 (19.18%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 441|ppo_ep: 1|act_loss: 0.02313232421875|cri_loss: 0.02130126953125|unsuper_loss: 0.0 average reward score: -2.44921875 ------------------------------------------------------------------------------------- |E2E latency=13.61s |Gather latency=0.71s (5.23%) |Generate time=8.61s (63.26%) |Training time=2.38s (17.49%) |Others=2.62 (19.25%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 442|ppo_ep: 1|act_loss: 0.011077880859375|cri_loss: 0.0087432861328125|unsuper_loss: 0.0 average reward score: -2.3359375 ------------------------------------------------------------------------------------- |E2E latency=13.90s |Gather latency=0.85s (6.12%) |Generate time=8.72s (62.78%) |Training time=2.39s (17.18%) |Others=2.79 (20.05%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.30 epoch: 0|step: 443|ppo_ep: 1|act_loss: 0.01074981689453125|cri_loss: 0.03411865234375|unsuper_loss: 0.0 average reward score: -2.462890625 ------------------------------------------------------------------------------------- |E2E latency=14.16s |Gather latency=0.85s (5.98%) |Generate time=9.16s (64.68%) |Training time=2.39s (16.89%) |Others=2.61 (18.43%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.30 epoch: 0|step: 444|ppo_ep: 1|act_loss: -0.051910400390625|cri_loss: 0.06231689453125|unsuper_loss: 0.0 average reward score: -1.5087890625 ------------------------------------------------------------------------------------- |E2E latency=14.28s |Gather latency=0.86s (6.05%) |Generate time=9.13s (63.92%) |Training time=2.47s (17.28%) |Others=2.69 (18.80%)|CurSamplesPerSec=2.24 |AvgSamplesPerSec=2.30 epoch: 0|step: 445|ppo_ep: 1|act_loss: -0.036102294921875|cri_loss: 0.02020263671875|unsuper_loss: 0.0 average reward score: -1.9951171875 ------------------------------------------------------------------------------------- |E2E latency=13.79s |Gather latency=0.40s (2.90%) |Generate time=8.65s (62.73%) |Training time=2.47s (17.88%) |Others=2.68 (19.40%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 446|ppo_ep: 1|act_loss: -0.042449951171875|cri_loss: 0.01479339599609375|unsuper_loss: 0.0 average reward score: -2.609375 ------------------------------------------------------------------------------------- |E2E latency=13.89s |Gather latency=0.71s (5.13%) |Generate time=8.69s (62.55%) |Training time=2.42s (17.43%) |Others=2.78 (20.02%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.30 epoch: 0|step: 447|ppo_ep: 1|act_loss: -0.055999755859375|cri_loss: 0.0316162109375|unsuper_loss: 0.0 average reward score: -2.3046875 ------------------------------------------------------------------------------------- |E2E latency=13.77s |Gather latency=0.99s (7.21%) |Generate time=8.62s (62.55%) |Training time=2.38s (17.28%) |Others=2.78 (20.17%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 448|ppo_ep: 1|act_loss: -0.03802490234375|cri_loss: 0.024749755859375|unsuper_loss: 0.0 average reward score: -2.0 ------------------------------------------------------------------------------------- |E2E latency=13.85s |Gather latency=0.65s (4.71%) |Generate time=8.71s (62.90%) |Training time=2.34s (16.86%) |Others=2.80 (20.24%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.30 [2023-05-17 01:45:36,681] [INFO] [logging.py:96:log_dist] [Rank 0] step=450, skipped=6, lr=[6.275435249378385e-07, 6.275435249378385e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 01:45:36,682] [INFO] [timer.py:199:stop] epoch=0/micro_step=450/global_step=450, RunningAvgSamplesPerSec=28.252185439942828, CurrSamplesPerSec=27.69544645477015, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 01:45:37,467] [INFO] [logging.py:96:log_dist] [Rank 0] step=450, skipped=9, lr=[3.277788548620639e-06, 3.277788548620639e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 449|ppo_ep: 1|act_loss: 0.02667236328125|cri_loss: 0.0176849365234375|unsuper_loss: 0.0 average reward score: -2.15625 ------------------------------------------------------------------------------------- |E2E latency=14.09s |Gather latency=0.89s (6.30%) |Generate time=9.14s (64.85%) |Training time=2.39s (16.96%) |Others=2.56 (18.19%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.30 epoch: 0|step: 450|ppo_ep: 1|act_loss: 0.01227569580078125|cri_loss: 0.0159454345703125|unsuper_loss: 0.0 average reward score: -2.48828125 ------------------------------------------------------------------------------------- |E2E latency=13.59s |Gather latency=0.84s (6.16%) |Generate time=8.63s (63.52%) |Training time=2.35s (17.27%) |Others=2.61 (19.21%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 451|ppo_ep: 1|act_loss: -0.017608642578125|cri_loss: 0.0123443603515625|unsuper_loss: 0.0 average reward score: -2.28125 ------------------------------------------------------------------------------------- |E2E latency=13.94s |Gather latency=0.97s (6.95%) |Generate time=8.95s (64.24%) |Training time=2.35s (16.87%) |Others=2.63 (18.89%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.30 epoch: 0|step: 452|ppo_ep: 1|act_loss: -0.07696533203125|cri_loss: 0.01824951171875|unsuper_loss: 0.0 average reward score: -1.154296875 ------------------------------------------------------------------------------------- |E2E latency=13.80s |Gather latency=1.21s (8.78%) |Generate time=8.61s (62.40%) |Training time=2.41s (17.49%) |Others=2.77 (20.11%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 453|ppo_ep: 1|act_loss: -0.038543701171875|cri_loss: 0.0338134765625|unsuper_loss: 0.0 average reward score: -0.7060546875 ------------------------------------------------------------------------------------- |E2E latency=14.10s |Gather latency=1.00s (7.12%) |Generate time=9.16s (64.95%) |Training time=2.36s (16.77%) |Others=2.58 (18.28%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.30 epoch: 0|step: 454|ppo_ep: 1|act_loss: -0.040435791015625|cri_loss: 0.00861358642578125|unsuper_loss: 0.0 average reward score: -1.404296875 ------------------------------------------------------------------------------------- |E2E latency=14.18s |Gather latency=0.78s (5.52%) |Generate time=9.16s (64.62%) |Training time=2.33s (16.42%) |Others=2.69 (18.95%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.30 epoch: 0|step: 455|ppo_ep: 1|act_loss: -0.0266265869140625|cri_loss: 0.020782470703125|unsuper_loss: 0.0 average reward score: -1.658203125 ------------------------------------------------------------------------------------- |E2E latency=13.87s |Gather latency=0.88s (6.35%) |Generate time=8.75s (63.08%) |Training time=2.33s (16.78%) |Others=2.79 (20.14%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.30 epoch: 0|step: 456|ppo_ep: 1|act_loss: -0.01090240478515625|cri_loss: 0.010223388671875|unsuper_loss: 0.0 average reward score: -1.60546875 ------------------------------------------------------------------------------------- |E2E latency=13.59s |Gather latency=1.00s (7.38%) |Generate time=8.66s (63.71%) |Training time=2.32s (17.08%) |Others=2.61 (19.22%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.30 epoch: 0|step: 457|ppo_ep: 1|act_loss: -0.061370849609375|cri_loss: 0.0201568603515625|unsuper_loss: 0.0 average reward score: -2.078125 ------------------------------------------------------------------------------------- |E2E latency=13.61s |Gather latency=0.68s (4.98%) |Generate time=8.61s (63.25%) |Training time=2.32s (17.08%) |Others=2.68 (19.67%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 458|ppo_ep: 1|act_loss: 0.033660888671875|cri_loss: 0.0180206298828125|unsuper_loss: 0.0 average reward score: -1.2705078125 ------------------------------------------------------------------------------------- |E2E latency=14.09s |Gather latency=0.67s (4.79%) |Generate time=9.19s (65.19%) |Training time=2.32s (16.50%) |Others=2.58 (18.31%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.30 [2023-05-17 01:47:55,595] [INFO] [logging.py:96:log_dist] [Rank 0] step=460, skipped=6, lr=[6.105205435806642e-07, 6.105205435806642e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 01:47:55,596] [INFO] [timer.py:199:stop] epoch=0/micro_step=460/global_step=460, RunningAvgSamplesPerSec=28.26242167814436, CurrSamplesPerSec=26.816219871950484, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 01:47:56,358] [INFO] [logging.py:96:log_dist] [Rank 0] step=460, skipped=9, lr=[3.189879169154723e-06, 3.189879169154723e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 459|ppo_ep: 1|act_loss: 0.056060791015625|cri_loss: 0.00977325439453125|unsuper_loss: 0.0 average reward score: -1.765625 ------------------------------------------------------------------------------------- |E2E latency=14.14s |Gather latency=1.07s (7.53%) |Generate time=9.02s (63.78%) |Training time=2.43s (17.18%) |Others=2.69 (19.04%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.30 epoch: 0|step: 460|ppo_ep: 1|act_loss: 0.0018663406372070312|cri_loss: 0.0137481689453125|unsuper_loss: 0.0 average reward score: -2.5859375 ------------------------------------------------------------------------------------- |E2E latency=13.82s |Gather latency=0.92s (6.65%) |Generate time=8.65s (62.60%) |Training time=2.29s (16.58%) |Others=2.88 (20.82%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 461|ppo_ep: 1|act_loss: -0.022613525390625|cri_loss: 0.00975799560546875|unsuper_loss: 0.0 average reward score: -1.4541015625 ------------------------------------------------------------------------------------- |E2E latency=14.02s |Gather latency=0.62s (4.43%) |Generate time=9.17s (65.38%) |Training time=2.33s (16.65%) |Others=2.52 (17.97%)|CurSamplesPerSec=2.28 |AvgSamplesPerSec=2.30 epoch: 0|step: 462|ppo_ep: 1|act_loss: -0.0099945068359375|cri_loss: 0.005214691162109375|unsuper_loss: 0.0 average reward score: -2.318359375 ------------------------------------------------------------------------------------- |E2E latency=14.23s |Gather latency=0.37s (2.63%) |Generate time=9.16s (64.37%) |Training time=2.42s (17.02%) |Others=2.65 (18.61%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.30 epoch: 0|step: 463|ppo_ep: 1|act_loss: 0.042236328125|cri_loss: 0.023101806640625|unsuper_loss: 0.0 average reward score: -2.142578125 ------------------------------------------------------------------------------------- |E2E latency=13.60s |Gather latency=0.50s (3.70%) |Generate time=8.63s (63.43%) |Training time=2.38s (17.53%) |Others=2.59 (19.04%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 464|ppo_ep: 1|act_loss: -0.0294647216796875|cri_loss: 0.018341064453125|unsuper_loss: 0.0 average reward score: -1.65625 ------------------------------------------------------------------------------------- |E2E latency=13.86s |Gather latency=0.78s (5.63%) |Generate time=8.79s (63.42%) |Training time=2.37s (17.08%) |Others=2.70 (19.49%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.30 epoch: 0|step: 465|ppo_ep: 1|act_loss: 0.04949951171875|cri_loss: 0.0191802978515625|unsuper_loss: 0.0 average reward score: -1.611328125 ------------------------------------------------------------------------------------- |E2E latency=14.36s |Gather latency=0.72s (5.04%) |Generate time=9.16s (63.79%) |Training time=2.37s (16.54%) |Others=2.82 (19.67%)|CurSamplesPerSec=2.23 |AvgSamplesPerSec=2.30 epoch: 0|step: 466|ppo_ep: 1|act_loss: 0.034088134765625|cri_loss: 0.00754547119140625|unsuper_loss: 0.0 average reward score: -1.623046875 ------------------------------------------------------------------------------------- |E2E latency=14.25s |Gather latency=0.74s (5.18%) |Generate time=9.18s (64.40%) |Training time=2.47s (17.31%) |Others=2.61 (18.29%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.30 epoch: 0|step: 467|ppo_ep: 1|act_loss: -0.031402587890625|cri_loss: 0.0140838623046875|unsuper_loss: 0.0 average reward score: -1.9091796875 ------------------------------------------------------------------------------------- |E2E latency=14.12s |Gather latency=0.69s (4.87%) |Generate time=8.98s (63.63%) |Training time=2.41s (17.08%) |Others=2.72 (19.29%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.30 epoch: 0|step: 468|ppo_ep: 1|act_loss: 0.01253509521484375|cri_loss: 0.0113067626953125|unsuper_loss: 0.0 average reward score: -1.982421875 ------------------------------------------------------------------------------------- |E2E latency=14.25s |Gather latency=0.66s (4.63%) |Generate time=9.24s (64.87%) |Training time=2.41s (16.91%) |Others=2.60 (18.22%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.30 [2023-05-17 01:50:15,934] [INFO] [logging.py:96:log_dist] [Rank 0] step=470, skipped=6, lr=[5.933243355785916e-07, 5.933243355785916e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 01:50:15,934] [INFO] [timer.py:199:stop] epoch=0/micro_step=470/global_step=470, RunningAvgSamplesPerSec=28.256114632317125, CurrSamplesPerSec=27.228231500282288, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 01:50:16,678] [INFO] [logging.py:96:log_dist] [Rank 0] step=470, skipped=9, lr=[3.101036303152072e-06, 3.101036303152072e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 469|ppo_ep: 1|act_loss: -0.012298583984375|cri_loss: 0.017333984375|unsuper_loss: 0.0 average reward score: -1.166015625 ------------------------------------------------------------------------------------- |E2E latency=13.80s |Gather latency=1.18s (8.51%) |Generate time=8.68s (62.86%) |Training time=2.38s (17.27%) |Others=2.74 (19.87%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 470|ppo_ep: 1|act_loss: -0.01181793212890625|cri_loss: 0.0162353515625|unsuper_loss: 0.0 average reward score: -1.7421875 ------------------------------------------------------------------------------------- |E2E latency=13.83s |Gather latency=1.18s (8.54%) |Generate time=8.58s (62.08%) |Training time=2.38s (17.21%) |Others=2.86 (20.71%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.30 epoch: 0|step: 471|ppo_ep: 1|act_loss: -0.01415252685546875|cri_loss: 0.01531982421875|unsuper_loss: 0.0 average reward score: -1.283203125 ------------------------------------------------------------------------------------- |E2E latency=13.53s |Gather latency=0.87s (6.41%) |Generate time=8.64s (63.89%) |Training time=2.34s (17.31%) |Others=2.54 (18.81%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.30 epoch: 0|step: 472|ppo_ep: 1|act_loss: 0.010650634765625|cri_loss: 0.0069580078125|unsuper_loss: 0.0 average reward score: -1.818359375 ------------------------------------------------------------------------------------- |E2E latency=13.62s |Gather latency=0.96s (7.07%) |Generate time=8.67s (63.62%) |Training time=2.32s (17.06%) |Others=2.63 (19.32%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 473|ppo_ep: 1|act_loss: -0.01229095458984375|cri_loss: 0.0081634521484375|unsuper_loss: 0.0 average reward score: -1.65625 ------------------------------------------------------------------------------------- |E2E latency=13.85s |Gather latency=0.76s (5.48%) |Generate time=8.85s (63.86%) |Training time=2.35s (16.96%) |Others=2.66 (19.18%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.30 epoch: 0|step: 474|ppo_ep: 1|act_loss: -0.044464111328125|cri_loss: 0.015472412109375|unsuper_loss: 0.0 average reward score: -1.8125 ------------------------------------------------------------------------------------- |E2E latency=13.59s |Gather latency=0.59s (4.36%) |Generate time=8.68s (63.87%) |Training time=2.28s (16.78%) |Others=2.63 (19.35%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.30 epoch: 0|step: 475|ppo_ep: 1|act_loss: -0.0006113052368164062|cri_loss: 0.015655517578125|unsuper_loss: 0.0 average reward score: -1.197265625 ------------------------------------------------------------------------------------- |E2E latency=13.69s |Gather latency=0.66s (4.85%) |Generate time=8.63s (63.00%) |Training time=2.39s (17.49%) |Others=2.67 (19.51%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 476|ppo_ep: 1|act_loss: 0.00295257568359375|cri_loss: 0.007549285888671875|unsuper_loss: 0.0 average reward score: -0.9189453125 ------------------------------------------------------------------------------------- |E2E latency=13.71s |Gather latency=0.62s (4.53%) |Generate time=8.69s (63.38%) |Training time=2.35s (17.14%) |Others=2.67 (19.48%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 477|ppo_ep: 1|act_loss: 0.035247802734375|cri_loss: 0.01525115966796875|unsuper_loss: 0.0 average reward score: -1.5107421875 ------------------------------------------------------------------------------------- |E2E latency=14.32s |Gather latency=0.68s (4.75%) |Generate time=9.17s (64.05%) |Training time=2.42s (16.87%) |Others=2.73 (19.08%)|CurSamplesPerSec=2.23 |AvgSamplesPerSec=2.30 epoch: 0|step: 478|ppo_ep: 1|act_loss: 0.034942626953125|cri_loss: 0.024505615234375|unsuper_loss: 0.0 average reward score: -1.1015625 ------------------------------------------------------------------------------------- |E2E latency=14.41s |Gather latency=0.82s (5.71%) |Generate time=9.21s (63.92%) |Training time=2.35s (16.32%) |Others=2.85 (19.75%)|CurSamplesPerSec=2.22 |AvgSamplesPerSec=2.30 [2023-05-17 01:52:34,077] [INFO] [loss_scaler.py:188:update_scale] [deepspeed] OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 65536, but hysteresis is 2. Reducing hysteresis to 1 [2023-05-17 01:52:34,077] [INFO] [logging.py:96:log_dist] [Rank 0] step=480, skipped=7, lr=[5.777188666442662e-07, 5.777188666442662e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 01:52:34,078] [INFO] [timer.py:199:stop] epoch=0/micro_step=480/global_step=480, RunningAvgSamplesPerSec=28.273640048240903, CurrSamplesPerSec=32.831860212478716, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 01:52:34,802] [INFO] [logging.py:96:log_dist] [Rank 0] step=480, skipped=9, lr=[3.011380165315503e-06, 3.011380165315503e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 479|ppo_ep: 1|act_loss: 0.000949859619140625|cri_loss: 0.007457733154296875|unsuper_loss: 0.0 average reward score: -2.4921875 ------------------------------------------------------------------------------------- |E2E latency=13.58s |Gather latency=0.83s (6.12%) |Generate time=8.66s (63.76%) |Training time=2.23s (16.46%) |Others=2.69 (19.78%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.30 [2023-05-17 01:52:48,047] [INFO] [loss_scaler.py:181:update_scale] [deepspeed] OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 65536, reducing to 32768 epoch: 0|step: 480|ppo_ep: 1|act_loss: 0.0300140380859375|cri_loss: 0.01412200927734375|unsuper_loss: 0.0 average reward score: -1.3408203125 ------------------------------------------------------------------------------------- |E2E latency=13.97s |Gather latency=0.79s (5.63%) |Generate time=9.01s (64.49%) |Training time=2.41s (17.26%) |Others=2.55 (18.25%)|CurSamplesPerSec=2.29 |AvgSamplesPerSec=2.30 epoch: 0|step: 481|ppo_ep: 1|act_loss: 0.006305694580078125|cri_loss: 0.0191497802734375|unsuper_loss: 0.0 average reward score: -0.9189453125 ------------------------------------------------------------------------------------- |E2E latency=13.77s |Gather latency=0.69s (4.99%) |Generate time=8.85s (64.28%) |Training time=2.40s (17.43%) |Others=2.52 (18.28%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 482|ppo_ep: 1|act_loss: -0.0006203651428222656|cri_loss: 0.0091400146484375|unsuper_loss: 0.0 average reward score: -2.095703125 ------------------------------------------------------------------------------------- |E2E latency=13.71s |Gather latency=0.87s (6.36%) |Generate time=8.70s (63.48%) |Training time=2.29s (16.73%) |Others=2.71 (19.79%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 483|ppo_ep: 1|act_loss: -0.048583984375|cri_loss: 0.0233306884765625|unsuper_loss: 0.0 average reward score: -1.4697265625 ------------------------------------------------------------------------------------- |E2E latency=14.22s |Gather latency=0.71s (5.01%) |Generate time=9.25s (65.02%) |Training time=2.27s (15.94%) |Others=2.71 (19.04%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.30 epoch: 0|step: 484|ppo_ep: 1|act_loss: 0.03167724609375|cri_loss: 0.01049041748046875|unsuper_loss: 0.0 average reward score: -1.3466796875 ------------------------------------------------------------------------------------- |E2E latency=13.52s |Gather latency=0.66s (4.88%) |Generate time=8.72s (64.48%) |Training time=2.30s (17.05%) |Others=2.50 (18.48%)|CurSamplesPerSec=2.37 |AvgSamplesPerSec=2.30 epoch: 0|step: 485|ppo_ep: 1|act_loss: 0.03289794921875|cri_loss: 0.0095367431640625|unsuper_loss: 0.0 average reward score: -1.484375 ------------------------------------------------------------------------------------- |E2E latency=13.75s |Gather latency=0.90s (6.55%) |Generate time=8.68s (63.17%) |Training time=2.34s (17.00%) |Others=2.73 (19.84%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 486|ppo_ep: 1|act_loss: -0.037322998046875|cri_loss: 0.042266845703125|unsuper_loss: 0.0 average reward score: -1.556640625 ------------------------------------------------------------------------------------- |E2E latency=14.13s |Gather latency=0.66s (4.68%) |Generate time=9.14s (64.66%) |Training time=2.34s (16.54%) |Others=2.66 (18.80%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.30 epoch: 0|step: 487|ppo_ep: 1|act_loss: -0.01364898681640625|cri_loss: 0.01007080078125|unsuper_loss: 0.0 average reward score: -1.8564453125 ------------------------------------------------------------------------------------- |E2E latency=13.74s |Gather latency=0.88s (6.37%) |Generate time=8.71s (63.38%) |Training time=2.33s (16.98%) |Others=2.70 (19.64%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 488|ppo_ep: 1|act_loss: -0.010955810546875|cri_loss: 0.01346588134765625|unsuper_loss: 0.0 average reward score: -2.2265625 ------------------------------------------------------------------------------------- |E2E latency=14.22s |Gather latency=0.66s (4.63%) |Generate time=9.15s (64.32%) |Training time=2.39s (16.84%) |Others=2.68 (18.83%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.30 [2023-05-17 01:54:53,076] [INFO] [logging.py:96:log_dist] [Rank 0] step=490, skipped=8, lr=[5.62009033315015e-07, 5.62009033315015e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 01:54:53,076] [INFO] [timer.py:199:stop] epoch=0/micro_step=490/global_step=490, RunningAvgSamplesPerSec=28.293091016746228, CurrSamplesPerSec=28.067513739136253, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 01:54:53,800] [INFO] [logging.py:96:log_dist] [Rank 0] step=490, skipped=9, lr=[2.9210320707989525e-06, 2.9210320707989525e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 489|ppo_ep: 1|act_loss: -0.016204833984375|cri_loss: 0.0171356201171875|unsuper_loss: 0.0 average reward score: -1.255859375 ------------------------------------------------------------------------------------- |E2E latency=13.98s |Gather latency=1.11s (7.92%) |Generate time=9.11s (65.17%) |Training time=2.37s (16.95%) |Others=2.50 (17.88%)|CurSamplesPerSec=2.29 |AvgSamplesPerSec=2.30 epoch: 0|step: 490|ppo_ep: 1|act_loss: 0.0148773193359375|cri_loss: 0.023895263671875|unsuper_loss: 0.0 average reward score: -1.7236328125 ------------------------------------------------------------------------------------- |E2E latency=13.85s |Gather latency=0.74s (5.33%) |Generate time=8.74s (63.14%) |Training time=2.36s (17.05%) |Others=2.74 (19.82%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.30 epoch: 0|step: 491|ppo_ep: 1|act_loss: 0.0096588134765625|cri_loss: 0.01361846923828125|unsuper_loss: 0.0 average reward score: -1.318359375 ------------------------------------------------------------------------------------- |E2E latency=14.29s |Gather latency=0.83s (5.78%) |Generate time=9.13s (63.94%) |Training time=2.39s (16.72%) |Others=2.76 (19.34%)|CurSamplesPerSec=2.24 |AvgSamplesPerSec=2.30 epoch: 0|step: 492|ppo_ep: 1|act_loss: 0.0189361572265625|cri_loss: 0.00762176513671875|unsuper_loss: 0.0 average reward score: -1.47265625 ------------------------------------------------------------------------------------- |E2E latency=13.73s |Gather latency=0.65s (4.74%) |Generate time=8.70s (63.40%) |Training time=2.32s (16.92%) |Others=2.70 (19.68%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 493|ppo_ep: 1|act_loss: 0.01690673828125|cri_loss: 0.0177764892578125|unsuper_loss: 0.0 average reward score: -1.9306640625 ------------------------------------------------------------------------------------- |E2E latency=13.78s |Gather latency=1.13s (8.22%) |Generate time=8.70s (63.15%) |Training time=2.30s (16.69%) |Others=2.78 (20.16%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 494|ppo_ep: 1|act_loss: 0.021484375|cri_loss: 0.016510009765625|unsuper_loss: 0.0 average reward score: -2.306640625 ------------------------------------------------------------------------------------- |E2E latency=14.21s |Gather latency=1.03s (7.27%) |Generate time=9.23s (64.98%) |Training time=2.45s (17.21%) |Others=2.53 (17.81%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.30 epoch: 0|step: 495|ppo_ep: 1|act_loss: -0.01239013671875|cri_loss: 0.010406494140625|unsuper_loss: 0.0 average reward score: -1.431640625 ------------------------------------------------------------------------------------- |E2E latency=14.06s |Gather latency=0.70s (4.96%) |Generate time=9.08s (64.58%) |Training time=2.32s (16.50%) |Others=2.66 (18.93%)|CurSamplesPerSec=2.28 |AvgSamplesPerSec=2.30 epoch: 0|step: 496|ppo_ep: 1|act_loss: -0.042755126953125|cri_loss: 0.019775390625|unsuper_loss: 0.0 average reward score: -1.58984375 ------------------------------------------------------------------------------------- |E2E latency=13.59s |Gather latency=0.70s (5.15%) |Generate time=8.62s (63.43%) |Training time=2.34s (17.25%) |Others=2.63 (19.32%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 497|ppo_ep: 1|act_loss: 0.01099395751953125|cri_loss: 0.011993408203125|unsuper_loss: 0.0 average reward score: -1.94140625 ------------------------------------------------------------------------------------- |E2E latency=13.58s |Gather latency=0.69s (5.08%) |Generate time=8.65s (63.72%) |Training time=2.31s (16.98%) |Others=2.62 (19.30%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.30 epoch: 0|step: 498|ppo_ep: 1|act_loss: 0.01546478271484375|cri_loss: 0.0088653564453125|unsuper_loss: 0.0 average reward score: -2.5390625 ------------------------------------------------------------------------------------- |E2E latency=13.77s |Gather latency=0.87s (6.31%) |Generate time=8.66s (62.90%) |Training time=2.39s (17.34%) |Others=2.72 (19.76%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 [2023-05-17 01:57:12,008] [INFO] [logging.py:96:log_dist] [Rank 0] step=500, skipped=8, lr=[5.44452206358211e-07, 5.44452206358211e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 01:57:12,008] [INFO] [timer.py:199:stop] epoch=0/micro_step=500/global_step=500, RunningAvgSamplesPerSec=28.300048200888558, CurrSamplesPerSec=27.792268019569633, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 01:57:12,752] [INFO] [logging.py:96:log_dist] [Rank 0] step=500, skipped=9, lr=[2.830114271054013e-06, 2.830114271054013e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 499|ppo_ep: 1|act_loss: 0.00042319297790527344|cri_loss: 0.00955963134765625|unsuper_loss: 0.0 average reward score: -1.8955078125 ------------------------------------------------------------------------------------- |E2E latency=14.09s |Gather latency=0.67s (4.75%) |Generate time=9.14s (64.86%) |Training time=2.39s (16.98%) |Others=2.56 (18.16%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.30 epoch: 0|step: 500|ppo_ep: 1|act_loss: 0.0175628662109375|cri_loss: 0.016937255859375|unsuper_loss: 0.0 average reward score: -1.5419921875 ------------------------------------------------------------------------------------- |E2E latency=14.26s |Gather latency=0.90s (6.34%) |Generate time=9.14s (64.13%) |Training time=2.41s (16.93%) |Others=2.70 (18.93%)|CurSamplesPerSec=2.24 |AvgSamplesPerSec=2.30 epoch: 0|step: 501|ppo_ep: 1|act_loss: -0.0035114288330078125|cri_loss: 0.0094757080078125|unsuper_loss: 0.0 average reward score: -2.591796875 ------------------------------------------------------------------------------------- |E2E latency=13.71s |Gather latency=0.67s (4.92%) |Generate time=8.61s (62.83%) |Training time=2.40s (17.49%) |Others=2.70 (19.68%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 502|ppo_ep: 1|act_loss: -0.0023059844970703125|cri_loss: 0.0240936279296875|unsuper_loss: 0.0 average reward score: -2.048828125 ------------------------------------------------------------------------------------- |E2E latency=13.59s |Gather latency=0.88s (6.50%) |Generate time=8.71s (64.06%) |Training time=2.34s (17.25%) |Others=2.54 (18.69%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 503|ppo_ep: 1|act_loss: -0.006816864013671875|cri_loss: 0.02197265625|unsuper_loss: 0.0 average reward score: -1.658203125 ------------------------------------------------------------------------------------- |E2E latency=14.11s |Gather latency=0.78s (5.54%) |Generate time=8.98s (63.60%) |Training time=2.45s (17.34%) |Others=2.69 (19.06%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.30 epoch: 0|step: 504|ppo_ep: 1|act_loss: -0.04266357421875|cri_loss: 0.0265960693359375|unsuper_loss: 0.0 average reward score: -1.7275390625 ------------------------------------------------------------------------------------- |E2E latency=13.70s |Gather latency=0.90s (6.56%) |Generate time=8.65s (63.16%) |Training time=2.33s (17.01%) |Others=2.72 (19.83%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 505|ppo_ep: 1|act_loss: -0.01084136962890625|cri_loss: 0.024505615234375|unsuper_loss: 0.0 average reward score: -1.6142578125 ------------------------------------------------------------------------------------- |E2E latency=14.04s |Gather latency=0.70s (4.99%) |Generate time=9.04s (64.35%) |Training time=2.46s (17.51%) |Others=2.55 (18.14%)|CurSamplesPerSec=2.28 |AvgSamplesPerSec=2.30 epoch: 0|step: 506|ppo_ep: 1|act_loss: -0.01148223876953125|cri_loss: 0.0187225341796875|unsuper_loss: 0.0 average reward score: -2.37890625 ------------------------------------------------------------------------------------- |E2E latency=14.33s |Gather latency=0.77s (5.40%) |Generate time=9.06s (63.21%) |Training time=2.46s (17.19%) |Others=2.81 (19.60%)|CurSamplesPerSec=2.23 |AvgSamplesPerSec=2.30 epoch: 0|step: 507|ppo_ep: 1|act_loss: 0.0257568359375|cri_loss: 0.01397705078125|unsuper_loss: 0.0 average reward score: -1.5703125 ------------------------------------------------------------------------------------- |E2E latency=13.85s |Gather latency=0.88s (6.36%) |Generate time=8.79s (63.47%) |Training time=2.42s (17.47%) |Others=2.64 (19.06%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.30 epoch: 0|step: 508|ppo_ep: 1|act_loss: -0.07977294921875|cri_loss: 0.04217529296875|unsuper_loss: 0.0 average reward score: -1.3330078125 ------------------------------------------------------------------------------------- |E2E latency=13.95s |Gather latency=0.86s (6.13%) |Generate time=8.73s (62.61%) |Training time=2.43s (17.42%) |Others=2.79 (19.97%)|CurSamplesPerSec=2.29 |AvgSamplesPerSec=2.30 [2023-05-17 01:59:31,133] [INFO] [logging.py:96:log_dist] [Rank 0] step=510, skipped=8, lr=[5.268115508802076e-07, 5.268115508802076e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 01:59:31,133] [INFO] [timer.py:199:stop] epoch=0/micro_step=510/global_step=510, RunningAvgSamplesPerSec=28.285941412562273, CurrSamplesPerSec=27.291536166167642, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 01:59:31,894] [INFO] [logging.py:96:log_dist] [Rank 0] step=510, skipped=9, lr=[2.7387497884095297e-06, 2.7387497884095297e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 509|ppo_ep: 1|act_loss: -0.052581787109375|cri_loss: 0.0231170654296875|unsuper_loss: 0.0 average reward score: -1.8642578125 ------------------------------------------------------------------------------------- |E2E latency=13.59s |Gather latency=0.57s (4.19%) |Generate time=8.62s (63.41%) |Training time=2.42s (17.78%) |Others=2.56 (18.81%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.30 epoch: 0|step: 510|ppo_ep: 1|act_loss: 0.0183258056640625|cri_loss: 0.0200653076171875|unsuper_loss: 0.0 average reward score: -0.9658203125 ------------------------------------------------------------------------------------- |E2E latency=14.09s |Gather latency=1.01s (7.14%) |Generate time=9.13s (64.75%) |Training time=2.38s (16.88%) |Others=2.59 (18.37%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.30 epoch: 0|step: 511|ppo_ep: 1|act_loss: -0.02728271484375|cri_loss: 0.0158538818359375|unsuper_loss: 0.0 average reward score: -2.3203125 ------------------------------------------------------------------------------------- |E2E latency=13.98s |Gather latency=0.80s (5.75%) |Generate time=8.72s (62.35%) |Training time=2.41s (17.21%) |Others=2.86 (20.43%)|CurSamplesPerSec=2.29 |AvgSamplesPerSec=2.30 epoch: 0|step: 512|ppo_ep: 1|act_loss: -0.021759033203125|cri_loss: 0.01148223876953125|unsuper_loss: 0.0 average reward score: -1.7236328125 ------------------------------------------------------------------------------------- |E2E latency=13.54s |Gather latency=0.71s (5.24%) |Generate time=8.61s (63.55%) |Training time=2.37s (17.50%) |Others=2.57 (18.95%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.30 epoch: 0|step: 513|ppo_ep: 1|act_loss: -0.0259552001953125|cri_loss: 0.01629638671875|unsuper_loss: 0.0 average reward score: -1.384765625 ------------------------------------------------------------------------------------- |E2E latency=13.45s |Gather latency=1.03s (7.68%) |Generate time=8.66s (64.38%) |Training time=2.27s (16.86%) |Others=2.52 (18.75%)|CurSamplesPerSec=2.38 |AvgSamplesPerSec=2.30 epoch: 0|step: 514|ppo_ep: 1|act_loss: -0.0163726806640625|cri_loss: 0.013763427734375|unsuper_loss: 0.0 average reward score: -1.4453125 ------------------------------------------------------------------------------------- |E2E latency=13.69s |Gather latency=0.84s (6.15%) |Generate time=8.64s (63.10%) |Training time=2.35s (17.19%) |Others=2.70 (19.71%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 515|ppo_ep: 1|act_loss: -0.003986358642578125|cri_loss: 0.01355743408203125|unsuper_loss: 0.0 average reward score: -1.853515625 ------------------------------------------------------------------------------------- |E2E latency=13.81s |Gather latency=0.93s (6.77%) |Generate time=9.03s (65.39%) |Training time=2.30s (16.67%) |Others=2.48 (17.94%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 516|ppo_ep: 1|act_loss: 0.039520263671875|cri_loss: 0.0428466796875|unsuper_loss: 0.0 average reward score: -1.349609375 ------------------------------------------------------------------------------------- |E2E latency=13.96s |Gather latency=0.90s (6.42%) |Generate time=8.71s (62.38%) |Training time=2.41s (17.25%) |Others=2.84 (20.38%)|CurSamplesPerSec=2.29 |AvgSamplesPerSec=2.30 epoch: 0|step: 517|ppo_ep: 1|act_loss: 0.0184783935546875|cri_loss: 0.01154327392578125|unsuper_loss: 0.0 average reward score: -2.029296875 ------------------------------------------------------------------------------------- |E2E latency=14.20s |Gather latency=0.69s (4.83%) |Generate time=9.17s (64.59%) |Training time=2.41s (16.98%) |Others=2.62 (18.43%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.30 epoch: 0|step: 518|ppo_ep: 1|act_loss: 0.058074951171875|cri_loss: 0.0262451171875|unsuper_loss: 0.0 average reward score: -2.083984375 ------------------------------------------------------------------------------------- |E2E latency=14.16s |Gather latency=0.85s (6.04%) |Generate time=9.08s (64.11%) |Training time=2.33s (16.48%) |Others=2.75 (19.41%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.30 [2023-05-17 02:01:49,717] [INFO] [logging.py:96:log_dist] [Rank 0] step=520, skipped=8, lr=[5.091109367340521e-07, 5.091109367340521e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 02:01:49,717] [INFO] [timer.py:199:stop] epoch=0/micro_step=520/global_step=520, RunningAvgSamplesPerSec=28.292922281157555, CurrSamplesPerSec=28.16052315425019, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 02:01:50,495] [INFO] [logging.py:96:log_dist] [Rank 0] step=520, skipped=9, lr=[2.647062249608123e-06, 2.647062249608123e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 519|ppo_ep: 1|act_loss: 0.05584716796875|cri_loss: 0.00732421875|unsuper_loss: 0.0 average reward score: -2.26953125 ------------------------------------------------------------------------------------- |E2E latency=13.71s |Gather latency=0.67s (4.88%) |Generate time=8.62s (62.84%) |Training time=2.37s (17.28%) |Others=2.73 (19.89%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 520|ppo_ep: 1|act_loss: 0.0254364013671875|cri_loss: 0.01291656494140625|unsuper_loss: 0.0 average reward score: -1.9169921875 ------------------------------------------------------------------------------------- |E2E latency=13.54s |Gather latency=0.65s (4.82%) |Generate time=8.71s (64.30%) |Training time=2.37s (17.51%) |Others=2.46 (18.20%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.30 epoch: 0|step: 521|ppo_ep: 1|act_loss: -0.03472900390625|cri_loss: 0.017669677734375|unsuper_loss: 0.0 average reward score: -1.99609375 ------------------------------------------------------------------------------------- |E2E latency=13.67s |Gather latency=0.79s (5.77%) |Generate time=8.73s (63.85%) |Training time=2.29s (16.77%) |Others=2.65 (19.38%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 522|ppo_ep: 1|act_loss: 0.0179901123046875|cri_loss: 0.01152801513671875|unsuper_loss: 0.0 average reward score: -1.9404296875 ------------------------------------------------------------------------------------- |E2E latency=13.59s |Gather latency=0.73s (5.37%) |Generate time=8.62s (63.43%) |Training time=2.30s (16.95%) |Others=2.66 (19.61%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.30 epoch: 0|step: 523|ppo_ep: 1|act_loss: -0.033660888671875|cri_loss: 0.012542724609375|unsuper_loss: 0.0 average reward score: -2.109375 ------------------------------------------------------------------------------------- |E2E latency=13.62s |Gather latency=0.89s (6.51%) |Generate time=8.70s (63.88%) |Training time=2.30s (16.89%) |Others=2.62 (19.23%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 524|ppo_ep: 1|act_loss: 0.0199737548828125|cri_loss: 0.0143585205078125|unsuper_loss: 0.0 average reward score: -1.3076171875 ------------------------------------------------------------------------------------- |E2E latency=13.89s |Gather latency=0.81s (5.86%) |Generate time=8.78s (63.23%) |Training time=2.43s (17.48%) |Others=2.68 (19.29%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.30 [2023-05-17 02:03:12,634] [INFO] [loss_scaler.py:188:update_scale] [deepspeed] OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 65536, but hysteresis is 2. Reducing hysteresis to 1 epoch: 0|step: 525|ppo_ep: 1|act_loss: -0.0012683868408203125|cri_loss: 0.01409149169921875|unsuper_loss: 0.0 average reward score: -2.09375 ------------------------------------------------------------------------------------- |E2E latency=13.83s |Gather latency=0.74s (5.34%) |Generate time=8.91s (64.38%) |Training time=2.36s (17.07%) |Others=2.57 (18.55%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.30 [2023-05-17 02:03:26,636] [INFO] [loss_scaler.py:181:update_scale] [deepspeed] OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 65536, reducing to 32768 epoch: 0|step: 526|ppo_ep: 1|act_loss: -0.040191650390625|cri_loss: 0.01543426513671875|unsuper_loss: 0.0 average reward score: -0.71630859375 ------------------------------------------------------------------------------------- |E2E latency=14.00s |Gather latency=0.96s (6.83%) |Generate time=8.85s (63.19%) |Training time=2.55s (18.18%) |Others=2.61 (18.63%)|CurSamplesPerSec=2.29 |AvgSamplesPerSec=2.30 epoch: 0|step: 527|ppo_ep: 1|act_loss: 0.0265350341796875|cri_loss: 0.01708984375|unsuper_loss: 0.0 average reward score: -0.896484375 ------------------------------------------------------------------------------------- |E2E latency=13.96s |Gather latency=0.71s (5.09%) |Generate time=8.62s (61.73%) |Training time=2.54s (18.22%) |Others=2.80 (20.04%)|CurSamplesPerSec=2.29 |AvgSamplesPerSec=2.30 epoch: 0|step: 528|ppo_ep: 1|act_loss: -0.0262908935546875|cri_loss: 0.01148223876953125|unsuper_loss: 0.0 average reward score: -1.6396484375 ------------------------------------------------------------------------------------- |E2E latency=14.16s |Gather latency=0.86s (6.07%) |Generate time=9.18s (64.86%) |Training time=2.42s (17.07%) |Others=2.56 (18.07%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.30 [2023-05-17 02:04:07,748] [INFO] [logging.py:96:log_dist] [Rank 0] step=530, skipped=8, lr=[4.91374314903824e-07, 4.91374314903824e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 02:04:07,749] [INFO] [timer.py:199:stop] epoch=0/micro_step=530/global_step=530, RunningAvgSamplesPerSec=28.295423579671326, CurrSamplesPerSec=27.585070925578854, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 02:04:08,453] [INFO] [logging.py:96:log_dist] [Rank 0] step=530, skipped=11, lr=[2.573562978654986e-06, 2.573562978654986e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 529|ppo_ep: 1|act_loss: 0.004917144775390625|cri_loss: 0.0079498291015625|unsuper_loss: 0.0 average reward score: -1.8427734375 ------------------------------------------------------------------------------------- |E2E latency=13.70s |Gather latency=1.03s (7.52%) |Generate time=8.65s (63.12%) |Training time=2.35s (17.19%) |Others=2.70 (19.69%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 530|ppo_ep: 1|act_loss: -0.040924072265625|cri_loss: 0.01418304443359375|unsuper_loss: 0.0 average reward score: -1.6484375 ------------------------------------------------------------------------------------- |E2E latency=13.89s |Gather latency=0.89s (6.39%) |Generate time=8.88s (63.89%) |Training time=2.38s (17.14%) |Others=2.64 (18.97%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.30 epoch: 0|step: 531|ppo_ep: 1|act_loss: -0.035308837890625|cri_loss: 0.028656005859375|unsuper_loss: 0.0 average reward score: -1.845703125 ------------------------------------------------------------------------------------- |E2E latency=13.62s |Gather latency=1.12s (8.23%) |Generate time=8.65s (63.48%) |Training time=2.31s (16.94%) |Others=2.67 (19.58%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 532|ppo_ep: 1|act_loss: -0.03753662109375|cri_loss: 0.009521484375|unsuper_loss: 0.0 average reward score: -1.6572265625 ------------------------------------------------------------------------------------- |E2E latency=13.70s |Gather latency=0.66s (4.78%) |Generate time=8.63s (63.01%) |Training time=2.32s (16.98%) |Others=2.74 (20.02%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 533|ppo_ep: 1|act_loss: 0.002704620361328125|cri_loss: 0.013763427734375|unsuper_loss: 0.0 average reward score: -1.7333984375 ------------------------------------------------------------------------------------- |E2E latency=13.73s |Gather latency=0.64s (4.68%) |Generate time=8.92s (64.97%) |Training time=2.34s (17.05%) |Others=2.47 (17.98%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 534|ppo_ep: 1|act_loss: 0.0435791015625|cri_loss: 0.0271453857421875|unsuper_loss: 0.0 average reward score: -1.9921875 ------------------------------------------------------------------------------------- |E2E latency=13.82s |Gather latency=0.57s (4.13%) |Generate time=8.74s (63.22%) |Training time=2.44s (17.67%) |Others=2.64 (19.11%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 535|ppo_ep: 1|act_loss: 0.0254058837890625|cri_loss: 0.00804901123046875|unsuper_loss: 0.0 average reward score: -1.982421875 ------------------------------------------------------------------------------------- |E2E latency=13.74s |Gather latency=0.70s (5.07%) |Generate time=8.62s (62.77%) |Training time=2.33s (16.98%) |Others=2.78 (20.24%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 536|ppo_ep: 1|act_loss: -0.025421142578125|cri_loss: 0.01428985595703125|unsuper_loss: 0.0 average reward score: -1.767578125 ------------------------------------------------------------------------------------- |E2E latency=13.56s |Gather latency=0.70s (5.16%) |Generate time=8.61s (63.52%) |Training time=2.32s (17.09%) |Others=2.63 (19.39%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.30 epoch: 0|step: 537|ppo_ep: 1|act_loss: -0.004375457763671875|cri_loss: 0.007678985595703125|unsuper_loss: 0.0 average reward score: -1.8837890625 ------------------------------------------------------------------------------------- |E2E latency=13.76s |Gather latency=0.63s (4.57%) |Generate time=8.73s (63.45%) |Training time=2.41s (17.55%) |Others=2.62 (19.00%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 538|ppo_ep: 1|act_loss: -0.03265380859375|cri_loss: 0.01166534423828125|unsuper_loss: 0.0 average reward score: -1.5048828125 ------------------------------------------------------------------------------------- |E2E latency=13.53s |Gather latency=0.69s (5.07%) |Generate time=8.62s (63.65%) |Training time=2.31s (17.10%) |Others=2.60 (19.24%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.30 [2023-05-17 02:06:24,719] [INFO] [logging.py:96:log_dist] [Rank 0] step=540, skipped=8, lr=[4.7362568509617615e-07, 4.7362568509617615e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 02:06:24,719] [INFO] [timer.py:199:stop] epoch=0/micro_step=540/global_step=540, RunningAvgSamplesPerSec=28.306237652268212, CurrSamplesPerSec=28.522924025349322, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 02:06:25,481] [INFO] [logging.py:96:log_dist] [Rank 0] step=540, skipped=11, lr=[2.4816067663297705e-06, 2.4816067663297705e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 539|ppo_ep: 1|act_loss: 0.026641845703125|cri_loss: 0.0223846435546875|unsuper_loss: 0.0 average reward score: -2.04296875 ------------------------------------------------------------------------------------- |E2E latency=13.68s |Gather latency=0.58s (4.27%) |Generate time=8.69s (63.57%) |Training time=2.38s (17.39%) |Others=2.60 (19.04%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 540|ppo_ep: 1|act_loss: -0.0183258056640625|cri_loss: 0.02581787109375|unsuper_loss: 0.0 average reward score: -1.3505859375 ------------------------------------------------------------------------------------- |E2E latency=14.17s |Gather latency=0.83s (5.84%) |Generate time=9.13s (64.42%) |Training time=2.40s (16.96%) |Others=2.64 (18.62%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.30 epoch: 0|step: 541|ppo_ep: 1|act_loss: -0.0511474609375|cri_loss: 0.0145111083984375|unsuper_loss: 0.0 average reward score: -1.6416015625 ------------------------------------------------------------------------------------- |E2E latency=13.77s |Gather latency=0.73s (5.27%) |Generate time=8.65s (62.82%) |Training time=2.37s (17.23%) |Others=2.75 (19.95%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 542|ppo_ep: 1|act_loss: -0.0224761962890625|cri_loss: 0.01244354248046875|unsuper_loss: 0.0 average reward score: -1.4931640625 ------------------------------------------------------------------------------------- |E2E latency=14.17s |Gather latency=0.75s (5.31%) |Generate time=9.18s (64.78%) |Training time=2.33s (16.47%) |Others=2.66 (18.75%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.30 epoch: 0|step: 543|ppo_ep: 1|act_loss: -0.005584716796875|cri_loss: 0.007564544677734375|unsuper_loss: 0.0 average reward score: -1.6474609375 ------------------------------------------------------------------------------------- |E2E latency=14.16s |Gather latency=1.00s (7.03%) |Generate time=9.12s (64.41%) |Training time=2.33s (16.43%) |Others=2.71 (19.15%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.30 epoch: 0|step: 544|ppo_ep: 1|act_loss: 0.0166168212890625|cri_loss: 0.01001739501953125|unsuper_loss: 0.0 average reward score: -1.818359375 ------------------------------------------------------------------------------------- |E2E latency=13.87s |Gather latency=0.44s (3.14%) |Generate time=8.86s (63.91%) |Training time=2.26s (16.30%) |Others=2.74 (19.80%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.30 epoch: 0|step: 545|ppo_ep: 1|act_loss: -0.01274871826171875|cri_loss: 0.00800323486328125|unsuper_loss: 0.0 average reward score: -2.150390625 ------------------------------------------------------------------------------------- |E2E latency=13.84s |Gather latency=0.66s (4.75%) |Generate time=8.72s (63.02%) |Training time=2.36s (17.08%) |Others=2.75 (19.89%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.30 epoch: 0|step: 546|ppo_ep: 1|act_loss: -0.0153350830078125|cri_loss: 0.0211181640625|unsuper_loss: 0.0 average reward score: -1.724609375 ------------------------------------------------------------------------------------- |E2E latency=13.60s |Gather latency=1.18s (8.65%) |Generate time=8.79s (64.61%) |Training time=2.25s (16.55%) |Others=2.56 (18.83%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 547|ppo_ep: 1|act_loss: 0.02545166015625|cri_loss: 0.00830841064453125|unsuper_loss: 0.0 average reward score: -1.88671875 ------------------------------------------------------------------------------------- |E2E latency=13.84s |Gather latency=0.63s (4.53%) |Generate time=8.83s (63.81%) |Training time=2.39s (17.28%) |Others=2.62 (18.91%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.30 epoch: 0|step: 548|ppo_ep: 1|act_loss: -0.007053375244140625|cri_loss: 0.0176239013671875|unsuper_loss: 0.0 average reward score: -2.310546875 ------------------------------------------------------------------------------------- |E2E latency=14.04s |Gather latency=0.71s (5.07%) |Generate time=8.92s (63.49%) |Training time=2.35s (16.73%) |Others=2.78 (19.78%)|CurSamplesPerSec=2.28 |AvgSamplesPerSec=2.30 [2023-05-17 02:08:43,727] [INFO] [logging.py:96:log_dist] [Rank 0] step=550, skipped=8, lr=[4.5588906326594805e-07, 4.5588906326594805e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 02:08:43,727] [INFO] [timer.py:199:stop] epoch=0/micro_step=550/global_step=550, RunningAvgSamplesPerSec=28.3118191492018, CurrSamplesPerSec=28.167940177686425, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 02:08:44,469] [INFO] [logging.py:96:log_dist] [Rank 0] step=550, skipped=11, lr=[2.38967544218319e-06, 2.38967544218319e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 549|ppo_ep: 1|act_loss: -0.02679443359375|cri_loss: 0.0069122314453125|unsuper_loss: 0.0 average reward score: -2.04296875 ------------------------------------------------------------------------------------- |E2E latency=13.52s |Gather latency=0.60s (4.45%) |Generate time=8.61s (63.71%) |Training time=2.36s (17.44%) |Others=2.55 (18.85%)|CurSamplesPerSec=2.37 |AvgSamplesPerSec=2.30 epoch: 0|step: 550|ppo_ep: 1|act_loss: -0.0207672119140625|cri_loss: 0.01071929931640625|unsuper_loss: 0.0 average reward score: -2.080078125 ------------------------------------------------------------------------------------- |E2E latency=13.78s |Gather latency=0.52s (3.77%) |Generate time=8.69s (63.08%) |Training time=2.43s (17.66%) |Others=2.65 (19.26%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 551|ppo_ep: 1|act_loss: -0.01910400390625|cri_loss: 0.032928466796875|unsuper_loss: 0.0 average reward score: -1.732421875 ------------------------------------------------------------------------------------- |E2E latency=14.12s |Gather latency=0.73s (5.19%) |Generate time=9.10s (64.43%) |Training time=2.41s (17.04%) |Others=2.62 (18.53%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.30 epoch: 0|step: 552|ppo_ep: 1|act_loss: -0.01035308837890625|cri_loss: 0.007183074951171875|unsuper_loss: 0.0 average reward score: -2.3125 ------------------------------------------------------------------------------------- |E2E latency=14.22s |Gather latency=0.69s (4.83%) |Generate time=9.11s (64.08%) |Training time=2.41s (16.97%) |Others=2.69 (18.95%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.30 epoch: 0|step: 553|ppo_ep: 1|act_loss: -0.03460693359375|cri_loss: 0.0236968994140625|unsuper_loss: 0.0 average reward score: -1.361328125 ------------------------------------------------------------------------------------- |E2E latency=13.96s |Gather latency=0.76s (5.47%) |Generate time=8.61s (61.69%) |Training time=2.42s (17.35%) |Others=2.93 (20.96%)|CurSamplesPerSec=2.29 |AvgSamplesPerSec=2.30 epoch: 0|step: 554|ppo_ep: 1|act_loss: -0.0124053955078125|cri_loss: 0.01348876953125|unsuper_loss: 0.0 average reward score: -1.943359375 ------------------------------------------------------------------------------------- |E2E latency=14.12s |Gather latency=0.60s (4.22%) |Generate time=9.13s (64.65%) |Training time=2.39s (16.94%) |Others=2.60 (18.41%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.30 epoch: 0|step: 555|ppo_ep: 1|act_loss: 0.0009937286376953125|cri_loss: 0.01456451416015625|unsuper_loss: 0.0 average reward score: -2.197265625 ------------------------------------------------------------------------------------- |E2E latency=13.81s |Gather latency=0.76s (5.48%) |Generate time=8.68s (62.81%) |Training time=2.45s (17.75%) |Others=2.69 (19.44%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 556|ppo_ep: 1|act_loss: 0.011627197265625|cri_loss: 0.00685882568359375|unsuper_loss: 0.0 average reward score: -1.759765625 ------------------------------------------------------------------------------------- |E2E latency=13.70s |Gather latency=0.76s (5.55%) |Generate time=8.67s (63.27%) |Training time=2.35s (17.19%) |Others=2.68 (19.55%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 557|ppo_ep: 1|act_loss: -0.0171661376953125|cri_loss: 0.0152435302734375|unsuper_loss: 0.0 average reward score: -2.115234375 ------------------------------------------------------------------------------------- |E2E latency=13.74s |Gather latency=0.86s (6.26%) |Generate time=8.74s (63.59%) |Training time=2.37s (17.23%) |Others=2.63 (19.18%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 558|ppo_ep: 1|act_loss: -0.010284423828125|cri_loss: 0.0125732421875|unsuper_loss: 0.0 average reward score: -1.853515625 ------------------------------------------------------------------------------------- |E2E latency=13.77s |Gather latency=0.88s (6.39%) |Generate time=8.65s (62.80%) |Training time=2.33s (16.92%) |Others=2.79 (20.28%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 [2023-05-17 02:11:02,694] [INFO] [logging.py:96:log_dist] [Rank 0] step=560, skipped=8, lr=[4.3818844911979245e-07, 4.3818844911979245e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 02:11:02,695] [INFO] [timer.py:199:stop] epoch=0/micro_step=560/global_step=560, RunningAvgSamplesPerSec=28.31006491181583, CurrSamplesPerSec=27.492883520131883, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 02:11:03,470] [INFO] [logging.py:96:log_dist] [Rank 0] step=560, skipped=11, lr=[2.2978933999601656e-06, 2.2978933999601656e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 559|ppo_ep: 1|act_loss: -0.01270294189453125|cri_loss: 0.007061004638671875|unsuper_loss: 0.0 average reward score: -1.4814453125 ------------------------------------------------------------------------------------- |E2E latency=13.78s |Gather latency=0.37s (2.68%) |Generate time=8.77s (63.62%) |Training time=2.41s (17.51%) |Others=2.60 (18.88%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 560|ppo_ep: 1|act_loss: 0.02313232421875|cri_loss: 0.01142120361328125|unsuper_loss: 0.0 average reward score: -1.8212890625 ------------------------------------------------------------------------------------- |E2E latency=14.15s |Gather latency=0.56s (3.98%) |Generate time=9.13s (64.52%) |Training time=2.39s (16.89%) |Others=2.63 (18.59%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.30 epoch: 0|step: 561|ppo_ep: 1|act_loss: -0.0105438232421875|cri_loss: 0.0114593505859375|unsuper_loss: 0.0 average reward score: -1.9072265625 ------------------------------------------------------------------------------------- |E2E latency=13.77s |Gather latency=0.88s (6.39%) |Generate time=8.72s (63.34%) |Training time=2.37s (17.23%) |Others=2.68 (19.43%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 562|ppo_ep: 1|act_loss: 0.0007605552673339844|cri_loss: 0.0123748779296875|unsuper_loss: 0.0 average reward score: -2.087890625 ------------------------------------------------------------------------------------- |E2E latency=14.19s |Gather latency=0.76s (5.32%) |Generate time=9.15s (64.49%) |Training time=2.41s (16.99%) |Others=2.63 (18.52%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.30 epoch: 0|step: 563|ppo_ep: 1|act_loss: 0.016845703125|cri_loss: 0.01082611083984375|unsuper_loss: 0.0 average reward score: -0.03350830078125 ------------------------------------------------------------------------------------- |E2E latency=14.10s |Gather latency=0.63s (4.47%) |Generate time=8.92s (63.30%) |Training time=2.44s (17.29%) |Others=2.74 (19.41%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.30 epoch: 0|step: 564|ppo_ep: 1|act_loss: -0.0254669189453125|cri_loss: 0.01454925537109375|unsuper_loss: 0.0 average reward score: -1.26171875 ------------------------------------------------------------------------------------- |E2E latency=13.89s |Gather latency=0.86s (6.16%) |Generate time=8.80s (63.38%) |Training time=2.36s (17.00%) |Others=2.73 (19.62%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.30 epoch: 0|step: 565|ppo_ep: 1|act_loss: -0.00955963134765625|cri_loss: 0.0119781494140625|unsuper_loss: 0.0 average reward score: -1.42578125 ------------------------------------------------------------------------------------- |E2E latency=13.83s |Gather latency=0.88s (6.35%) |Generate time=8.81s (63.72%) |Training time=2.37s (17.16%) |Others=2.64 (19.12%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.30 epoch: 0|step: 566|ppo_ep: 1|act_loss: -0.03912353515625|cri_loss: 0.01214599609375|unsuper_loss: 0.0 average reward score: -1.4853515625 ------------------------------------------------------------------------------------- |E2E latency=12.78s |Gather latency=0.93s (7.29%) |Generate time=7.69s (60.19%) |Training time=2.36s (18.49%) |Others=2.73 (21.32%)|CurSamplesPerSec=2.50 |AvgSamplesPerSec=2.30 epoch: 0|step: 567|ppo_ep: 1|act_loss: -0.03741455078125|cri_loss: 0.006565093994140625|unsuper_loss: 0.0 average reward score: -1.6982421875 ------------------------------------------------------------------------------------- |E2E latency=13.79s |Gather latency=0.66s (4.76%) |Generate time=8.69s (63.01%) |Training time=2.40s (17.44%) |Others=2.70 (19.55%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 568|ppo_ep: 1|act_loss: 0.004852294921875|cri_loss: 0.01080322265625|unsuper_loss: 0.0 average reward score: -1.35546875 ------------------------------------------------------------------------------------- |E2E latency=13.97s |Gather latency=0.71s (5.07%) |Generate time=8.78s (62.86%) |Training time=2.39s (17.12%) |Others=2.80 (20.01%)|CurSamplesPerSec=2.29 |AvgSamplesPerSec=2.30 [2023-05-17 02:13:20,913] [INFO] [logging.py:96:log_dist] [Rank 0] step=570, skipped=8, lr=[4.2054779364178917e-07, 4.2054779364178917e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 02:13:20,913] [INFO] [timer.py:199:stop] epoch=0/micro_step=570/global_step=570, RunningAvgSamplesPerSec=28.303955652999544, CurrSamplesPerSec=28.20730090104328, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 02:13:21,658] [INFO] [logging.py:96:log_dist] [Rank 0] step=570, skipped=11, lr=[2.206384831409863e-06, 2.206384831409863e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 569|ppo_ep: 1|act_loss: -0.040496826171875|cri_loss: 0.012603759765625|unsuper_loss: 0.0 average reward score: -1.69921875 ------------------------------------------------------------------------------------- |E2E latency=13.73s |Gather latency=0.69s (5.04%) |Generate time=8.75s (63.71%) |Training time=2.36s (17.21%) |Others=2.62 (19.08%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 570|ppo_ep: 1|act_loss: -0.00286102294921875|cri_loss: 0.0241851806640625|unsuper_loss: 0.0 average reward score: -2.455078125 ------------------------------------------------------------------------------------- |E2E latency=13.69s |Gather latency=0.63s (4.58%) |Generate time=8.67s (63.35%) |Training time=2.38s (17.39%) |Others=2.64 (19.25%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 571|ppo_ep: 1|act_loss: -0.0207366943359375|cri_loss: 0.0177459716796875|unsuper_loss: 0.0 average reward score: -1.939453125 ------------------------------------------------------------------------------------- |E2E latency=14.05s |Gather latency=0.72s (5.15%) |Generate time=9.04s (64.30%) |Training time=2.43s (17.33%) |Others=2.58 (18.37%)|CurSamplesPerSec=2.28 |AvgSamplesPerSec=2.30 epoch: 0|step: 572|ppo_ep: 1|act_loss: 0.01214599609375|cri_loss: 0.015655517578125|unsuper_loss: 0.0 average reward score: -1.2138671875 ------------------------------------------------------------------------------------- |E2E latency=13.80s |Gather latency=0.70s (5.10%) |Generate time=8.82s (63.94%) |Training time=2.36s (17.10%) |Others=2.62 (18.96%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 573|ppo_ep: 1|act_loss: 0.0160064697265625|cri_loss: 0.009979248046875|unsuper_loss: 0.0 average reward score: -1.1875 ------------------------------------------------------------------------------------- |E2E latency=13.64s |Gather latency=1.10s (8.09%) |Generate time=8.68s (63.62%) |Training time=2.30s (16.88%) |Others=2.66 (19.50%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 574|ppo_ep: 1|act_loss: -0.0113067626953125|cri_loss: 0.01280975341796875|unsuper_loss: 0.0 average reward score: -2.16796875 ------------------------------------------------------------------------------------- |E2E latency=14.30s |Gather latency=0.87s (6.06%) |Generate time=9.14s (63.92%) |Training time=2.35s (16.43%) |Others=2.81 (19.64%)|CurSamplesPerSec=2.24 |AvgSamplesPerSec=2.30 epoch: 0|step: 575|ppo_ep: 1|act_loss: -0.0352783203125|cri_loss: 0.00933837890625|unsuper_loss: 0.0 average reward score: -1.75 ------------------------------------------------------------------------------------- |E2E latency=13.65s |Gather latency=1.13s (8.31%) |Generate time=8.69s (63.64%) |Training time=2.33s (17.08%) |Others=2.63 (19.28%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 576|ppo_ep: 1|act_loss: -0.034759521484375|cri_loss: 0.0140228271484375|unsuper_loss: 0.0 average reward score: -1.638671875 ------------------------------------------------------------------------------------- |E2E latency=13.83s |Gather latency=0.44s (3.18%) |Generate time=8.73s (63.07%) |Training time=2.43s (17.56%) |Others=2.68 (19.37%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.30 epoch: 0|step: 577|ppo_ep: 1|act_loss: 0.005645751953125|cri_loss: 0.016754150390625|unsuper_loss: 0.0 average reward score: -1.787109375 ------------------------------------------------------------------------------------- |E2E latency=15.80s |Gather latency=0.59s (3.74%) |Generate time=9.18s (58.11%) |Training time=4.11s (26.03%) |Others=2.51 (15.86%)|CurSamplesPerSec=2.03 |AvgSamplesPerSec=2.30 epoch: 0|step: 578|ppo_ep: 1|act_loss: -0.05126953125|cri_loss: 0.0115814208984375|unsuper_loss: 0.0 average reward score: -2.384765625 ------------------------------------------------------------------------------------- |E2E latency=13.85s |Gather latency=0.55s (4.01%) |Generate time=8.86s (64.00%) |Training time=2.37s (17.10%) |Others=2.62 (18.90%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.30 [2023-05-17 02:15:41,036] [INFO] [logging.py:96:log_dist] [Rank 0] step=580, skipped=8, lr=[4.029909666849853e-07, 4.029909666849853e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 02:15:41,037] [INFO] [timer.py:199:stop] epoch=0/micro_step=580/global_step=580, RunningAvgSamplesPerSec=28.27714449868952, CurrSamplesPerSec=29.309705231790147, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 02:15:41,808] [INFO] [logging.py:96:log_dist] [Rank 0] step=580, skipped=11, lr=[2.1152735582398453e-06, 2.1152735582398453e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 579|ppo_ep: 1|act_loss: 0.002349853515625|cri_loss: 0.01096343994140625|unsuper_loss: 0.0 average reward score: -1.87890625 ------------------------------------------------------------------------------------- |E2E latency=13.53s |Gather latency=0.60s (4.45%) |Generate time=8.66s (63.98%) |Training time=2.30s (16.97%) |Others=2.58 (19.05%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.30 epoch: 0|step: 580|ppo_ep: 1|act_loss: -0.01593017578125|cri_loss: 0.0191650390625|unsuper_loss: 0.0 average reward score: -1.65234375 ------------------------------------------------------------------------------------- |E2E latency=13.68s |Gather latency=0.76s (5.53%) |Generate time=8.69s (63.57%) |Training time=2.36s (17.25%) |Others=2.62 (19.18%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 [2023-05-17 02:16:08,857] [INFO] [loss_scaler.py:188:update_scale] [deepspeed] OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 65536, but hysteresis is 2. Reducing hysteresis to 1 epoch: 0|step: 581|ppo_ep: 1|act_loss: -0.0047760009765625|cri_loss: 0.01457977294921875|unsuper_loss: 0.0 average reward score: -2.4609375 ------------------------------------------------------------------------------------- |E2E latency=14.13s |Gather latency=0.49s (3.49%) |Generate time=9.17s (64.89%) |Training time=2.29s (16.18%) |Others=2.67 (18.93%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.30 [2023-05-17 02:16:22,736] [INFO] [loss_scaler.py:181:update_scale] [deepspeed] OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 65536, reducing to 32768 epoch: 0|step: 582|ppo_ep: 1|act_loss: -0.003002166748046875|cri_loss: 0.01251983642578125|unsuper_loss: 0.0 average reward score: -1.55078125 ------------------------------------------------------------------------------------- |E2E latency=13.84s |Gather latency=0.71s (5.13%) |Generate time=8.82s (63.77%) |Training time=2.41s (17.45%) |Others=2.60 (18.78%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.30 epoch: 0|step: 583|ppo_ep: 1|act_loss: -0.01139068603515625|cri_loss: 0.029083251953125|unsuper_loss: 0.0 average reward score: -2.046875 ------------------------------------------------------------------------------------- |E2E latency=14.17s |Gather latency=0.95s (6.72%) |Generate time=9.15s (64.58%) |Training time=2.46s (17.36%) |Others=2.56 (18.06%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.30 epoch: 0|step: 584|ppo_ep: 1|act_loss: -0.0083160400390625|cri_loss: 0.010040283203125|unsuper_loss: 0.0 average reward score: -2.42578125 ------------------------------------------------------------------------------------- |E2E latency=13.70s |Gather latency=0.60s (4.37%) |Generate time=8.62s (62.90%) |Training time=2.40s (17.48%) |Others=2.69 (19.61%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 585|ppo_ep: 1|act_loss: 0.0044403076171875|cri_loss: 0.00605010986328125|unsuper_loss: 0.0 average reward score: -1.5439453125 ------------------------------------------------------------------------------------- |E2E latency=13.69s |Gather latency=0.76s (5.54%) |Generate time=8.72s (63.65%) |Training time=2.34s (17.08%) |Others=2.64 (19.27%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 586|ppo_ep: 1|act_loss: -0.0228118896484375|cri_loss: 0.006500244140625|unsuper_loss: 0.0 average reward score: -2.1640625 ------------------------------------------------------------------------------------- |E2E latency=13.68s |Gather latency=0.93s (6.83%) |Generate time=8.70s (63.59%) |Training time=2.30s (16.82%) |Others=2.68 (19.59%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 587|ppo_ep: 1|act_loss: 0.00244903564453125|cri_loss: 0.0193939208984375|unsuper_loss: 0.0 average reward score: -1.24609375 ------------------------------------------------------------------------------------- |E2E latency=13.53s |Gather latency=0.86s (6.35%) |Generate time=8.61s (63.62%) |Training time=2.32s (17.12%) |Others=2.61 (19.27%)|CurSamplesPerSec=2.37 |AvgSamplesPerSec=2.30 epoch: 0|step: 588|ppo_ep: 1|act_loss: -0.041168212890625|cri_loss: 0.01300811767578125|unsuper_loss: 0.0 average reward score: -1.59765625 ------------------------------------------------------------------------------------- |E2E latency=13.97s |Gather latency=0.81s (5.81%) |Generate time=8.98s (64.30%) |Training time=2.38s (17.05%) |Others=2.60 (18.65%)|CurSamplesPerSec=2.29 |AvgSamplesPerSec=2.30 [2023-05-17 02:17:59,779] [INFO] [logging.py:96:log_dist] [Rank 0] step=590, skipped=10, lr=[3.8902183060432073e-07, 3.8902183060432073e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 02:17:59,779] [INFO] [timer.py:199:stop] epoch=0/micro_step=590/global_step=590, RunningAvgSamplesPerSec=28.292739277115313, CurrSamplesPerSec=29.195258724574984, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 02:18:00,544] [INFO] [logging.py:96:log_dist] [Rank 0] step=590, skipped=11, lr=[2.024682864570945e-06, 2.024682864570945e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 589|ppo_ep: 1|act_loss: -0.0015249252319335938|cri_loss: 0.01241302490234375|unsuper_loss: 0.0 average reward score: -1.591796875 ------------------------------------------------------------------------------------- |E2E latency=14.35s |Gather latency=0.98s (6.80%) |Generate time=9.20s (64.10%) |Training time=2.34s (16.32%) |Others=2.81 (19.58%)|CurSamplesPerSec=2.23 |AvgSamplesPerSec=2.30 epoch: 0|step: 590|ppo_ep: 1|act_loss: 0.03179931640625|cri_loss: 0.00539398193359375|unsuper_loss: 0.0 average reward score: -1.998046875 ------------------------------------------------------------------------------------- |E2E latency=13.72s |Gather latency=0.74s (5.40%) |Generate time=8.81s (64.19%) |Training time=2.32s (16.94%) |Others=2.59 (18.87%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 591|ppo_ep: 1|act_loss: -0.0014324188232421875|cri_loss: 0.0158538818359375|unsuper_loss: 0.0 average reward score: -1.6630859375 ------------------------------------------------------------------------------------- |E2E latency=14.14s |Gather latency=0.68s (4.84%) |Generate time=9.13s (64.56%) |Training time=2.36s (16.69%) |Others=2.65 (18.76%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.30 epoch: 0|step: 592|ppo_ep: 1|act_loss: 0.00978851318359375|cri_loss: 0.01529693603515625|unsuper_loss: 0.0 average reward score: -1.900390625 ------------------------------------------------------------------------------------- |E2E latency=13.55s |Gather latency=0.63s (4.66%) |Generate time=8.70s (64.19%) |Training time=2.33s (17.16%) |Others=2.53 (18.65%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.30 epoch: 0|step: 593|ppo_ep: 1|act_loss: -0.0115203857421875|cri_loss: 0.01503753662109375|unsuper_loss: 0.0 average reward score: -1.828125 ------------------------------------------------------------------------------------- |E2E latency=13.61s |Gather latency=0.66s (4.83%) |Generate time=8.64s (63.49%) |Training time=2.30s (16.92%) |Others=2.66 (19.58%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 594|ppo_ep: 1|act_loss: -0.051849365234375|cri_loss: 0.01324462890625|unsuper_loss: 0.0 average reward score: -0.417236328125 ------------------------------------------------------------------------------------- |E2E latency=12.81s |Gather latency=0.85s (6.63%) |Generate time=7.67s (59.85%) |Training time=2.43s (18.99%) |Others=2.71 (21.16%)|CurSamplesPerSec=2.50 |AvgSamplesPerSec=2.30 epoch: 0|step: 595|ppo_ep: 1|act_loss: -0.0262451171875|cri_loss: 0.013275146484375|unsuper_loss: 0.0 average reward score: -2.529296875 ------------------------------------------------------------------------------------- |E2E latency=14.40s |Gather latency=0.76s (5.27%) |Generate time=9.19s (63.84%) |Training time=2.40s (16.70%) |Others=2.80 (19.46%)|CurSamplesPerSec=2.22 |AvgSamplesPerSec=2.30 epoch: 0|step: 596|ppo_ep: 1|act_loss: -0.093017578125|cri_loss: 0.0193634033203125|unsuper_loss: 0.0 average reward score: -2.083984375 ------------------------------------------------------------------------------------- |E2E latency=13.81s |Gather latency=0.66s (4.80%) |Generate time=8.69s (62.96%) |Training time=2.35s (17.05%) |Others=2.76 (19.99%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 597|ppo_ep: 1|act_loss: 0.0312042236328125|cri_loss: 0.010711669921875|unsuper_loss: 0.0 average reward score: -2.4375 ------------------------------------------------------------------------------------- |E2E latency=13.91s |Gather latency=0.71s (5.13%) |Generate time=8.66s (62.21%) |Training time=2.38s (17.07%) |Others=2.88 (20.71%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.30 epoch: 0|step: 598|ppo_ep: 1|act_loss: 0.0184326171875|cri_loss: 0.00579833984375|unsuper_loss: 0.0 average reward score: -2.09375 ------------------------------------------------------------------------------------- |E2E latency=13.69s |Gather latency=0.82s (5.96%) |Generate time=8.78s (64.14%) |Training time=2.39s (17.43%) |Others=2.52 (18.42%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 [2023-05-17 02:20:17,602] [INFO] [logging.py:96:log_dist] [Rank 0] step=600, skipped=10, lr=[3.7167566442140874e-07, 3.7167566442140874e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 02:20:17,603] [INFO] [timer.py:199:stop] epoch=0/micro_step=600/global_step=600, RunningAvgSamplesPerSec=28.293895340325843, CurrSamplesPerSec=28.061522301677552, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 02:20:18,351] [INFO] [logging.py:96:log_dist] [Rank 0] step=600, skipped=11, lr=[1.9347353301195425e-06, 1.9347353301195425e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 599|ppo_ep: 1|act_loss: -0.01448822021484375|cri_loss: 0.00982666015625|unsuper_loss: 0.0 average reward score: -1.689453125 ------------------------------------------------------------------------------------- |E2E latency=14.16s |Gather latency=0.76s (5.34%) |Generate time=9.15s (64.58%) |Training time=2.43s (17.15%) |Others=2.59 (18.27%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.30 epoch: 0|step: 600|ppo_ep: 1|act_loss: 0.0132293701171875|cri_loss: 0.007152557373046875|unsuper_loss: 0.0 average reward score: -1.583984375 ------------------------------------------------------------------------------------- |E2E latency=13.56s |Gather latency=1.00s (7.41%) |Generate time=8.69s (64.07%) |Training time=2.28s (16.82%) |Others=2.59 (19.12%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.30 epoch: 0|step: 601|ppo_ep: 1|act_loss: -0.0223388671875|cri_loss: 0.0223541259765625|unsuper_loss: 0.0 average reward score: -1.9677734375 ------------------------------------------------------------------------------------- |E2E latency=13.44s |Gather latency=0.83s (6.19%) |Generate time=8.46s (62.97%) |Training time=2.34s (17.38%) |Others=2.64 (19.65%)|CurSamplesPerSec=2.38 |AvgSamplesPerSec=2.30 epoch: 0|step: 602|ppo_ep: 1|act_loss: 0.034271240234375|cri_loss: 0.014373779296875|unsuper_loss: 0.0 average reward score: -1.9501953125 ------------------------------------------------------------------------------------- |E2E latency=13.74s |Gather latency=1.10s (8.03%) |Generate time=8.74s (63.63%) |Training time=2.39s (17.39%) |Others=2.61 (18.97%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 603|ppo_ep: 1|act_loss: 0.033599853515625|cri_loss: 0.0145111083984375|unsuper_loss: 0.0 average reward score: -1.6015625 ------------------------------------------------------------------------------------- |E2E latency=13.57s |Gather latency=0.55s (4.04%) |Generate time=8.65s (63.74%) |Training time=2.37s (17.48%) |Others=2.55 (18.79%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.30 epoch: 0|step: 604|ppo_ep: 1|act_loss: 0.0085296630859375|cri_loss: 0.00676727294921875|unsuper_loss: 0.0 average reward score: -2.021484375 ------------------------------------------------------------------------------------- |E2E latency=13.95s |Gather latency=0.62s (4.41%) |Generate time=8.80s (63.07%) |Training time=2.45s (17.59%) |Others=2.70 (19.34%)|CurSamplesPerSec=2.29 |AvgSamplesPerSec=2.30 epoch: 0|step: 605|ppo_ep: 1|act_loss: -0.01873779296875|cri_loss: 0.015899658203125|unsuper_loss: 0.0 average reward score: -2.080078125 ------------------------------------------------------------------------------------- |E2E latency=14.36s |Gather latency=0.74s (5.12%) |Generate time=9.13s (63.57%) |Training time=2.43s (16.90%) |Others=2.80 (19.53%)|CurSamplesPerSec=2.23 |AvgSamplesPerSec=2.30 epoch: 0|step: 606|ppo_ep: 1|act_loss: -0.0131072998046875|cri_loss: 0.0124359130859375|unsuper_loss: 0.0 average reward score: -1.642578125 ------------------------------------------------------------------------------------- |E2E latency=14.21s |Gather latency=0.69s (4.83%) |Generate time=9.13s (64.28%) |Training time=2.44s (17.15%) |Others=2.64 (18.57%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.30 epoch: 0|step: 607|ppo_ep: 1|act_loss: -0.0138397216796875|cri_loss: 0.02825927734375|unsuper_loss: 0.0 average reward score: -1.3544921875 ------------------------------------------------------------------------------------- |E2E latency=13.80s |Gather latency=0.82s (5.91%) |Generate time=8.66s (62.74%) |Training time=2.31s (16.76%) |Others=2.83 (20.49%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 608|ppo_ep: 1|act_loss: 0.022918701171875|cri_loss: 0.00794219970703125|unsuper_loss: 0.0 average reward score: -2.26171875 ------------------------------------------------------------------------------------- |E2E latency=13.73s |Gather latency=0.44s (3.23%) |Generate time=8.62s (62.79%) |Training time=2.37s (17.29%) |Others=2.74 (19.93%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 [2023-05-17 02:22:35,540] [INFO] [logging.py:96:log_dist] [Rank 0] step=610, skipped=10, lr=[3.5447945641933586e-07, 3.5447945641933586e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 02:22:35,541] [INFO] [timer.py:199:stop] epoch=0/micro_step=610/global_step=610, RunningAvgSamplesPerSec=28.293036325776146, CurrSamplesPerSec=29.72971198549417, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 02:22:36,291] [INFO] [logging.py:96:log_dist] [Rank 0] step=610, skipped=11, lr=[1.8455526643329995e-06, 1.8455526643329995e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 609|ppo_ep: 1|act_loss: 0.0136566162109375|cri_loss: 0.007198333740234375|unsuper_loss: 0.0 average reward score: -1.8662109375 ------------------------------------------------------------------------------------- |E2E latency=13.58s |Gather latency=0.58s (4.29%) |Generate time=8.68s (63.94%) |Training time=2.29s (16.86%) |Others=2.61 (19.20%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.30 epoch: 0|step: 610|ppo_ep: 1|act_loss: 0.0017995834350585938|cri_loss: 0.0103607177734375|unsuper_loss: 0.0 average reward score: -1.2109375 ------------------------------------------------------------------------------------- |E2E latency=14.18s |Gather latency=0.74s (5.21%) |Generate time=9.14s (64.43%) |Training time=2.31s (16.32%) |Others=2.73 (19.25%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.30 epoch: 0|step: 611|ppo_ep: 1|act_loss: 0.0244598388671875|cri_loss: 0.00830841064453125|unsuper_loss: 0.0 average reward score: -1.947265625 ------------------------------------------------------------------------------------- |E2E latency=13.74s |Gather latency=0.82s (5.98%) |Generate time=8.80s (64.04%) |Training time=2.36s (17.18%) |Others=2.58 (18.78%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 612|ppo_ep: 1|act_loss: -0.00016367435455322266|cri_loss: 0.01001739501953125|unsuper_loss: 0.0 average reward score: -1.875 ------------------------------------------------------------------------------------- |E2E latency=13.65s |Gather latency=0.74s (5.43%) |Generate time=8.49s (62.17%) |Training time=2.39s (17.48%) |Others=2.78 (20.35%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 613|ppo_ep: 1|act_loss: -0.05609130859375|cri_loss: 0.026458740234375|unsuper_loss: 0.0 average reward score: -1.701171875 ------------------------------------------------------------------------------------- |E2E latency=13.53s |Gather latency=0.63s (4.67%) |Generate time=8.61s (63.65%) |Training time=2.37s (17.49%) |Others=2.55 (18.86%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.30 epoch: 0|step: 614|ppo_ep: 1|act_loss: -0.0007758140563964844|cri_loss: 0.01580810546875|unsuper_loss: 0.0 average reward score: -2.14453125 ------------------------------------------------------------------------------------- |E2E latency=13.83s |Gather latency=1.18s (8.55%) |Generate time=8.75s (63.28%) |Training time=2.38s (17.21%) |Others=2.70 (19.51%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.30 epoch: 0|step: 615|ppo_ep: 1|act_loss: -0.0418701171875|cri_loss: 0.0252685546875|unsuper_loss: 0.0 average reward score: -1.939453125 ------------------------------------------------------------------------------------- |E2E latency=14.36s |Gather latency=0.95s (6.59%) |Generate time=9.19s (64.02%) |Training time=2.45s (17.04%) |Others=2.72 (18.94%)|CurSamplesPerSec=2.23 |AvgSamplesPerSec=2.30 epoch: 0|step: 616|ppo_ep: 1|act_loss: -0.01110076904296875|cri_loss: 0.01412200927734375|unsuper_loss: 0.0 average reward score: -1.7626953125 ------------------------------------------------------------------------------------- |E2E latency=13.96s |Gather latency=1.10s (7.86%) |Generate time=8.90s (63.77%) |Training time=2.38s (17.07%) |Others=2.67 (19.16%)|CurSamplesPerSec=2.29 |AvgSamplesPerSec=2.30 epoch: 0|step: 617|ppo_ep: 1|act_loss: 0.049072265625|cri_loss: 0.035064697265625|unsuper_loss: 0.0 average reward score: -1.5 ------------------------------------------------------------------------------------- |E2E latency=13.72s |Gather latency=0.97s (7.09%) |Generate time=8.67s (63.19%) |Training time=2.39s (17.46%) |Others=2.66 (19.36%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 618|ppo_ep: 1|act_loss: 0.01531219482421875|cri_loss: 0.019134521484375|unsuper_loss: 0.0 average reward score: -2.349609375 ------------------------------------------------------------------------------------- |E2E latency=13.97s |Gather latency=1.00s (7.16%) |Generate time=8.89s (63.64%) |Training time=2.41s (17.25%) |Others=2.67 (19.11%)|CurSamplesPerSec=2.29 |AvgSamplesPerSec=2.30 [2023-05-17 02:24:54,243] [INFO] [logging.py:96:log_dist] [Rank 0] step=620, skipped=10, lr=[3.3745647506216155e-07, 3.3745647506216155e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 02:24:54,243] [INFO] [timer.py:199:stop] epoch=0/micro_step=620/global_step=620, RunningAvgSamplesPerSec=28.29016468163528, CurrSamplesPerSec=28.920861684747358, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 02:24:55,008] [INFO] [logging.py:96:log_dist] [Rank 0] step=620, skipped=11, lr=[1.7572555417026524e-06, 1.7572555417026524e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 619|ppo_ep: 1|act_loss: -0.03778076171875|cri_loss: 0.0270843505859375|unsuper_loss: 0.0 average reward score: -1.494140625 ------------------------------------------------------------------------------------- |E2E latency=13.76s |Gather latency=0.54s (3.89%) |Generate time=8.73s (63.41%) |Training time=2.29s (16.67%) |Others=2.74 (19.92%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 620|ppo_ep: 1|act_loss: 0.02178955078125|cri_loss: 0.0120086669921875|unsuper_loss: 0.0 average reward score: -1.923828125 ------------------------------------------------------------------------------------- |E2E latency=13.74s |Gather latency=0.88s (6.40%) |Generate time=8.75s (63.68%) |Training time=2.28s (16.61%) |Others=2.71 (19.71%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 621|ppo_ep: 1|act_loss: -0.046661376953125|cri_loss: 0.021392822265625|unsuper_loss: 0.0 average reward score: -1.9501953125 ------------------------------------------------------------------------------------- |E2E latency=13.73s |Gather latency=0.78s (5.69%) |Generate time=8.65s (62.98%) |Training time=2.32s (16.91%) |Others=2.76 (20.12%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 622|ppo_ep: 1|act_loss: 0.00014162063598632812|cri_loss: 0.0308685302734375|unsuper_loss: 0.0 average reward score: -1.28125 ------------------------------------------------------------------------------------- |E2E latency=13.64s |Gather latency=0.84s (6.17%) |Generate time=8.65s (63.43%) |Training time=2.35s (17.24%) |Others=2.64 (19.33%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 623|ppo_ep: 1|act_loss: 0.04180908203125|cri_loss: 0.0135040283203125|unsuper_loss: 0.0 average reward score: -1.712890625 ------------------------------------------------------------------------------------- |E2E latency=13.82s |Gather latency=1.01s (7.33%) |Generate time=8.68s (62.83%) |Training time=2.34s (16.91%) |Others=2.80 (20.26%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 624|ppo_ep: 1|act_loss: -0.00894927978515625|cri_loss: 0.017333984375|unsuper_loss: 0.0 average reward score: -1.6328125 ------------------------------------------------------------------------------------- |E2E latency=13.64s |Gather latency=0.84s (6.17%) |Generate time=8.80s (64.52%) |Training time=2.32s (17.05%) |Others=2.51 (18.43%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 625|ppo_ep: 1|act_loss: -0.0034656524658203125|cri_loss: 0.0094451904296875|unsuper_loss: 0.0 average reward score: -1.3193359375 ------------------------------------------------------------------------------------- |E2E latency=13.55s |Gather latency=0.81s (5.99%) |Generate time=8.69s (64.15%) |Training time=2.26s (16.69%) |Others=2.60 (19.16%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.30 epoch: 0|step: 626|ppo_ep: 1|act_loss: 0.06646728515625|cri_loss: 0.011932373046875|unsuper_loss: 0.0 average reward score: -1.677734375 ------------------------------------------------------------------------------------- |E2E latency=13.35s |Gather latency=0.77s (5.78%) |Generate time=8.19s (61.36%) |Training time=2.43s (18.19%) |Others=2.73 (20.45%)|CurSamplesPerSec=2.40 |AvgSamplesPerSec=2.30 [2023-05-17 02:26:44,042] [INFO] [loss_scaler.py:188:update_scale] [deepspeed] OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 65536, but hysteresis is 2. Reducing hysteresis to 1 epoch: 0|step: 627|ppo_ep: 1|act_loss: -0.00013935565948486328|cri_loss: 0.005840301513671875|unsuper_loss: 0.0 average reward score: -2.2734375 ------------------------------------------------------------------------------------- |E2E latency=13.56s |Gather latency=0.60s (4.41%) |Generate time=8.66s (63.84%) |Training time=2.36s (17.43%) |Others=2.54 (18.74%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.30 [2023-05-17 02:26:58,054] [INFO] [loss_scaler.py:181:update_scale] [deepspeed] OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 65536, reducing to 32768 epoch: 0|step: 628|ppo_ep: 1|act_loss: -0.0229949951171875|cri_loss: 0.01898193359375|unsuper_loss: 0.0 average reward score: -0.8720703125 ------------------------------------------------------------------------------------- |E2E latency=14.01s |Gather latency=0.74s (5.28%) |Generate time=8.75s (62.42%) |Training time=2.56s (18.29%) |Others=2.70 (19.29%)|CurSamplesPerSec=2.28 |AvgSamplesPerSec=2.30 [2023-05-17 02:27:11,521] [INFO] [logging.py:96:log_dist] [Rank 0] step=630, skipped=10, lr=[3.2062975441820165e-07, 3.2062975441820165e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 02:27:11,521] [INFO] [timer.py:199:stop] epoch=0/micro_step=630/global_step=630, RunningAvgSamplesPerSec=28.29563141004547, CurrSamplesPerSec=27.360751355014102, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 02:27:12,264] [INFO] [logging.py:96:log_dist] [Rank 0] step=630, skipped=13, lr=[1.6873357671082662e-06, 1.6873357671082662e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 629|ppo_ep: 1|act_loss: 0.08697509765625|cri_loss: 0.0202178955078125|unsuper_loss: 0.0 average reward score: -2.318359375 ------------------------------------------------------------------------------------- |E2E latency=14.21s |Gather latency=0.67s (4.75%) |Generate time=9.12s (64.14%) |Training time=2.57s (18.07%) |Others=2.53 (17.79%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.30 epoch: 0|step: 630|ppo_ep: 1|act_loss: 0.01219940185546875|cri_loss: 0.0068511962890625|unsuper_loss: 0.0 average reward score: -1.3515625 ------------------------------------------------------------------------------------- |E2E latency=13.69s |Gather latency=0.71s (5.16%) |Generate time=8.74s (63.82%) |Training time=2.33s (17.03%) |Others=2.62 (19.16%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 631|ppo_ep: 1|act_loss: -0.04315185546875|cri_loss: 0.01654052734375|unsuper_loss: 0.0 average reward score: -1.2431640625 ------------------------------------------------------------------------------------- |E2E latency=13.78s |Gather latency=0.63s (4.61%) |Generate time=8.63s (62.64%) |Training time=2.39s (17.35%) |Others=2.76 (20.01%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 632|ppo_ep: 1|act_loss: -0.01373291015625|cri_loss: 0.01151275634765625|unsuper_loss: 0.0 average reward score: -1.8349609375 ------------------------------------------------------------------------------------- |E2E latency=14.02s |Gather latency=0.61s (4.37%) |Generate time=9.15s (65.27%) |Training time=2.33s (16.61%) |Others=2.54 (18.11%)|CurSamplesPerSec=2.28 |AvgSamplesPerSec=2.30 epoch: 0|step: 633|ppo_ep: 1|act_loss: -0.0024261474609375|cri_loss: 0.01013946533203125|unsuper_loss: 0.0 average reward score: -2.123046875 ------------------------------------------------------------------------------------- |E2E latency=13.70s |Gather latency=0.80s (5.84%) |Generate time=8.70s (63.51%) |Training time=2.26s (16.51%) |Others=2.74 (19.98%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 634|ppo_ep: 1|act_loss: -0.0134429931640625|cri_loss: 0.006458282470703125|unsuper_loss: 0.0 average reward score: -2.1484375 ------------------------------------------------------------------------------------- |E2E latency=14.03s |Gather latency=0.66s (4.69%) |Generate time=8.86s (63.14%) |Training time=2.42s (17.23%) |Others=2.75 (19.63%)|CurSamplesPerSec=2.28 |AvgSamplesPerSec=2.30 epoch: 0|step: 635|ppo_ep: 1|act_loss: -0.0635986328125|cri_loss: 0.013580322265625|unsuper_loss: 0.0 average reward score: -2.015625 ------------------------------------------------------------------------------------- |E2E latency=13.67s |Gather latency=0.68s (4.97%) |Generate time=8.77s (64.15%) |Training time=2.31s (16.94%) |Others=2.59 (18.92%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 636|ppo_ep: 1|act_loss: -0.046173095703125|cri_loss: 0.006603240966796875|unsuper_loss: 0.0 average reward score: -2.517578125 ------------------------------------------------------------------------------------- |E2E latency=13.79s |Gather latency=0.73s (5.27%) |Generate time=8.74s (63.40%) |Training time=2.45s (17.78%) |Others=2.60 (18.83%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 637|ppo_ep: 1|act_loss: 0.023590087890625|cri_loss: 0.00945281982421875|unsuper_loss: 0.0 average reward score: -2.2265625 ------------------------------------------------------------------------------------- |E2E latency=14.08s |Gather latency=0.96s (6.81%) |Generate time=9.07s (64.40%) |Training time=2.39s (17.00%) |Others=2.62 (18.60%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.30 epoch: 0|step: 638|ppo_ep: 1|act_loss: -0.0216522216796875|cri_loss: 0.01483917236328125|unsuper_loss: 0.0 average reward score: -1.73046875 ------------------------------------------------------------------------------------- |E2E latency=14.19s |Gather latency=0.99s (6.98%) |Generate time=9.20s (64.81%) |Training time=2.35s (16.52%) |Others=2.65 (18.67%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.30 [2023-05-17 02:29:29,955] [INFO] [logging.py:96:log_dist] [Rank 0] step=640, skipped=10, lr=[3.040220629922622e-07, 3.040220629922622e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 02:29:29,955] [INFO] [timer.py:199:stop] epoch=0/micro_step=640/global_step=640, RunningAvgSamplesPerSec=28.299792760702655, CurrSamplesPerSec=27.93538910917652, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 02:29:30,708] [INFO] [logging.py:96:log_dist] [Rank 0] step=640, skipped=13, lr=[1.6009327938338826e-06, 1.6009327938338826e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 639|ppo_ep: 1|act_loss: -0.00714874267578125|cri_loss: 0.00881195068359375|unsuper_loss: 0.0 average reward score: -1.337890625 ------------------------------------------------------------------------------------- |E2E latency=13.51s |Gather latency=0.83s (6.16%) |Generate time=8.47s (62.70%) |Training time=2.34s (17.32%) |Others=2.70 (19.98%)|CurSamplesPerSec=2.37 |AvgSamplesPerSec=2.30 epoch: 0|step: 640|ppo_ep: 1|act_loss: -0.0042572021484375|cri_loss: 0.0196075439453125|unsuper_loss: 0.0 average reward score: -3.0546875 ------------------------------------------------------------------------------------- |E2E latency=14.05s |Gather latency=0.67s (4.80%) |Generate time=9.17s (65.29%) |Training time=2.32s (16.53%) |Others=2.56 (18.19%)|CurSamplesPerSec=2.28 |AvgSamplesPerSec=2.30 epoch: 0|step: 641|ppo_ep: 1|act_loss: 0.05078125|cri_loss: 0.0168914794921875|unsuper_loss: 0.0 average reward score: -1.228515625 ------------------------------------------------------------------------------------- |E2E latency=14.30s |Gather latency=0.70s (4.92%) |Generate time=9.09s (63.58%) |Training time=2.46s (17.17%) |Others=2.75 (19.25%)|CurSamplesPerSec=2.24 |AvgSamplesPerSec=2.30 epoch: 0|step: 642|ppo_ep: 1|act_loss: 0.03826904296875|cri_loss: 0.0136871337890625|unsuper_loss: 0.0 average reward score: -1.5048828125 ------------------------------------------------------------------------------------- |E2E latency=13.57s |Gather latency=0.67s (4.90%) |Generate time=8.73s (64.31%) |Training time=2.26s (16.67%) |Others=2.58 (19.02%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.30 epoch: 0|step: 643|ppo_ep: 1|act_loss: -0.01512908935546875|cri_loss: 0.01261138916015625|unsuper_loss: 0.0 average reward score: -2.150390625 ------------------------------------------------------------------------------------- |E2E latency=13.76s |Gather latency=0.86s (6.24%) |Generate time=8.76s (63.64%) |Training time=2.35s (17.04%) |Others=2.66 (19.32%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 644|ppo_ep: 1|act_loss: 0.020660400390625|cri_loss: 0.0116424560546875|unsuper_loss: 0.0 average reward score: -0.52783203125 ------------------------------------------------------------------------------------- |E2E latency=14.09s |Gather latency=0.86s (6.08%) |Generate time=9.02s (64.03%) |Training time=2.32s (16.43%) |Others=2.75 (19.54%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.30 epoch: 0|step: 645|ppo_ep: 1|act_loss: 0.038482666015625|cri_loss: 0.0166168212890625|unsuper_loss: 0.0 average reward score: -0.81884765625 ------------------------------------------------------------------------------------- |E2E latency=13.62s |Gather latency=0.67s (4.89%) |Generate time=8.70s (63.87%) |Training time=2.36s (17.30%) |Others=2.57 (18.84%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 646|ppo_ep: 1|act_loss: -0.0236358642578125|cri_loss: 0.0205535888671875|unsuper_loss: 0.0 average reward score: -1.6396484375 ------------------------------------------------------------------------------------- |E2E latency=13.94s |Gather latency=0.78s (5.63%) |Generate time=8.78s (63.00%) |Training time=2.33s (16.69%) |Others=2.83 (20.31%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.30 epoch: 0|step: 647|ppo_ep: 1|act_loss: -0.003261566162109375|cri_loss: 0.00977325439453125|unsuper_loss: 0.0 average reward score: -1.361328125 ------------------------------------------------------------------------------------- |E2E latency=13.03s |Gather latency=0.75s (5.74%) |Generate time=8.02s (61.59%) |Training time=2.36s (18.09%) |Others=2.65 (20.33%)|CurSamplesPerSec=2.46 |AvgSamplesPerSec=2.30 epoch: 0|step: 648|ppo_ep: 1|act_loss: 0.01056671142578125|cri_loss: 0.00994873046875|unsuper_loss: 0.0 average reward score: -1.732421875 ------------------------------------------------------------------------------------- |E2E latency=14.40s |Gather latency=0.67s (4.66%) |Generate time=9.27s (64.39%) |Training time=2.38s (16.55%) |Others=2.75 (19.06%)|CurSamplesPerSec=2.22 |AvgSamplesPerSec=2.30 [2023-05-17 02:31:49,022] [INFO] [logging.py:96:log_dist] [Rank 0] step=650, skipped=10, lr=[2.8765587291721167e-07, 2.8765587291721167e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 02:31:49,022] [INFO] [timer.py:199:stop] epoch=0/micro_step=650/global_step=650, RunningAvgSamplesPerSec=28.31047362162067, CurrSamplesPerSec=29.322172280618506, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 02:31:49,791] [INFO] [logging.py:96:log_dist] [Rank 0] step=650, skipped=13, lr=[1.5157463627745716e-06, 1.5157463627745716e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 649|ppo_ep: 1|act_loss: -0.00870513916015625|cri_loss: 0.01296234130859375|unsuper_loss: 0.0 average reward score: -1.45703125 ------------------------------------------------------------------------------------- |E2E latency=14.31s |Gather latency=0.77s (5.38%) |Generate time=9.15s (63.95%) |Training time=2.44s (17.08%) |Others=2.72 (18.97%)|CurSamplesPerSec=2.24 |AvgSamplesPerSec=2.30 epoch: 0|step: 650|ppo_ep: 1|act_loss: -0.00559234619140625|cri_loss: 0.009735107421875|unsuper_loss: 0.0 average reward score: -1.5400390625 ------------------------------------------------------------------------------------- |E2E latency=14.26s |Gather latency=0.35s (2.49%) |Generate time=9.22s (64.67%) |Training time=2.38s (16.70%) |Others=2.66 (18.63%)|CurSamplesPerSec=2.24 |AvgSamplesPerSec=2.30 epoch: 0|step: 651|ppo_ep: 1|act_loss: -0.02899169921875|cri_loss: 0.0116119384765625|unsuper_loss: 0.0 average reward score: -1.4462890625 ------------------------------------------------------------------------------------- |E2E latency=13.73s |Gather latency=0.52s (3.81%) |Generate time=8.77s (63.86%) |Training time=2.37s (17.26%) |Others=2.59 (18.87%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 652|ppo_ep: 1|act_loss: 0.050628662109375|cri_loss: 0.0245819091796875|unsuper_loss: 0.0 average reward score: -1.7919921875 ------------------------------------------------------------------------------------- |E2E latency=14.21s |Gather latency=1.02s (7.21%) |Generate time=9.18s (64.55%) |Training time=2.43s (17.12%) |Others=2.60 (18.32%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.30 epoch: 0|step: 653|ppo_ep: 1|act_loss: -0.013031005859375|cri_loss: 0.01236724853515625|unsuper_loss: 0.0 average reward score: -1.779296875 ------------------------------------------------------------------------------------- |E2E latency=13.69s |Gather latency=0.85s (6.18%) |Generate time=8.71s (63.66%) |Training time=2.29s (16.75%) |Others=2.68 (19.59%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 654|ppo_ep: 1|act_loss: -0.0029544830322265625|cri_loss: 0.009552001953125|unsuper_loss: 0.0 average reward score: -1.9453125 ------------------------------------------------------------------------------------- |E2E latency=13.91s |Gather latency=0.70s (5.00%) |Generate time=8.92s (64.09%) |Training time=2.35s (16.89%) |Others=2.65 (19.02%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.30 epoch: 0|step: 655|ppo_ep: 1|act_loss: -0.029052734375|cri_loss: 0.01348876953125|unsuper_loss: 0.0 average reward score: -1.49609375 ------------------------------------------------------------------------------------- |E2E latency=13.54s |Gather latency=1.11s (8.17%) |Generate time=8.66s (63.99%) |Training time=2.33s (17.24%) |Others=2.54 (18.77%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.30 epoch: 0|step: 656|ppo_ep: 1|act_loss: -0.0101470947265625|cri_loss: 0.0090789794921875|unsuper_loss: 0.0 average reward score: -1.64453125 ------------------------------------------------------------------------------------- |E2E latency=13.59s |Gather latency=0.64s (4.68%) |Generate time=8.68s (63.85%) |Training time=2.37s (17.42%) |Others=2.54 (18.73%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 657|ppo_ep: 1|act_loss: 0.0830078125|cri_loss: 0.0179290771484375|unsuper_loss: 0.0 average reward score: -1.2373046875 ------------------------------------------------------------------------------------- |E2E latency=14.35s |Gather latency=0.88s (6.15%) |Generate time=9.12s (63.59%) |Training time=2.47s (17.21%) |Others=2.76 (19.20%)|CurSamplesPerSec=2.23 |AvgSamplesPerSec=2.30 epoch: 0|step: 658|ppo_ep: 1|act_loss: -0.04168701171875|cri_loss: 0.01244354248046875|unsuper_loss: 0.0 average reward score: -2.029296875 ------------------------------------------------------------------------------------- |E2E latency=13.73s |Gather latency=0.72s (5.21%) |Generate time=8.64s (62.92%) |Training time=2.37s (17.24%) |Others=2.72 (19.84%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 [2023-05-17 02:34:08,369] [INFO] [logging.py:96:log_dist] [Rank 0] step=660, skipped=10, lr=[2.7155332954657816e-07, 2.7155332954657816e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 02:34:08,370] [INFO] [timer.py:199:stop] epoch=0/micro_step=660/global_step=660, RunningAvgSamplesPerSec=28.312949839956236, CurrSamplesPerSec=29.307471632062068, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 02:34:09,125] [INFO] [logging.py:96:log_dist] [Rank 0] step=660, skipped=13, lr=[1.4318917410525508e-06, 1.4318917410525508e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 659|ppo_ep: 1|act_loss: 0.00978851318359375|cri_loss: 0.0124053955078125|unsuper_loss: 0.0 average reward score: -1.990234375 ------------------------------------------------------------------------------------- |E2E latency=14.33s |Gather latency=1.25s (8.70%) |Generate time=9.17s (63.98%) |Training time=2.39s (16.65%) |Others=2.78 (19.37%)|CurSamplesPerSec=2.23 |AvgSamplesPerSec=2.30 epoch: 0|step: 660|ppo_ep: 1|act_loss: -0.037078857421875|cri_loss: 0.0157470703125|unsuper_loss: 0.0 average reward score: -1.3095703125 ------------------------------------------------------------------------------------- |E2E latency=14.27s |Gather latency=0.99s (6.91%) |Generate time=9.17s (64.30%) |Training time=2.37s (16.59%) |Others=2.73 (19.11%)|CurSamplesPerSec=2.24 |AvgSamplesPerSec=2.30 epoch: 0|step: 661|ppo_ep: 1|act_loss: -0.016754150390625|cri_loss: 0.0091552734375|unsuper_loss: 0.0 average reward score: -1.3642578125 ------------------------------------------------------------------------------------- |E2E latency=13.85s |Gather latency=0.96s (6.95%) |Generate time=8.78s (63.43%) |Training time=2.36s (17.01%) |Others=2.71 (19.56%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.30 epoch: 0|step: 662|ppo_ep: 1|act_loss: -0.08233642578125|cri_loss: 0.0275421142578125|unsuper_loss: 0.0 average reward score: -1.6416015625 ------------------------------------------------------------------------------------- |E2E latency=13.66s |Gather latency=1.10s (8.03%) |Generate time=8.61s (63.02%) |Training time=2.34s (17.11%) |Others=2.71 (19.87%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 663|ppo_ep: 1|act_loss: 0.0267181396484375|cri_loss: 0.0156707763671875|unsuper_loss: 0.0 average reward score: -1.5634765625 ------------------------------------------------------------------------------------- |E2E latency=13.66s |Gather latency=0.93s (6.78%) |Generate time=8.78s (64.22%) |Training time=2.32s (16.96%) |Others=2.57 (18.82%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 664|ppo_ep: 1|act_loss: 0.039306640625|cri_loss: 0.00724029541015625|unsuper_loss: 0.0 average reward score: -1.841796875 ------------------------------------------------------------------------------------- |E2E latency=14.37s |Gather latency=0.55s (3.85%) |Generate time=9.33s (64.93%) |Training time=2.42s (16.83%) |Others=2.62 (18.24%)|CurSamplesPerSec=2.23 |AvgSamplesPerSec=2.30 epoch: 0|step: 665|ppo_ep: 1|act_loss: 0.03790283203125|cri_loss: 0.01329803466796875|unsuper_loss: 0.0 average reward score: -0.599609375 ------------------------------------------------------------------------------------- |E2E latency=13.84s |Gather latency=0.59s (4.25%) |Generate time=8.78s (63.42%) |Training time=2.35s (17.00%) |Others=2.71 (19.58%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.30 epoch: 0|step: 666|ppo_ep: 1|act_loss: 0.02227783203125|cri_loss: 0.0086212158203125|unsuper_loss: 0.0 average reward score: -1.947265625 ------------------------------------------------------------------------------------- |E2E latency=13.86s |Gather latency=0.72s (5.18%) |Generate time=8.73s (63.00%) |Training time=2.39s (17.27%) |Others=2.73 (19.73%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.30 epoch: 0|step: 667|ppo_ep: 1|act_loss: -0.0037212371826171875|cri_loss: 0.019287109375|unsuper_loss: 0.0 average reward score: -1.052734375 ------------------------------------------------------------------------------------- |E2E latency=14.01s |Gather latency=0.99s (7.04%) |Generate time=8.75s (62.46%) |Training time=2.39s (17.09%) |Others=2.86 (20.45%)|CurSamplesPerSec=2.28 |AvgSamplesPerSec=2.30 epoch: 0|step: 668|ppo_ep: 1|act_loss: 0.01959228515625|cri_loss: 0.00812530517578125|unsuper_loss: 0.0 average reward score: -1.48046875 ------------------------------------------------------------------------------------- |E2E latency=13.65s |Gather latency=1.02s (7.47%) |Generate time=8.73s (63.90%) |Training time=2.39s (17.49%) |Others=2.54 (18.60%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 [2023-05-17 02:36:27,134] [INFO] [logging.py:96:log_dist] [Rank 0] step=670, skipped=10, lr=[2.557362214893183e-07, 2.557362214893183e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 02:36:27,134] [INFO] [timer.py:199:stop] epoch=0/micro_step=670/global_step=670, RunningAvgSamplesPerSec=28.313127425007544, CurrSamplesPerSec=28.771624207330692, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 02:36:27,911] [INFO] [logging.py:96:log_dist] [Rank 0] step=670, skipped=13, lr=[1.3494823936975263e-06, 1.3494823936975263e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 669|ppo_ep: 1|act_loss: -0.08087158203125|cri_loss: 0.011749267578125|unsuper_loss: 0.0 average reward score: -0.8876953125 ------------------------------------------------------------------------------------- |E2E latency=13.61s |Gather latency=1.21s (8.87%) |Generate time=8.65s (63.58%) |Training time=2.30s (16.87%) |Others=2.66 (19.55%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 670|ppo_ep: 1|act_loss: -0.007282257080078125|cri_loss: 0.01079559326171875|unsuper_loss: 0.0 average reward score: -1.6201171875 ------------------------------------------------------------------------------------- |E2E latency=13.53s |Gather latency=1.07s (7.90%) |Generate time=8.60s (63.58%) |Training time=2.36s (17.48%) |Others=2.56 (18.94%)|CurSamplesPerSec=2.37 |AvgSamplesPerSec=2.30 epoch: 0|step: 671|ppo_ep: 1|act_loss: 0.027740478515625|cri_loss: 0.01302337646484375|unsuper_loss: 0.0 average reward score: -2.42578125 ------------------------------------------------------------------------------------- |E2E latency=13.70s |Gather latency=0.73s (5.33%) |Generate time=8.74s (63.77%) |Training time=2.33s (17.01%) |Others=2.63 (19.22%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 672|ppo_ep: 1|act_loss: -0.0003845691680908203|cri_loss: 0.008514404296875|unsuper_loss: 0.0 average reward score: -1.705078125 ------------------------------------------------------------------------------------- |E2E latency=13.85s |Gather latency=0.79s (5.69%) |Generate time=8.79s (63.45%) |Training time=2.39s (17.24%) |Others=2.68 (19.31%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.30 epoch: 0|step: 673|ppo_ep: 1|act_loss: 0.00873565673828125|cri_loss: 0.006557464599609375|unsuper_loss: 0.0 average reward score: -1.96484375 ------------------------------------------------------------------------------------- |E2E latency=13.82s |Gather latency=1.01s (7.27%) |Generate time=8.71s (63.02%) |Training time=2.40s (17.33%) |Others=2.71 (19.64%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 674|ppo_ep: 1|act_loss: 0.01171112060546875|cri_loss: 0.00666046142578125|unsuper_loss: 0.0 average reward score: -2.021484375 ------------------------------------------------------------------------------------- |E2E latency=13.64s |Gather latency=1.09s (7.98%) |Generate time=8.71s (63.84%) |Training time=2.31s (16.94%) |Others=2.62 (19.22%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 675|ppo_ep: 1|act_loss: -0.044586181640625|cri_loss: 0.02587890625|unsuper_loss: 0.0 average reward score: -1.470703125 ------------------------------------------------------------------------------------- |E2E latency=13.80s |Gather latency=0.69s (5.00%) |Generate time=8.71s (63.15%) |Training time=2.36s (17.09%) |Others=2.73 (19.76%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 676|ppo_ep: 1|act_loss: 0.0462646484375|cri_loss: 0.006374359130859375|unsuper_loss: 0.0 average reward score: -2.24609375 ------------------------------------------------------------------------------------- |E2E latency=13.58s |Gather latency=0.89s (6.53%) |Generate time=8.73s (64.25%) |Training time=2.32s (17.07%) |Others=2.54 (18.67%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.30 epoch: 0|step: 677|ppo_ep: 1|act_loss: -0.0194854736328125|cri_loss: 0.00836181640625|unsuper_loss: 0.0 average reward score: -1.49609375 ------------------------------------------------------------------------------------- |E2E latency=13.69s |Gather latency=0.90s (6.57%) |Generate time=8.70s (63.55%) |Training time=2.37s (17.35%) |Others=2.61 (19.10%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 678|ppo_ep: 1|act_loss: -0.0132598876953125|cri_loss: 0.01184844970703125|unsuper_loss: 0.0 average reward score: -2.138671875 ------------------------------------------------------------------------------------- |E2E latency=13.76s |Gather latency=0.68s (4.97%) |Generate time=8.69s (63.17%) |Training time=2.38s (17.28%) |Others=2.69 (19.55%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 [2023-05-17 02:38:44,092] [INFO] [logging.py:96:log_dist] [Rank 0] step=680, skipped=10, lr=[2.402259511273e-07, 2.402259511273e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 02:38:44,092] [INFO] [timer.py:199:stop] epoch=0/micro_step=680/global_step=680, RunningAvgSamplesPerSec=28.317132611047473, CurrSamplesPerSec=29.473274981614576, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 02:38:44,874] [INFO] [logging.py:96:log_dist] [Rank 0] step=680, skipped=13, lr=[1.2686298301153394e-06, 1.2686298301153394e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 679|ppo_ep: 1|act_loss: -0.0240631103515625|cri_loss: 0.0211639404296875|unsuper_loss: 0.0 average reward score: -1.5771484375 ------------------------------------------------------------------------------------- |E2E latency=13.59s |Gather latency=0.69s (5.06%) |Generate time=8.65s (63.69%) |Training time=2.31s (16.97%) |Others=2.63 (19.34%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.30 epoch: 0|step: 680|ppo_ep: 1|act_loss: 0.0126800537109375|cri_loss: 0.01204681396484375|unsuper_loss: 0.0 average reward score: -1.1513671875 ------------------------------------------------------------------------------------- |E2E latency=13.76s |Gather latency=0.78s (5.67%) |Generate time=8.79s (63.89%) |Training time=2.35s (17.06%) |Others=2.62 (19.06%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 681|ppo_ep: 1|act_loss: -0.041839599609375|cri_loss: 0.057098388671875|unsuper_loss: 0.0 average reward score: -0.8935546875 ------------------------------------------------------------------------------------- |E2E latency=13.54s |Gather latency=0.80s (5.90%) |Generate time=8.65s (63.88%) |Training time=2.30s (16.97%) |Others=2.59 (19.16%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.30 epoch: 0|step: 682|ppo_ep: 1|act_loss: -0.01641845703125|cri_loss: 0.0109710693359375|unsuper_loss: 0.0 average reward score: -1.69140625 ------------------------------------------------------------------------------------- |E2E latency=13.56s |Gather latency=0.71s (5.27%) |Generate time=8.67s (63.98%) |Training time=2.30s (16.97%) |Others=2.58 (19.05%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.30 [2023-05-17 02:39:38,591] [INFO] [loss_scaler.py:188:update_scale] [deepspeed] OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 65536, but hysteresis is 2. Reducing hysteresis to 1 epoch: 0|step: 683|ppo_ep: 1|act_loss: -0.018524169921875|cri_loss: 0.00911712646484375|unsuper_loss: 0.0 average reward score: -2.15234375 ------------------------------------------------------------------------------------- |E2E latency=13.65s |Gather latency=0.97s (7.07%) |Generate time=8.65s (63.37%) |Training time=2.21s (16.18%) |Others=2.79 (20.45%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 [2023-05-17 02:39:52,306] [INFO] [loss_scaler.py:181:update_scale] [deepspeed] OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 65536, reducing to 32768 epoch: 0|step: 684|ppo_ep: 1|act_loss: 0.01346588134765625|cri_loss: 0.01232147216796875|unsuper_loss: 0.0 average reward score: -1.771484375 ------------------------------------------------------------------------------------- |E2E latency=13.64s |Gather latency=0.76s (5.56%) |Generate time=8.89s (65.18%) |Training time=2.37s (17.35%) |Others=2.38 (17.47%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 685|ppo_ep: 1|act_loss: 0.0117645263671875|cri_loss: 0.01473236083984375|unsuper_loss: 0.0 average reward score: -1.533203125 ------------------------------------------------------------------------------------- |E2E latency=14.48s |Gather latency=0.79s (5.44%) |Generate time=9.27s (64.06%) |Training time=2.59s (17.90%) |Others=2.61 (18.05%)|CurSamplesPerSec=2.21 |AvgSamplesPerSec=2.30 epoch: 0|step: 686|ppo_ep: 1|act_loss: 0.0004067420959472656|cri_loss: 0.01219940185546875|unsuper_loss: 0.0 average reward score: -2.47265625 ------------------------------------------------------------------------------------- |E2E latency=13.51s |Gather latency=0.73s (5.43%) |Generate time=8.31s (61.56%) |Training time=2.42s (17.94%) |Others=2.77 (20.50%)|CurSamplesPerSec=2.37 |AvgSamplesPerSec=2.30 epoch: 0|step: 687|ppo_ep: 1|act_loss: 0.01061248779296875|cri_loss: 0.0206298828125|unsuper_loss: 0.0 average reward score: -0.681640625 ------------------------------------------------------------------------------------- |E2E latency=13.63s |Gather latency=0.75s (5.49%) |Generate time=8.70s (63.83%) |Training time=2.33s (17.12%) |Others=2.60 (19.04%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 688|ppo_ep: 1|act_loss: 0.032745361328125|cri_loss: 0.00878143310546875|unsuper_loss: 0.0 average reward score: -2.091796875 ------------------------------------------------------------------------------------- |E2E latency=13.64s |Gather latency=0.91s (6.67%) |Generate time=8.69s (63.67%) |Training time=2.26s (16.60%) |Others=2.69 (19.73%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 [2023-05-17 02:41:01,673] [INFO] [logging.py:96:log_dist] [Rank 0] step=690, skipped=12, lr=[2.2805277605167803e-07, 2.2805277605167803e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 02:41:01,674] [INFO] [timer.py:199:stop] epoch=0/micro_step=690/global_step=690, RunningAvgSamplesPerSec=28.335173200914603, CurrSamplesPerSec=29.493754061173217, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 02:41:02,404] [INFO] [logging.py:96:log_dist] [Rank 0] step=690, skipped=13, lr=[1.1894434532028176e-06, 1.1894434532028176e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 689|ppo_ep: 1|act_loss: -0.019775390625|cri_loss: 0.01354217529296875|unsuper_loss: 0.0 average reward score: -1.68359375 ------------------------------------------------------------------------------------- |E2E latency=14.13s |Gather latency=0.62s (4.38%) |Generate time=9.22s (65.27%) |Training time=2.34s (16.53%) |Others=2.57 (18.20%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.30 epoch: 0|step: 690|ppo_ep: 1|act_loss: 0.004909515380859375|cri_loss: 0.0097198486328125|unsuper_loss: 0.0 average reward score: -1.8525390625 ------------------------------------------------------------------------------------- |E2E latency=13.63s |Gather latency=0.67s (4.94%) |Generate time=8.61s (63.18%) |Training time=2.31s (16.96%) |Others=2.71 (19.86%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 691|ppo_ep: 1|act_loss: -0.0211181640625|cri_loss: 0.00823974609375|unsuper_loss: 0.0 average reward score: -1.390625 ------------------------------------------------------------------------------------- |E2E latency=14.19s |Gather latency=0.85s (6.00%) |Generate time=9.09s (64.08%) |Training time=2.33s (16.42%) |Others=2.77 (19.49%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.30 epoch: 0|step: 692|ppo_ep: 1|act_loss: 0.0081939697265625|cri_loss: 0.016387939453125|unsuper_loss: 0.0 average reward score: -1.84375 ------------------------------------------------------------------------------------- |E2E latency=13.53s |Gather latency=0.65s (4.84%) |Generate time=8.64s (63.80%) |Training time=2.33s (17.18%) |Others=2.57 (19.01%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.30 epoch: 0|step: 693|ppo_ep: 1|act_loss: 0.0252685546875|cri_loss: 0.00525665283203125|unsuper_loss: 0.0 average reward score: -0.93115234375 ------------------------------------------------------------------------------------- |E2E latency=14.03s |Gather latency=0.75s (5.37%) |Generate time=8.89s (63.36%) |Training time=2.42s (17.27%) |Others=2.72 (19.37%)|CurSamplesPerSec=2.28 |AvgSamplesPerSec=2.30 epoch: 0|step: 694|ppo_ep: 1|act_loss: -0.0036525726318359375|cri_loss: 0.00811767578125|unsuper_loss: 0.0 average reward score: -1.6796875 ------------------------------------------------------------------------------------- |E2E latency=13.66s |Gather latency=0.96s (7.03%) |Generate time=8.70s (63.71%) |Training time=2.36s (17.30%) |Others=2.59 (18.98%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.30 epoch: 0|step: 695|ppo_ep: 1|act_loss: 0.0433349609375|cri_loss: 0.0093536376953125|unsuper_loss: 0.0 average reward score: -1.73046875 ------------------------------------------------------------------------------------- |E2E latency=13.74s |Gather latency=0.75s (5.49%) |Generate time=8.71s (63.42%) |Training time=2.33s (16.94%) |Others=2.70 (19.63%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 696|ppo_ep: 1|act_loss: 0.013885498046875|cri_loss: 0.00783538818359375|unsuper_loss: 0.0 average reward score: -0.9873046875 ------------------------------------------------------------------------------------- |E2E latency=14.18s |Gather latency=0.85s (5.98%) |Generate time=9.10s (64.17%) |Training time=2.41s (17.02%) |Others=2.67 (18.82%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.30 epoch: 0|step: 697|ppo_ep: 1|act_loss: 0.01031494140625|cri_loss: 0.0089874267578125|unsuper_loss: 0.0 average reward score: -0.8291015625 ------------------------------------------------------------------------------------- |E2E latency=14.18s |Gather latency=0.68s (4.76%) |Generate time=9.14s (64.43%) |Training time=2.36s (16.64%) |Others=2.68 (18.93%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.30 epoch: 0|step: 698|ppo_ep: 1|act_loss: 0.023406982421875|cri_loss: 0.0195159912109375|unsuper_loss: 0.0 average reward score: -1.064453125 ------------------------------------------------------------------------------------- |E2E latency=13.76s |Gather latency=0.69s (4.98%) |Generate time=8.66s (62.95%) |Training time=2.40s (17.42%) |Others=2.70 (19.62%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 [2023-05-17 02:43:20,805] [INFO] [logging.py:96:log_dist] [Rank 0] step=700, skipped=12, lr=[2.1314740412203787e-07, 2.1314740412203787e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 02:43:20,806] [INFO] [timer.py:199:stop] epoch=0/micro_step=700/global_step=700, RunningAvgSamplesPerSec=28.34132714757388, CurrSamplesPerSec=28.401396266915857, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 02:43:21,516] [INFO] [logging.py:96:log_dist] [Rank 0] step=700, skipped=13, lr=[1.112030411312954e-06, 1.112030411312954e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 699|ppo_ep: 1|act_loss: -0.033477783203125|cri_loss: 0.00823974609375|unsuper_loss: 0.0 average reward score: -1.408203125 ------------------------------------------------------------------------------------- |E2E latency=14.21s |Gather latency=0.72s (5.04%) |Generate time=9.15s (64.39%) |Training time=2.36s (16.58%) |Others=2.70 (19.03%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.30 epoch: 0|step: 700|ppo_ep: 1|act_loss: -0.02325439453125|cri_loss: 0.011505126953125|unsuper_loss: 0.0 average reward score: -1.484375 ------------------------------------------------------------------------------------- |E2E latency=14.19s |Gather latency=0.73s (5.16%) |Generate time=9.14s (64.42%) |Training time=2.35s (16.53%) |Others=2.70 (19.06%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.30 epoch: 0|step: 701|ppo_ep: 1|act_loss: 0.035369873046875|cri_loss: 0.0099945068359375|unsuper_loss: 0.0 average reward score: -0.72802734375 ------------------------------------------------------------------------------------- |E2E latency=12.79s |Gather latency=0.90s (7.00%) |Generate time=7.59s (59.33%) |Training time=2.35s (18.40%) |Others=2.85 (22.27%)|CurSamplesPerSec=2.50 |AvgSamplesPerSec=2.30 epoch: 0|step: 702|ppo_ep: 1|act_loss: 0.026397705078125|cri_loss: 0.00609588623046875|unsuper_loss: 0.0 average reward score: -1.732421875 ------------------------------------------------------------------------------------- |E2E latency=14.07s |Gather latency=0.78s (5.57%) |Generate time=9.02s (64.08%) |Training time=2.33s (16.57%) |Others=2.72 (19.35%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.30 epoch: 0|step: 703|ppo_ep: 1|act_loss: 0.06982421875|cri_loss: 0.020751953125|unsuper_loss: 0.0 average reward score: -2.119140625 ------------------------------------------------------------------------------------- |E2E latency=13.74s |Gather latency=0.68s (4.95%) |Generate time=8.71s (63.44%) |Training time=2.35s (17.09%) |Others=2.67 (19.47%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 704|ppo_ep: 1|act_loss: 0.035308837890625|cri_loss: 0.0085296630859375|unsuper_loss: 0.0 average reward score: -2.060546875 ------------------------------------------------------------------------------------- |E2E latency=13.86s |Gather latency=0.87s (6.28%) |Generate time=8.68s (62.64%) |Training time=2.40s (17.33%) |Others=2.78 (20.03%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.30 epoch: 0|step: 705|ppo_ep: 1|act_loss: 0.017303466796875|cri_loss: 0.00820159912109375|unsuper_loss: 0.0 average reward score: -2.373046875 ------------------------------------------------------------------------------------- |E2E latency=13.54s |Gather latency=0.73s (5.36%) |Generate time=8.61s (63.61%) |Training time=2.35s (17.34%) |Others=2.58 (19.05%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.30 epoch: 0|step: 706|ppo_ep: 1|act_loss: 0.04205322265625|cri_loss: 0.016265869140625|unsuper_loss: 0.0 average reward score: -1.5458984375 ------------------------------------------------------------------------------------- |E2E latency=13.79s |Gather latency=1.07s (7.73%) |Generate time=8.71s (63.20%) |Training time=2.44s (17.71%) |Others=2.63 (19.09%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 707|ppo_ep: 1|act_loss: 0.026580810546875|cri_loss: 0.0125885009765625|unsuper_loss: 0.0 average reward score: -1.314453125 ------------------------------------------------------------------------------------- |E2E latency=14.34s |Gather latency=0.89s (6.18%) |Generate time=8.62s (60.13%) |Training time=2.97s (20.70%) |Others=2.75 (19.16%)|CurSamplesPerSec=2.23 |AvgSamplesPerSec=2.30 epoch: 0|step: 708|ppo_ep: 1|act_loss: -0.010406494140625|cri_loss: 0.0091705322265625|unsuper_loss: 0.0 average reward score: -1.76953125 ------------------------------------------------------------------------------------- |E2E latency=14.79s |Gather latency=0.60s (4.07%) |Generate time=9.50s (64.26%) |Training time=2.41s (16.28%) |Others=2.88 (19.46%)|CurSamplesPerSec=2.16 |AvgSamplesPerSec=2.30 [2023-05-17 02:45:39,454] [INFO] [logging.py:96:log_dist] [Rank 0] step=710, skipped=12, lr=[1.986064974975245e-07, 1.986064974975245e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 02:45:39,454] [INFO] [timer.py:199:stop] epoch=0/micro_step=710/global_step=710, RunningAvgSamplesPerSec=28.31769669551533, CurrSamplesPerSec=27.386236744577904, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 02:45:40,195] [INFO] [logging.py:96:log_dist] [Rank 0] step=710, skipped=13, lr=[1.0364954532707656e-06, 1.0364954532707656e-06], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 709|ppo_ep: 1|act_loss: -0.020172119140625|cri_loss: 0.01122283935546875|unsuper_loss: 0.0 average reward score: -1.62109375 ------------------------------------------------------------------------------------- |E2E latency=13.58s |Gather latency=0.96s (7.06%) |Generate time=8.39s (61.77%) |Training time=2.42s (17.83%) |Others=2.77 (20.40%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.30 epoch: 0|step: 710|ppo_ep: 1|act_loss: 0.0088348388671875|cri_loss: 0.00907135009765625|unsuper_loss: 0.0 average reward score: -1.0048828125 ------------------------------------------------------------------------------------- |E2E latency=13.73s |Gather latency=0.72s (5.26%) |Generate time=8.69s (63.33%) |Training time=2.31s (16.84%) |Others=2.72 (19.83%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.30 epoch: 0|step: 711|ppo_ep: 1|act_loss: 0.0233001708984375|cri_loss: 0.01079559326171875|unsuper_loss: 0.0 average reward score: -1.3232421875 ------------------------------------------------------------------------------------- |E2E latency=13.78s |Gather latency=1.18s (8.57%) |Generate time=8.73s (63.30%) |Training time=2.35s (17.04%) |Others=2.71 (19.66%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 712|ppo_ep: 1|act_loss: 0.00971221923828125|cri_loss: 0.01053619384765625|unsuper_loss: 0.0 average reward score: -1.962890625 ------------------------------------------------------------------------------------- |E2E latency=13.79s |Gather latency=0.75s (5.46%) |Generate time=8.68s (62.95%) |Training time=2.35s (17.07%) |Others=2.76 (19.98%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 713|ppo_ep: 1|act_loss: -0.017486572265625|cri_loss: 0.00738525390625|unsuper_loss: 0.0 average reward score: -2.771484375 ------------------------------------------------------------------------------------- |E2E latency=13.93s |Gather latency=1.13s (8.11%) |Generate time=8.92s (64.03%) |Training time=2.45s (17.57%) |Others=2.56 (18.40%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.30 epoch: 0|step: 714|ppo_ep: 1|act_loss: -0.01702880859375|cri_loss: 0.01019287109375|unsuper_loss: 0.0 average reward score: -1.623046875 ------------------------------------------------------------------------------------- |E2E latency=13.78s |Gather latency=1.03s (7.50%) |Generate time=8.66s (62.86%) |Training time=2.40s (17.42%) |Others=2.72 (19.72%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.30 epoch: 0|step: 715|ppo_ep: 1|act_loss: -0.00604248046875|cri_loss: 0.00974273681640625|unsuper_loss: 0.0 average reward score: -1.4970703125 ------------------------------------------------------------------------------------- |E2E latency=13.84s |Gather latency=0.75s (5.43%) |Generate time=8.84s (63.91%) |Training time=2.37s (17.11%) |Others=2.63 (18.99%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.30 epoch: 0|step: 716|ppo_ep: 1|act_loss: 0.016021728515625|cri_loss: 0.0228729248046875|unsuper_loss: 0.0 average reward score: -2.25 ------------------------------------------------------------------------------------- |E2E latency=13.40s |Gather latency=0.59s (4.38%) |Generate time=8.47s (63.24%) |Training time=2.32s (17.31%) |Others=2.61 (19.44%)|CurSamplesPerSec=2.39 |AvgSamplesPerSec=2.30 epoch: 0|step: 717|ppo_ep: 1|act_loss: -0.0166778564453125|cri_loss: 0.0090179443359375|unsuper_loss: 0.0 average reward score: -1.8740234375 ------------------------------------------------------------------------------------- |E2E latency=14.20s |Gather latency=0.79s (5.56%) |Generate time=9.14s (64.32%) |Training time=2.35s (16.58%) |Others=2.71 (19.10%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.30 epoch: 0|step: 718|ppo_ep: 1|act_loss: 0.024200439453125|cri_loss: 0.00963592529296875|unsuper_loss: 0.0 average reward score: -1.181640625 ------------------------------------------------------------------------------------- |E2E latency=13.57s |Gather latency=0.72s (5.30%) |Generate time=8.71s (64.21%) |Training time=2.27s (16.75%) |Others=2.58 (19.04%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.30 [2023-05-17 02:47:56,210] [INFO] [logging.py:96:log_dist] [Rank 0] step=720, skipped=12, lr=[1.8444973171143855e-07, 1.8444973171143855e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 02:47:56,211] [INFO] [timer.py:199:stop] epoch=0/micro_step=720/global_step=720, RunningAvgSamplesPerSec=28.3241371336967, CurrSamplesPerSec=29.24327825705194, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 02:47:56,988] [INFO] [logging.py:96:log_dist] [Rank 0] step=720, skipped=13, lr=[9.62940786635979e-07, 9.62940786635979e-07], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 719|ppo_ep: 1|act_loss: 0.017669677734375|cri_loss: 0.0089111328125|unsuper_loss: 0.0 average reward score: -2.125 ------------------------------------------------------------------------------------- |E2E latency=12.76s |Gather latency=1.04s (8.17%) |Generate time=7.75s (60.75%) |Training time=2.34s (18.32%) |Others=2.67 (20.93%)|CurSamplesPerSec=2.51 |AvgSamplesPerSec=2.31 epoch: 0|step: 720|ppo_ep: 1|act_loss: -0.04400634765625|cri_loss: 0.01500701904296875|unsuper_loss: 0.0 average reward score: -1.6064453125 ------------------------------------------------------------------------------------- |E2E latency=14.46s |Gather latency=0.70s (4.85%) |Generate time=9.15s (63.27%) |Training time=2.45s (16.93%) |Others=2.86 (19.80%)|CurSamplesPerSec=2.21 |AvgSamplesPerSec=2.30 epoch: 0|step: 721|ppo_ep: 1|act_loss: 0.047515869140625|cri_loss: 0.0076141357421875|unsuper_loss: 0.0 average reward score: -1.7333984375 ------------------------------------------------------------------------------------- |E2E latency=14.19s |Gather latency=0.78s (5.48%) |Generate time=9.05s (63.81%) |Training time=2.41s (16.98%) |Others=2.72 (19.21%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.30 epoch: 0|step: 722|ppo_ep: 1|act_loss: -0.0179443359375|cri_loss: 0.01470184326171875|unsuper_loss: 0.0 average reward score: -2.3359375 ------------------------------------------------------------------------------------- |E2E latency=14.17s |Gather latency=0.65s (4.61%) |Generate time=9.10s (64.22%) |Training time=2.36s (16.68%) |Others=2.71 (19.09%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.30 epoch: 0|step: 723|ppo_ep: 1|act_loss: 0.024871826171875|cri_loss: 0.0172271728515625|unsuper_loss: 0.0 average reward score: -1.7666015625 ------------------------------------------------------------------------------------- |E2E latency=13.59s |Gather latency=0.71s (5.24%) |Generate time=8.66s (63.71%) |Training time=2.34s (17.23%) |Others=2.59 (19.06%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.30 epoch: 0|step: 724|ppo_ep: 1|act_loss: 0.0303497314453125|cri_loss: 0.0181427001953125|unsuper_loss: 0.0 average reward score: -1.4755859375 ------------------------------------------------------------------------------------- |E2E latency=12.47s |Gather latency=0.59s (4.70%) |Generate time=7.60s (61.00%) |Training time=2.30s (18.43%) |Others=2.56 (20.57%)|CurSamplesPerSec=2.57 |AvgSamplesPerSec=2.31 epoch: 0|step: 725|ppo_ep: 1|act_loss: -0.017547607421875|cri_loss: 0.01436614990234375|unsuper_loss: 0.0 average reward score: -1.43359375 ------------------------------------------------------------------------------------- |E2E latency=13.84s |Gather latency=0.80s (5.81%) |Generate time=8.71s (62.93%) |Training time=2.28s (16.46%) |Others=2.85 (20.60%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.31 epoch: 0|step: 726|ppo_ep: 1|act_loss: 0.0189056396484375|cri_loss: 0.01294708251953125|unsuper_loss: 0.0 average reward score: -2.1796875 ------------------------------------------------------------------------------------- |E2E latency=14.23s |Gather latency=0.68s (4.78%) |Generate time=9.10s (63.99%) |Training time=2.38s (16.72%) |Others=2.74 (19.29%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.31 epoch: 0|step: 727|ppo_ep: 1|act_loss: 0.0396728515625|cri_loss: 0.017333984375|unsuper_loss: 0.0 average reward score: -1.83984375 ------------------------------------------------------------------------------------- |E2E latency=14.13s |Gather latency=0.64s (4.51%) |Generate time=9.10s (64.42%) |Training time=2.34s (16.58%) |Others=2.69 (19.00%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.31 epoch: 0|step: 728|ppo_ep: 1|act_loss: 0.01078033447265625|cri_loss: 0.007709503173828125|unsuper_loss: 0.0 average reward score: -1.87109375 ------------------------------------------------------------------------------------- |E2E latency=13.72s |Gather latency=0.62s (4.51%) |Generate time=8.74s (63.68%) |Training time=2.34s (17.07%) |Others=2.64 (19.25%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.31 [2023-05-17 02:50:15,053] [INFO] [logging.py:96:log_dist] [Rank 0] step=730, skipped=12, lr=[1.706962625099682e-07, 1.706962625099682e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 02:50:15,054] [INFO] [timer.py:199:stop] epoch=0/micro_step=730/global_step=730, RunningAvgSamplesPerSec=28.33092273271402, CurrSamplesPerSec=29.057148207603436, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 02:50:15,746] [INFO] [loss_scaler.py:188:update_scale] [deepspeed] OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 65536, but hysteresis is 2. Reducing hysteresis to 1 [2023-05-17 02:50:15,747] [INFO] [logging.py:96:log_dist] [Rank 0] step=730, skipped=14, lr=[8.985170532545623e-07, 8.985170532545623e-07], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 729|ppo_ep: 1|act_loss: 0.01105499267578125|cri_loss: 0.01361083984375|unsuper_loss: 0.0 average reward score: -1.4755859375 ------------------------------------------------------------------------------------- |E2E latency=13.97s |Gather latency=0.83s (5.92%) |Generate time=9.11s (65.26%) |Training time=2.34s (16.76%) |Others=2.51 (17.98%)|CurSamplesPerSec=2.29 |AvgSamplesPerSec=2.31 [2023-05-17 02:50:28,901] [INFO] [loss_scaler.py:181:update_scale] [deepspeed] OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 65536, reducing to 32768 epoch: 0|step: 730|ppo_ep: 1|act_loss: 0.00467681884765625|cri_loss: 0.0127716064453125|unsuper_loss: 0.0 average reward score: -1.5791015625 ------------------------------------------------------------------------------------- |E2E latency=13.16s |Gather latency=0.71s (5.40%) |Generate time=7.91s (60.16%) |Training time=2.58s (19.60%) |Others=2.66 (20.25%)|CurSamplesPerSec=2.43 |AvgSamplesPerSec=2.31 epoch: 0|step: 731|ppo_ep: 1|act_loss: -0.0229644775390625|cri_loss: 0.00725555419921875|unsuper_loss: 0.0 average reward score: -1.88671875 ------------------------------------------------------------------------------------- |E2E latency=14.51s |Gather latency=0.75s (5.16%) |Generate time=9.16s (63.14%) |Training time=2.53s (17.42%) |Others=2.82 (19.43%)|CurSamplesPerSec=2.20 |AvgSamplesPerSec=2.31 epoch: 0|step: 732|ppo_ep: 1|act_loss: 0.01378631591796875|cri_loss: 0.022979736328125|unsuper_loss: 0.0 average reward score: -1.9375 ------------------------------------------------------------------------------------- |E2E latency=14.21s |Gather latency=0.63s (4.43%) |Generate time=9.20s (64.72%) |Training time=2.35s (16.52%) |Others=2.66 (18.75%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.31 epoch: 0|step: 733|ppo_ep: 1|act_loss: 0.013824462890625|cri_loss: 0.00653839111328125|unsuper_loss: 0.0 average reward score: -1.4580078125 ------------------------------------------------------------------------------------- |E2E latency=14.25s |Gather latency=0.81s (5.72%) |Generate time=9.12s (63.99%) |Training time=2.35s (16.47%) |Others=2.78 (19.54%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.31 epoch: 0|step: 734|ppo_ep: 1|act_loss: -0.054931640625|cri_loss: 0.01418304443359375|unsuper_loss: 0.0 average reward score: -1.8251953125 ------------------------------------------------------------------------------------- |E2E latency=13.64s |Gather latency=0.87s (6.37%) |Generate time=8.64s (63.37%) |Training time=2.33s (17.09%) |Others=2.67 (19.54%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.31 epoch: 0|step: 735|ppo_ep: 1|act_loss: 0.0259857177734375|cri_loss: 0.00937652587890625|unsuper_loss: 0.0 average reward score: -1.984375 ------------------------------------------------------------------------------------- |E2E latency=13.67s |Gather latency=0.59s (4.29%) |Generate time=8.66s (63.34%) |Training time=2.37s (17.33%) |Others=2.64 (19.32%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.31 epoch: 0|step: 736|ppo_ep: 1|act_loss: -0.0012521743774414062|cri_loss: 0.01480865478515625|unsuper_loss: 0.0 average reward score: -1.560546875 ------------------------------------------------------------------------------------- |E2E latency=13.65s |Gather latency=0.74s (5.39%) |Generate time=8.66s (63.43%) |Training time=2.37s (17.35%) |Others=2.62 (19.22%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.31 epoch: 0|step: 737|ppo_ep: 1|act_loss: 0.018310546875|cri_loss: 0.01151275634765625|unsuper_loss: 0.0 average reward score: -1.677734375 ------------------------------------------------------------------------------------- |E2E latency=13.36s |Gather latency=0.64s (4.77%) |Generate time=8.50s (63.62%) |Training time=2.34s (17.50%) |Others=2.52 (18.88%)|CurSamplesPerSec=2.39 |AvgSamplesPerSec=2.31 epoch: 0|step: 738|ppo_ep: 1|act_loss: 0.03155517578125|cri_loss: 0.012298583984375|unsuper_loss: 0.0 average reward score: -2.134765625 ------------------------------------------------------------------------------------- |E2E latency=14.32s |Gather latency=0.67s (4.69%) |Generate time=9.15s (63.89%) |Training time=2.39s (16.70%) |Others=2.78 (19.40%)|CurSamplesPerSec=2.24 |AvgSamplesPerSec=2.31 [2023-05-17 02:52:33,634] [INFO] [logging.py:96:log_dist] [Rank 0] step=740, skipped=12, lr=[1.5736469993224377e-07, 1.5736469993224377e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 02:52:33,634] [INFO] [timer.py:199:stop] epoch=0/micro_step=740/global_step=740, RunningAvgSamplesPerSec=28.33594394014235, CurrSamplesPerSec=28.34970893637155, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 02:52:34,392] [INFO] [logging.py:96:log_dist] [Rank 0] step=740, skipped=15, lr=[8.358486211554637e-07, 8.358486211554637e-07], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 739|ppo_ep: 1|act_loss: -0.035858154296875|cri_loss: 0.01013946533203125|unsuper_loss: 0.0 average reward score: -1.662109375 ------------------------------------------------------------------------------------- |E2E latency=13.89s |Gather latency=0.71s (5.08%) |Generate time=8.93s (64.27%) |Training time=2.37s (17.09%) |Others=2.59 (18.65%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.31 epoch: 0|step: 740|ppo_ep: 1|act_loss: -0.005855560302734375|cri_loss: 0.0057373046875|unsuper_loss: 0.0 average reward score: -1.685546875 ------------------------------------------------------------------------------------- |E2E latency=13.69s |Gather latency=0.77s (5.65%) |Generate time=8.65s (63.20%) |Training time=2.28s (16.63%) |Others=2.76 (20.17%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.31 epoch: 0|step: 741|ppo_ep: 1|act_loss: -0.035614013671875|cri_loss: 0.0227203369140625|unsuper_loss: 0.0 average reward score: -1.0751953125 ------------------------------------------------------------------------------------- |E2E latency=13.71s |Gather latency=0.54s (3.90%) |Generate time=8.79s (64.08%) |Training time=2.34s (17.06%) |Others=2.59 (18.86%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.31 epoch: 0|step: 742|ppo_ep: 1|act_loss: 0.01275634765625|cri_loss: 0.01019287109375|unsuper_loss: 0.0 average reward score: -1.6962890625 ------------------------------------------------------------------------------------- |E2E latency=14.07s |Gather latency=1.13s (8.06%) |Generate time=9.11s (64.74%) |Training time=2.40s (17.05%) |Others=2.56 (18.21%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.31 epoch: 0|step: 743|ppo_ep: 1|act_loss: 0.00616455078125|cri_loss: 0.0166778564453125|unsuper_loss: 0.0 average reward score: -1.7294921875 ------------------------------------------------------------------------------------- |E2E latency=14.18s |Gather latency=0.75s (5.30%) |Generate time=8.99s (63.40%) |Training time=2.45s (17.31%) |Others=2.74 (19.29%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.31 epoch: 0|step: 744|ppo_ep: 1|act_loss: 0.0198211669921875|cri_loss: 0.022247314453125|unsuper_loss: 0.0 average reward score: -1.2978515625 ------------------------------------------------------------------------------------- |E2E latency=13.67s |Gather latency=0.90s (6.58%) |Generate time=8.66s (63.37%) |Training time=2.35s (17.19%) |Others=2.66 (19.44%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.31 epoch: 0|step: 745|ppo_ep: 1|act_loss: -0.0291748046875|cri_loss: 0.01438140869140625|unsuper_loss: 0.0 average reward score: -2.583984375 ------------------------------------------------------------------------------------- |E2E latency=13.74s |Gather latency=0.74s (5.38%) |Generate time=8.72s (63.46%) |Training time=2.41s (17.54%) |Others=2.61 (19.00%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.31 epoch: 0|step: 746|ppo_ep: 1|act_loss: -0.028350830078125|cri_loss: 0.01346588134765625|unsuper_loss: 0.0 average reward score: -2.01953125 ------------------------------------------------------------------------------------- |E2E latency=13.35s |Gather latency=1.03s (7.72%) |Generate time=8.22s (61.57%) |Training time=2.37s (17.74%) |Others=2.76 (20.69%)|CurSamplesPerSec=2.40 |AvgSamplesPerSec=2.31 epoch: 0|step: 747|ppo_ep: 1|act_loss: -0.0246734619140625|cri_loss: 0.011566162109375|unsuper_loss: 0.0 average reward score: -1.544921875 ------------------------------------------------------------------------------------- |E2E latency=13.69s |Gather latency=0.49s (3.61%) |Generate time=8.69s (63.46%) |Training time=2.32s (16.97%) |Others=2.68 (19.56%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.31 epoch: 0|step: 748|ppo_ep: 1|act_loss: -0.049713134765625|cri_loss: 0.01190185546875|unsuper_loss: 0.0 average reward score: -1.2841796875 ------------------------------------------------------------------------------------- |E2E latency=14.16s |Gather latency=0.76s (5.38%) |Generate time=9.14s (64.53%) |Training time=2.38s (16.77%) |Others=2.65 (18.70%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.31 [2023-05-17 02:54:51,568] [INFO] [logging.py:96:log_dist] [Rank 0] step=750, skipped=12, lr=[1.4447308312879507e-07, 1.4447308312879507e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 02:54:51,568] [INFO] [timer.py:199:stop] epoch=0/micro_step=750/global_step=750, RunningAvgSamplesPerSec=28.334749605403594, CurrSamplesPerSec=28.538644023612928, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 02:54:52,343] [INFO] [logging.py:96:log_dist] [Rank 0] step=750, skipped=15, lr=[7.683591927095824e-07, 7.683591927095824e-07], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 749|ppo_ep: 1|act_loss: -0.0174102783203125|cri_loss: 0.0134124755859375|unsuper_loss: 0.0 average reward score: -1.0185546875 ------------------------------------------------------------------------------------- |E2E latency=13.69s |Gather latency=0.71s (5.20%) |Generate time=8.61s (62.87%) |Training time=2.35s (17.19%) |Others=2.73 (19.94%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.31 epoch: 0|step: 750|ppo_ep: 1|act_loss: 0.0177001953125|cri_loss: 0.01297760009765625|unsuper_loss: 0.0 average reward score: -1.91015625 ------------------------------------------------------------------------------------- |E2E latency=14.18s |Gather latency=0.83s (5.85%) |Generate time=9.22s (64.99%) |Training time=2.36s (16.64%) |Others=2.61 (18.37%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.31 epoch: 0|step: 751|ppo_ep: 1|act_loss: 0.0217437744140625|cri_loss: 0.02301025390625|unsuper_loss: 0.0 average reward score: -1.2490234375 ------------------------------------------------------------------------------------- |E2E latency=12.82s |Gather latency=0.80s (6.25%) |Generate time=7.60s (59.32%) |Training time=2.40s (18.70%) |Others=2.82 (21.98%)|CurSamplesPerSec=2.50 |AvgSamplesPerSec=2.31 epoch: 0|step: 752|ppo_ep: 1|act_loss: 0.01451873779296875|cri_loss: 0.005886077880859375|unsuper_loss: 0.0 average reward score: -2.49609375 ------------------------------------------------------------------------------------- |E2E latency=13.65s |Gather latency=0.72s (5.29%) |Generate time=8.66s (63.40%) |Training time=2.36s (17.26%) |Others=2.64 (19.35%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.31 epoch: 0|step: 753|ppo_ep: 1|act_loss: 0.071044921875|cri_loss: 0.0170745849609375|unsuper_loss: 0.0 average reward score: -1.224609375 ------------------------------------------------------------------------------------- |E2E latency=13.75s |Gather latency=0.86s (6.25%) |Generate time=8.80s (63.97%) |Training time=2.39s (17.35%) |Others=2.57 (18.68%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.31 epoch: 0|step: 754|ppo_ep: 1|act_loss: -0.00247955322265625|cri_loss: 0.025115966796875|unsuper_loss: 0.0 average reward score: -1.6494140625 ------------------------------------------------------------------------------------- |E2E latency=14.32s |Gather latency=0.94s (6.59%) |Generate time=9.22s (64.37%) |Training time=2.37s (16.53%) |Others=2.74 (19.10%)|CurSamplesPerSec=2.23 |AvgSamplesPerSec=2.31 epoch: 0|step: 755|ppo_ep: 1|act_loss: 0.0299072265625|cri_loss: 0.00701904296875|unsuper_loss: 0.0 average reward score: -0.798828125 ------------------------------------------------------------------------------------- |E2E latency=14.17s |Gather latency=1.08s (7.65%) |Generate time=9.13s (64.46%) |Training time=2.42s (17.09%) |Others=2.61 (18.45%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.31 epoch: 0|step: 756|ppo_ep: 1|act_loss: 0.04852294921875|cri_loss: 0.017303466796875|unsuper_loss: 0.0 average reward score: -1.3134765625 ------------------------------------------------------------------------------------- |E2E latency=14.25s |Gather latency=1.11s (7.80%) |Generate time=9.07s (63.65%) |Training time=2.40s (16.87%) |Others=2.78 (19.48%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.31 epoch: 0|step: 757|ppo_ep: 1|act_loss: 0.01476287841796875|cri_loss: 0.0221099853515625|unsuper_loss: 0.0 average reward score: -1.4677734375 ------------------------------------------------------------------------------------- |E2E latency=13.69s |Gather latency=0.81s (5.92%) |Generate time=8.62s (62.97%) |Training time=2.37s (17.34%) |Others=2.69 (19.68%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.31 epoch: 0|step: 758|ppo_ep: 1|act_loss: 0.038726806640625|cri_loss: 0.0124664306640625|unsuper_loss: 0.0 average reward score: -1.9208984375 ------------------------------------------------------------------------------------- |E2E latency=13.62s |Gather latency=0.58s (4.27%) |Generate time=8.67s (63.67%) |Training time=2.39s (17.52%) |Others=2.56 (18.81%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.31 [2023-05-17 02:57:09,964] [INFO] [logging.py:96:log_dist] [Rank 0] step=760, skipped=12, lr=[1.3203885595248936e-07, 1.3203885595248936e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 02:57:09,964] [INFO] [timer.py:199:stop] epoch=0/micro_step=760/global_step=760, RunningAvgSamplesPerSec=28.33242579643961, CurrSamplesPerSec=27.463579856999537, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 02:57:10,741] [INFO] [logging.py:96:log_dist] [Rank 0] step=760, skipped=15, lr=[7.032128751334607e-07, 7.032128751334607e-07], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 759|ppo_ep: 1|act_loss: 0.0208587646484375|cri_loss: 0.01213836669921875|unsuper_loss: 0.0 average reward score: -0.89404296875 ------------------------------------------------------------------------------------- |E2E latency=13.94s |Gather latency=0.85s (6.07%) |Generate time=8.66s (62.11%) |Training time=2.45s (17.59%) |Others=2.83 (20.30%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.31 epoch: 0|step: 760|ppo_ep: 1|act_loss: -0.03546142578125|cri_loss: 0.0137786865234375|unsuper_loss: 0.0 average reward score: -1.224609375 ------------------------------------------------------------------------------------- |E2E latency=13.59s |Gather latency=0.87s (6.41%) |Generate time=8.67s (63.82%) |Training time=2.39s (17.56%) |Others=2.53 (18.62%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.31 epoch: 0|step: 761|ppo_ep: 1|act_loss: -0.02490234375|cri_loss: 0.009918212890625|unsuper_loss: 0.0 average reward score: -1.3115234375 ------------------------------------------------------------------------------------- |E2E latency=13.69s |Gather latency=0.92s (6.70%) |Generate time=8.66s (63.28%) |Training time=2.38s (17.40%) |Others=2.64 (19.32%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.31 epoch: 0|step: 762|ppo_ep: 1|act_loss: -0.0096588134765625|cri_loss: 0.00848388671875|unsuper_loss: 0.0 average reward score: -1.7734375 ------------------------------------------------------------------------------------- |E2E latency=13.60s |Gather latency=0.97s (7.11%) |Generate time=8.61s (63.32%) |Training time=2.42s (17.76%) |Others=2.57 (18.92%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.31 epoch: 0|step: 763|ppo_ep: 1|act_loss: -0.0105743408203125|cri_loss: 0.01337432861328125|unsuper_loss: 0.0 average reward score: -0.404296875 ------------------------------------------------------------------------------------- |E2E latency=13.99s |Gather latency=0.40s (2.87%) |Generate time=9.00s (64.35%) |Training time=2.33s (16.69%) |Others=2.65 (18.96%)|CurSamplesPerSec=2.29 |AvgSamplesPerSec=2.31 epoch: 0|step: 764|ppo_ep: 1|act_loss: -0.0031528472900390625|cri_loss: 0.01132965087890625|unsuper_loss: 0.0 average reward score: -1.349609375 ------------------------------------------------------------------------------------- |E2E latency=13.77s |Gather latency=0.64s (4.63%) |Generate time=8.66s (62.85%) |Training time=2.35s (17.04%) |Others=2.77 (20.10%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.31 epoch: 0|step: 765|ppo_ep: 1|act_loss: -0.0011644363403320312|cri_loss: 0.00902557373046875|unsuper_loss: 0.0 average reward score: -2.1015625 ------------------------------------------------------------------------------------- |E2E latency=13.64s |Gather latency=0.73s (5.38%) |Generate time=8.60s (63.06%) |Training time=2.40s (17.61%) |Others=2.64 (19.33%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.31 epoch: 0|step: 766|ppo_ep: 1|act_loss: -0.004116058349609375|cri_loss: 0.0102996826171875|unsuper_loss: 0.0 average reward score: -1.9990234375 ------------------------------------------------------------------------------------- |E2E latency=13.64s |Gather latency=0.66s (4.87%) |Generate time=8.67s (63.59%) |Training time=2.38s (17.42%) |Others=2.59 (18.99%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.31 epoch: 0|step: 767|ppo_ep: 1|act_loss: 0.00855255126953125|cri_loss: 0.005062103271484375|unsuper_loss: 0.0 average reward score: -1.9453125 ------------------------------------------------------------------------------------- |E2E latency=13.90s |Gather latency=0.73s (5.22%) |Generate time=8.78s (63.15%) |Training time=2.36s (16.99%) |Others=2.76 (19.86%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.31 epoch: 0|step: 768|ppo_ep: 1|act_loss: 0.004070281982421875|cri_loss: 0.0133209228515625|unsuper_loss: 0.0 average reward score: -2.123046875 ------------------------------------------------------------------------------------- |E2E latency=14.01s |Gather latency=0.66s (4.68%) |Generate time=9.13s (65.15%) |Training time=2.38s (17.00%) |Others=2.50 (17.85%)|CurSamplesPerSec=2.28 |AvgSamplesPerSec=2.31 [2023-05-17 02:59:27,500] [INFO] [logging.py:96:log_dist] [Rank 0] step=770, skipped=12, lr=[1.2007884335497426e-07, 1.2007884335497426e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 02:59:27,501] [INFO] [timer.py:199:stop] epoch=0/micro_step=770/global_step=770, RunningAvgSamplesPerSec=28.33039213700656, CurrSamplesPerSec=27.70895729606207, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 02:59:28,268] [INFO] [logging.py:96:log_dist] [Rank 0] step=770, skipped=15, lr=[6.404978189512682e-07, 6.404978189512682e-07], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 769|ppo_ep: 1|act_loss: 0.014739990234375|cri_loss: 0.006439208984375|unsuper_loss: 0.0 average reward score: -1.73828125 ------------------------------------------------------------------------------------- |E2E latency=13.71s |Gather latency=0.81s (5.93%) |Generate time=8.61s (62.80%) |Training time=2.39s (17.41%) |Others=2.71 (19.78%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.31 epoch: 0|step: 770|ppo_ep: 1|act_loss: -0.0154571533203125|cri_loss: 0.00885009765625|unsuper_loss: 0.0 average reward score: -2.544921875 ------------------------------------------------------------------------------------- |E2E latency=13.93s |Gather latency=0.50s (3.58%) |Generate time=8.96s (64.36%) |Training time=2.43s (17.47%) |Others=2.53 (18.17%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.31 epoch: 0|step: 771|ppo_ep: 1|act_loss: -0.033843994140625|cri_loss: 0.008758544921875|unsuper_loss: 0.0 average reward score: -1.2109375 ------------------------------------------------------------------------------------- |E2E latency=13.78s |Gather latency=0.65s (4.69%) |Generate time=8.68s (62.96%) |Training time=2.42s (17.55%) |Others=2.68 (19.49%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.31 epoch: 0|step: 772|ppo_ep: 1|act_loss: -0.0679931640625|cri_loss: 0.037994384765625|unsuper_loss: 0.0 average reward score: -2.34765625 ------------------------------------------------------------------------------------- |E2E latency=13.94s |Gather latency=0.59s (4.25%) |Generate time=8.71s (62.51%) |Training time=2.34s (16.79%) |Others=2.89 (20.71%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.31 epoch: 0|step: 773|ppo_ep: 1|act_loss: -0.002201080322265625|cri_loss: 0.0084228515625|unsuper_loss: 0.0 average reward score: -1.6376953125 ------------------------------------------------------------------------------------- |E2E latency=13.55s |Gather latency=0.87s (6.45%) |Generate time=8.62s (63.63%) |Training time=2.32s (17.13%) |Others=2.61 (19.24%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.31 epoch: 0|step: 774|ppo_ep: 1|act_loss: -0.01235198974609375|cri_loss: 0.00960540771484375|unsuper_loss: 0.0 average reward score: -1.6328125 ------------------------------------------------------------------------------------- |E2E latency=13.48s |Gather latency=0.77s (5.74%) |Generate time=8.67s (64.29%) |Training time=2.28s (16.89%) |Others=2.54 (18.82%)|CurSamplesPerSec=2.37 |AvgSamplesPerSec=2.31 epoch: 0|step: 775|ppo_ep: 1|act_loss: 0.00811004638671875|cri_loss: 0.00783538818359375|unsuper_loss: 0.0 average reward score: -1.494140625 ------------------------------------------------------------------------------------- |E2E latency=13.69s |Gather latency=0.51s (3.76%) |Generate time=8.69s (63.47%) |Training time=2.22s (16.19%) |Others=2.78 (20.34%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.31 epoch: 0|step: 776|ppo_ep: 1|act_loss: 0.04351806640625|cri_loss: 0.0251312255859375|unsuper_loss: 0.0 average reward score: -0.29296875 ------------------------------------------------------------------------------------- |E2E latency=13.51s |Gather latency=0.61s (4.53%) |Generate time=8.74s (64.65%) |Training time=2.32s (17.14%) |Others=2.46 (18.22%)|CurSamplesPerSec=2.37 |AvgSamplesPerSec=2.31 epoch: 0|step: 777|ppo_ep: 1|act_loss: 0.057769775390625|cri_loss: 0.0213470458984375|unsuper_loss: 0.0 average reward score: -1.91015625 ------------------------------------------------------------------------------------- |E2E latency=13.70s |Gather latency=1.12s (8.18%) |Generate time=8.69s (63.44%) |Training time=2.36s (17.24%) |Others=2.65 (19.32%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.31 epoch: 0|step: 778|ppo_ep: 1|act_loss: -0.0109710693359375|cri_loss: 0.0128936767578125|unsuper_loss: 0.0 average reward score: -1.962890625 ------------------------------------------------------------------------------------- |E2E latency=13.52s |Gather latency=0.84s (6.19%) |Generate time=8.66s (64.05%) |Training time=2.27s (16.76%) |Others=2.59 (19.19%)|CurSamplesPerSec=2.37 |AvgSamplesPerSec=2.31 [2023-05-17 03:01:44,313] [INFO] [logging.py:96:log_dist] [Rank 0] step=780, skipped=12, lr=[1.0860922862056685e-07, 1.0860922862056685e-07], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 03:01:44,314] [INFO] [timer.py:199:stop] epoch=0/micro_step=780/global_step=780, RunningAvgSamplesPerSec=28.335492248347936, CurrSamplesPerSec=29.229191270642023, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 03:01:45,088] [INFO] [logging.py:96:log_dist] [Rank 0] step=780, skipped=15, lr=[5.802988849085001e-07, 5.802988849085001e-07], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 779|ppo_ep: 1|act_loss: 0.0185089111328125|cri_loss: 0.00800323486328125|unsuper_loss: 0.0 average reward score: -1.662109375 ------------------------------------------------------------------------------------- |E2E latency=13.73s |Gather latency=0.98s (7.15%) |Generate time=8.85s (64.47%) |Training time=2.33s (17.01%) |Others=2.54 (18.53%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.31 epoch: 0|step: 780|ppo_ep: 1|act_loss: -0.0230255126953125|cri_loss: 0.013427734375|unsuper_loss: 0.0 average reward score: -2.048828125 ------------------------------------------------------------------------------------- |E2E latency=14.14s |Gather latency=0.88s (6.22%) |Generate time=9.16s (64.79%) |Training time=2.39s (16.92%) |Others=2.59 (18.29%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.31 epoch: 0|step: 781|ppo_ep: 1|act_loss: -0.0130462646484375|cri_loss: 0.0088958740234375|unsuper_loss: 0.0 average reward score: -1.4833984375 ------------------------------------------------------------------------------------- |E2E latency=14.13s |Gather latency=0.85s (6.03%) |Generate time=9.15s (64.74%) |Training time=2.39s (16.89%) |Others=2.60 (18.37%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.31 epoch: 0|step: 782|ppo_ep: 1|act_loss: 0.08349609375|cri_loss: 0.0201568603515625|unsuper_loss: 0.0 average reward score: -0.6318359375 ------------------------------------------------------------------------------------- |E2E latency=14.14s |Gather latency=0.87s (6.13%) |Generate time=9.10s (64.37%) |Training time=2.34s (16.58%) |Others=2.69 (19.05%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.31 epoch: 0|step: 783|ppo_ep: 1|act_loss: -0.01318359375|cri_loss: 0.00913238525390625|unsuper_loss: 0.0 average reward score: -2.013671875 ------------------------------------------------------------------------------------- |E2E latency=12.59s |Gather latency=0.91s (7.20%) |Generate time=7.61s (60.45%) |Training time=2.30s (18.28%) |Others=2.68 (21.27%)|CurSamplesPerSec=2.54 |AvgSamplesPerSec=2.31 epoch: 0|step: 784|ppo_ep: 1|act_loss: -0.01885986328125|cri_loss: 0.01389312744140625|unsuper_loss: 0.0 average reward score: -1.56640625 ------------------------------------------------------------------------------------- |E2E latency=13.94s |Gather latency=0.48s (3.42%) |Generate time=8.88s (63.70%) |Training time=2.33s (16.75%) |Others=2.73 (19.56%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.31 [2023-05-17 03:03:07,246] [INFO] [loss_scaler.py:188:update_scale] [deepspeed] OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 65536, but hysteresis is 2. Reducing hysteresis to 1 epoch: 0|step: 785|ppo_ep: 1|act_loss: 0.0132904052734375|cri_loss: 0.005420684814453125|unsuper_loss: 0.0 average reward score: -1.7177734375 ------------------------------------------------------------------------------------- |E2E latency=13.97s |Gather latency=0.45s (3.19%) |Generate time=8.94s (63.99%) |Training time=2.31s (16.53%) |Others=2.72 (19.47%)|CurSamplesPerSec=2.29 |AvgSamplesPerSec=2.31 [2023-05-17 03:03:20,764] [INFO] [loss_scaler.py:181:update_scale] [deepspeed] OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 65536, reducing to 32768 epoch: 0|step: 786|ppo_ep: 1|act_loss: -0.02545166015625|cri_loss: 0.0221405029296875|unsuper_loss: 0.0 average reward score: -2.33203125 ------------------------------------------------------------------------------------- |E2E latency=13.49s |Gather latency=0.64s (4.76%) |Generate time=8.60s (63.76%) |Training time=2.35s (17.45%) |Others=2.53 (18.79%)|CurSamplesPerSec=2.37 |AvgSamplesPerSec=2.31 epoch: 0|step: 787|ppo_ep: 1|act_loss: -0.0236358642578125|cri_loss: 0.007144927978515625|unsuper_loss: 0.0 average reward score: -1.79296875 ------------------------------------------------------------------------------------- |E2E latency=13.62s |Gather latency=0.86s (6.33%) |Generate time=8.64s (63.42%) |Training time=2.42s (17.74%) |Others=2.57 (18.84%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.31 epoch: 0|step: 788|ppo_ep: 1|act_loss: 0.0031032562255859375|cri_loss: 0.022705078125|unsuper_loss: 0.0 average reward score: -0.947265625 ------------------------------------------------------------------------------------- |E2E latency=13.61s |Gather latency=0.48s (3.50%) |Generate time=8.66s (63.65%) |Training time=2.33s (17.09%) |Others=2.62 (19.26%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.31 [2023-05-17 03:04:01,653] [INFO] [logging.py:96:log_dist] [Rank 0] step=790, skipped=14, lr=[9.979707538246346e-08, 9.979707538246346e-08], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 03:04:01,653] [INFO] [timer.py:199:stop] epoch=0/micro_step=790/global_step=790, RunningAvgSamplesPerSec=28.351243304383974, CurrSamplesPerSec=29.725326863428837, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 03:04:02,409] [INFO] [logging.py:96:log_dist] [Rank 0] step=790, skipped=15, lr=[5.226975291455477e-07, 5.226975291455477e-07], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 789|ppo_ep: 1|act_loss: 0.030975341796875|cri_loss: 0.0234832763671875|unsuper_loss: 0.0 average reward score: -1.26171875 ------------------------------------------------------------------------------------- |E2E latency=13.69s |Gather latency=0.58s (4.25%) |Generate time=8.81s (64.37%) |Training time=2.29s (16.74%) |Others=2.59 (18.89%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.31 epoch: 0|step: 790|ppo_ep: 1|act_loss: 0.048370361328125|cri_loss: 0.009918212890625|unsuper_loss: 0.0 average reward score: -1.8623046875 ------------------------------------------------------------------------------------- |E2E latency=13.57s |Gather latency=0.69s (5.09%) |Generate time=8.65s (63.74%) |Training time=2.32s (17.11%) |Others=2.60 (19.16%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.31 epoch: 0|step: 791|ppo_ep: 1|act_loss: -0.03533935546875|cri_loss: 0.027740478515625|unsuper_loss: 0.0 average reward score: -1.064453125 ------------------------------------------------------------------------------------- |E2E latency=14.26s |Gather latency=0.85s (5.95%) |Generate time=9.12s (63.99%) |Training time=2.38s (16.72%) |Others=2.75 (19.29%)|CurSamplesPerSec=2.24 |AvgSamplesPerSec=2.31 epoch: 0|step: 792|ppo_ep: 1|act_loss: 0.033721923828125|cri_loss: 0.00897216796875|unsuper_loss: 0.0 average reward score: -1.419921875 ------------------------------------------------------------------------------------- |E2E latency=14.07s |Gather latency=0.69s (4.89%) |Generate time=9.16s (65.08%) |Training time=2.32s (16.49%) |Others=2.59 (18.43%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.31 epoch: 0|step: 793|ppo_ep: 1|act_loss: 0.0311126708984375|cri_loss: 0.0288543701171875|unsuper_loss: 0.0 average reward score: -1.5625 ------------------------------------------------------------------------------------- |E2E latency=13.85s |Gather latency=0.40s (2.91%) |Generate time=8.70s (62.80%) |Training time=2.38s (17.22%) |Others=2.77 (19.98%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.31 epoch: 0|step: 794|ppo_ep: 1|act_loss: -0.0118560791015625|cri_loss: 0.00656890869140625|unsuper_loss: 0.0 average reward score: -1.5625 ------------------------------------------------------------------------------------- |E2E latency=15.84s |Gather latency=0.50s (3.13%) |Generate time=9.08s (57.32%) |Training time=4.16s (26.25%) |Others=2.60 (16.43%)|CurSamplesPerSec=2.02 |AvgSamplesPerSec=2.31 epoch: 0|step: 795|ppo_ep: 1|act_loss: -0.0307464599609375|cri_loss: 0.0178375244140625|unsuper_loss: 0.0 average reward score: -1.0859375 ------------------------------------------------------------------------------------- |E2E latency=14.12s |Gather latency=0.71s (5.06%) |Generate time=9.04s (64.03%) |Training time=2.42s (17.15%) |Others=2.66 (18.82%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.31 epoch: 0|step: 796|ppo_ep: 1|act_loss: 0.01488494873046875|cri_loss: 0.006649017333984375|unsuper_loss: 0.0 average reward score: -2.10546875 ------------------------------------------------------------------------------------- |E2E latency=13.70s |Gather latency=1.06s (7.76%) |Generate time=8.63s (63.03%) |Training time=2.44s (17.83%) |Others=2.62 (19.14%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.31 epoch: 0|step: 797|ppo_ep: 1|act_loss: -0.01812744140625|cri_loss: 0.0091094970703125|unsuper_loss: 0.0 average reward score: -1.8720703125 ------------------------------------------------------------------------------------- |E2E latency=14.01s |Gather latency=0.98s (6.96%) |Generate time=8.99s (64.14%) |Training time=2.44s (17.42%) |Others=2.58 (18.44%)|CurSamplesPerSec=2.28 |AvgSamplesPerSec=2.31 epoch: 0|step: 798|ppo_ep: 1|act_loss: 0.02362060546875|cri_loss: 0.007320404052734375|unsuper_loss: 0.0 average reward score: -1.67578125 ------------------------------------------------------------------------------------- |E2E latency=13.70s |Gather latency=0.87s (6.36%) |Generate time=8.57s (62.51%) |Training time=2.30s (16.82%) |Others=2.83 (20.67%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.31 [2023-05-17 03:06:22,391] [INFO] [logging.py:96:log_dist] [Rank 0] step=800, skipped=14, lr=[8.924882714813857e-08, 8.924882714813857e-08], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 03:06:22,392] [INFO] [timer.py:199:stop] epoch=0/micro_step=800/global_step=800, RunningAvgSamplesPerSec=28.328985781656222, CurrSamplesPerSec=28.239538969414014, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 03:06:23,135] [INFO] [logging.py:96:log_dist] [Rank 0] step=800, skipped=15, lr=[4.6777169297808706e-07, 4.6777169297808706e-07], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 799|ppo_ep: 1|act_loss: -0.02020263671875|cri_loss: 0.00843048095703125|unsuper_loss: 0.0 average reward score: -1.5244140625 ------------------------------------------------------------------------------------- |E2E latency=13.61s |Gather latency=0.74s (5.43%) |Generate time=8.62s (63.29%) |Training time=2.35s (17.25%) |Others=2.65 (19.46%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.31 epoch: 0|step: 800|ppo_ep: 1|act_loss: 0.0128173828125|cri_loss: 0.01496124267578125|unsuper_loss: 0.0 average reward score: -1.8935546875 ------------------------------------------------------------------------------------- |E2E latency=13.82s |Gather latency=0.68s (4.95%) |Generate time=8.70s (62.98%) |Training time=2.44s (17.68%) |Others=2.67 (19.34%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.31 epoch: 0|step: 801|ppo_ep: 1|act_loss: 0.108154296875|cri_loss: 0.031524658203125|unsuper_loss: 0.0 average reward score: -1.6015625 ------------------------------------------------------------------------------------- |E2E latency=14.08s |Gather latency=0.75s (5.35%) |Generate time=8.94s (63.51%) |Training time=2.41s (17.11%) |Others=2.73 (19.38%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.31 epoch: 0|step: 802|ppo_ep: 1|act_loss: 0.026885986328125|cri_loss: 0.0127410888671875|unsuper_loss: 0.0 average reward score: -1.6728515625 ------------------------------------------------------------------------------------- |E2E latency=13.79s |Gather latency=0.97s (7.01%) |Generate time=8.74s (63.35%) |Training time=2.37s (17.16%) |Others=2.69 (19.49%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.31 epoch: 0|step: 803|ppo_ep: 1|act_loss: 0.046966552734375|cri_loss: 0.01776123046875|unsuper_loss: 0.0 average reward score: -2.4296875 ------------------------------------------------------------------------------------- |E2E latency=13.91s |Gather latency=0.87s (6.29%) |Generate time=8.75s (62.92%) |Training time=2.40s (17.26%) |Others=2.76 (19.82%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.31 epoch: 0|step: 804|ppo_ep: 1|act_loss: 0.002685546875|cri_loss: 0.0093536376953125|unsuper_loss: 0.0 average reward score: -1.4755859375 ------------------------------------------------------------------------------------- |E2E latency=13.54s |Gather latency=0.95s (6.98%) |Generate time=8.61s (63.54%) |Training time=2.37s (17.47%) |Others=2.57 (18.99%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.31 epoch: 0|step: 805|ppo_ep: 1|act_loss: 0.0296478271484375|cri_loss: 0.00983428955078125|unsuper_loss: 0.0 average reward score: -1.5869140625 ------------------------------------------------------------------------------------- |E2E latency=13.65s |Gather latency=0.77s (5.65%) |Generate time=8.59s (62.95%) |Training time=2.44s (17.87%) |Others=2.62 (19.18%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.31 epoch: 0|step: 806|ppo_ep: 1|act_loss: -0.004535675048828125|cri_loss: 0.0160675048828125|unsuper_loss: 0.0 average reward score: -1.4169921875 ------------------------------------------------------------------------------------- |E2E latency=13.84s |Gather latency=0.80s (5.75%) |Generate time=8.71s (62.89%) |Training time=2.30s (16.58%) |Others=2.84 (20.53%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.31 epoch: 0|step: 807|ppo_ep: 1|act_loss: -0.0257110595703125|cri_loss: 0.01256561279296875|unsuper_loss: 0.0 average reward score: -0.93017578125 ------------------------------------------------------------------------------------- |E2E latency=13.51s |Gather latency=0.62s (4.59%) |Generate time=8.62s (63.77%) |Training time=2.33s (17.23%) |Others=2.57 (19.01%)|CurSamplesPerSec=2.37 |AvgSamplesPerSec=2.31 epoch: 0|step: 808|ppo_ep: 1|act_loss: 0.0008172988891601562|cri_loss: 0.01305389404296875|unsuper_loss: 0.0 average reward score: -1.97265625 ------------------------------------------------------------------------------------- |E2E latency=13.46s |Gather latency=0.64s (4.76%) |Generate time=8.61s (63.96%) |Training time=2.28s (16.93%) |Others=2.57 (19.11%)|CurSamplesPerSec=2.38 |AvgSamplesPerSec=2.31 [2023-05-17 03:08:40,198] [INFO] [logging.py:96:log_dist] [Rank 0] step=810, skipped=14, lr=[7.923269336490251e-08, 7.923269336490251e-08], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 03:08:40,199] [INFO] [timer.py:199:stop] epoch=0/micro_step=810/global_step=810, RunningAvgSamplesPerSec=28.330530830897008, CurrSamplesPerSec=28.5792065398985, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 03:08:40,947] [INFO] [logging.py:96:log_dist] [Rank 0] step=810, skipped=15, lr=[4.1559569743344407e-07, 4.1559569743344407e-07], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 809|ppo_ep: 1|act_loss: 0.035919189453125|cri_loss: 0.01535797119140625|unsuper_loss: 0.0 average reward score: -1.4306640625 ------------------------------------------------------------------------------------- |E2E latency=14.20s |Gather latency=0.79s (5.54%) |Generate time=9.16s (64.49%) |Training time=2.34s (16.50%) |Others=2.70 (19.01%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.31 epoch: 0|step: 810|ppo_ep: 1|act_loss: 0.0286102294921875|cri_loss: 0.00849151611328125|unsuper_loss: 0.0 average reward score: -2.02734375 ------------------------------------------------------------------------------------- |E2E latency=13.70s |Gather latency=0.72s (5.28%) |Generate time=8.72s (63.64%) |Training time=2.36s (17.24%) |Others=2.62 (19.13%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.31 epoch: 0|step: 811|ppo_ep: 1|act_loss: 0.034454345703125|cri_loss: 0.007781982421875|unsuper_loss: 0.0 average reward score: -2.080078125 ------------------------------------------------------------------------------------- |E2E latency=14.25s |Gather latency=1.32s (9.29%) |Generate time=9.18s (64.43%) |Training time=2.34s (16.40%) |Others=2.73 (19.18%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.31 epoch: 0|step: 812|ppo_ep: 1|act_loss: 0.027587890625|cri_loss: 0.01473236083984375|unsuper_loss: 0.0 average reward score: -1.501953125 ------------------------------------------------------------------------------------- |E2E latency=13.70s |Gather latency=1.10s (8.05%) |Generate time=8.63s (62.97%) |Training time=2.28s (16.63%) |Others=2.79 (20.40%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.31 epoch: 0|step: 813|ppo_ep: 1|act_loss: -0.007152557373046875|cri_loss: 0.0136566162109375|unsuper_loss: 0.0 average reward score: -1.685546875 ------------------------------------------------------------------------------------- |E2E latency=13.94s |Gather latency=0.99s (7.08%) |Generate time=9.07s (65.09%) |Training time=2.31s (16.58%) |Others=2.55 (18.33%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.31 epoch: 0|step: 814|ppo_ep: 1|act_loss: -0.0097198486328125|cri_loss: 0.01129150390625|unsuper_loss: 0.0 average reward score: -1.15625 ------------------------------------------------------------------------------------- |E2E latency=14.24s |Gather latency=0.91s (6.39%) |Generate time=9.09s (63.81%) |Training time=2.35s (16.53%) |Others=2.80 (19.65%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.31 epoch: 0|step: 815|ppo_ep: 1|act_loss: 0.00982666015625|cri_loss: 0.0245513916015625|unsuper_loss: 0.0 average reward score: -1.6328125 ------------------------------------------------------------------------------------- |E2E latency=13.79s |Gather latency=1.07s (7.78%) |Generate time=8.81s (63.85%) |Training time=2.32s (16.84%) |Others=2.66 (19.32%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.31 epoch: 0|step: 816|ppo_ep: 1|act_loss: -0.01102447509765625|cri_loss: 0.01043701171875|unsuper_loss: 0.0 average reward score: -0.96728515625 ------------------------------------------------------------------------------------- |E2E latency=13.73s |Gather latency=0.93s (6.76%) |Generate time=8.69s (63.31%) |Training time=2.33s (17.00%) |Others=2.70 (19.69%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.31 epoch: 0|step: 817|ppo_ep: 1|act_loss: -0.0176544189453125|cri_loss: 0.00909423828125|unsuper_loss: 0.0 average reward score: -2.060546875 ------------------------------------------------------------------------------------- |E2E latency=13.49s |Gather latency=1.02s (7.52%) |Generate time=8.66s (64.20%) |Training time=2.28s (16.93%) |Others=2.55 (18.88%)|CurSamplesPerSec=2.37 |AvgSamplesPerSec=2.31 epoch: 0|step: 818|ppo_ep: 1|act_loss: 0.048583984375|cri_loss: 0.0134735107421875|unsuper_loss: 0.0 average reward score: -1.7529296875 ------------------------------------------------------------------------------------- |E2E latency=13.17s |Gather latency=1.19s (9.06%) |Generate time=8.23s (62.47%) |Training time=2.33s (17.71%) |Others=2.61 (19.82%)|CurSamplesPerSec=2.43 |AvgSamplesPerSec=2.31 [2023-05-17 03:10:58,077] [INFO] [logging.py:96:log_dist] [Rank 0] step=820, skipped=14, lr=[6.97622270235615e-08, 6.97622270235615e-08], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 03:10:58,078] [INFO] [timer.py:199:stop] epoch=0/micro_step=820/global_step=820, RunningAvgSamplesPerSec=28.341191151877464, CurrSamplesPerSec=29.042605197377075, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 03:10:58,808] [INFO] [logging.py:96:log_dist] [Rank 0] step=820, skipped=15, lr=[3.662401426856177e-07, 3.662401426856177e-07], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 819|ppo_ep: 1|act_loss: -0.054412841796875|cri_loss: 0.01071929931640625|unsuper_loss: 0.0 average reward score: -1.390625 ------------------------------------------------------------------------------------- |E2E latency=13.85s |Gather latency=0.56s (4.05%) |Generate time=8.70s (62.77%) |Training time=2.33s (16.80%) |Others=2.83 (20.43%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.31 epoch: 0|step: 820|ppo_ep: 1|act_loss: -6.401538848876953e-05|cri_loss: 0.0106353759765625|unsuper_loss: 0.0 average reward score: -1.765625 ------------------------------------------------------------------------------------- |E2E latency=14.07s |Gather latency=0.68s (4.85%) |Generate time=9.11s (64.75%) |Training time=2.34s (16.63%) |Others=2.62 (18.63%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.31 epoch: 0|step: 821|ppo_ep: 1|act_loss: -0.0186614990234375|cri_loss: 0.02197265625|unsuper_loss: 0.0 average reward score: -1.173828125 ------------------------------------------------------------------------------------- |E2E latency=13.70s |Gather latency=0.67s (4.90%) |Generate time=8.67s (63.25%) |Training time=2.35s (17.12%) |Others=2.69 (19.63%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.31 epoch: 0|step: 822|ppo_ep: 1|act_loss: 0.00392913818359375|cri_loss: 0.0099639892578125|unsuper_loss: 0.0 average reward score: -1.314453125 ------------------------------------------------------------------------------------- |E2E latency=13.71s |Gather latency=0.70s (5.13%) |Generate time=8.64s (63.02%) |Training time=2.32s (16.89%) |Others=2.75 (20.09%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.31 epoch: 0|step: 823|ppo_ep: 1|act_loss: 0.0030117034912109375|cri_loss: 0.015960693359375|unsuper_loss: 0.0 average reward score: -2.73046875 ------------------------------------------------------------------------------------- |E2E latency=14.10s |Gather latency=0.68s (4.81%) |Generate time=9.13s (64.78%) |Training time=2.41s (17.09%) |Others=2.56 (18.13%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.31 epoch: 0|step: 824|ppo_ep: 1|act_loss: 0.0287628173828125|cri_loss: 0.013763427734375|unsuper_loss: 0.0 average reward score: -0.5537109375 ------------------------------------------------------------------------------------- |E2E latency=13.70s |Gather latency=0.64s (4.71%) |Generate time=8.65s (63.14%) |Training time=2.43s (17.77%) |Others=2.61 (19.09%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.31 epoch: 0|step: 825|ppo_ep: 1|act_loss: 0.0038242340087890625|cri_loss: 0.011077880859375|unsuper_loss: 0.0 average reward score: -1.345703125 ------------------------------------------------------------------------------------- |E2E latency=13.60s |Gather latency=0.45s (3.34%) |Generate time=8.62s (63.40%) |Training time=2.37s (17.42%) |Others=2.61 (19.18%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.31 epoch: 0|step: 826|ppo_ep: 1|act_loss: 0.0146484375|cri_loss: 0.00498199462890625|unsuper_loss: 0.0 average reward score: -2.05859375 ------------------------------------------------------------------------------------- |E2E latency=14.03s |Gather latency=0.57s (4.08%) |Generate time=9.08s (64.75%) |Training time=2.35s (16.76%) |Others=2.59 (18.49%)|CurSamplesPerSec=2.28 |AvgSamplesPerSec=2.31 epoch: 0|step: 827|ppo_ep: 1|act_loss: -0.0445556640625|cri_loss: 0.0126190185546875|unsuper_loss: 0.0 average reward score: -2.3046875 ------------------------------------------------------------------------------------- |E2E latency=14.14s |Gather latency=0.78s (5.54%) |Generate time=9.14s (64.62%) |Training time=2.32s (16.37%) |Others=2.69 (19.01%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.31 epoch: 0|step: 828|ppo_ep: 1|act_loss: 0.02288818359375|cri_loss: 0.006771087646484375|unsuper_loss: 0.0 average reward score: -1.767578125 ------------------------------------------------------------------------------------- |E2E latency=13.85s |Gather latency=0.67s (4.84%) |Generate time=8.85s (63.90%) |Training time=2.30s (16.61%) |Others=2.70 (19.48%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.31 [2023-05-17 03:13:16,476] [INFO] [logging.py:96:log_dist] [Rank 0] step=830, skipped=14, lr=[6.085024276357906e-08, 6.085024276357906e-08], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 03:13:16,477] [INFO] [timer.py:199:stop] epoch=0/micro_step=830/global_step=830, RunningAvgSamplesPerSec=28.34763880417946, CurrSamplesPerSec=29.571498603250717, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 03:13:17,275] [INFO] [logging.py:96:log_dist] [Rank 0] step=830, skipped=15, lr=[3.1977181252506177e-07, 3.1977181252506177e-07], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 829|ppo_ep: 1|act_loss: 0.05316162109375|cri_loss: 0.019561767578125|unsuper_loss: 0.0 average reward score: -2.396484375 ------------------------------------------------------------------------------------- |E2E latency=13.57s |Gather latency=0.67s (4.95%) |Generate time=8.65s (63.75%) |Training time=2.28s (16.83%) |Others=2.64 (19.42%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.31 epoch: 0|step: 830|ppo_ep: 1|act_loss: 0.02081298828125|cri_loss: 0.007049560546875|unsuper_loss: 0.0 average reward score: -2.060546875 ------------------------------------------------------------------------------------- |E2E latency=13.66s |Gather latency=0.80s (5.87%) |Generate time=8.65s (63.32%) |Training time=2.30s (16.87%) |Others=2.71 (19.82%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.31 [2023-05-17 03:13:44,896] [INFO] [loss_scaler.py:188:update_scale] [deepspeed] OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 65536, but hysteresis is 2. Reducing hysteresis to 1 epoch: 0|step: 831|ppo_ep: 1|act_loss: -0.009857177734375|cri_loss: 0.01129150390625|unsuper_loss: 0.0 average reward score: -1.82421875 ------------------------------------------------------------------------------------- |E2E latency=13.96s |Gather latency=0.70s (5.03%) |Generate time=9.11s (65.28%) |Training time=2.36s (16.91%) |Others=2.49 (17.81%)|CurSamplesPerSec=2.29 |AvgSamplesPerSec=2.31 [2023-05-17 03:13:58,896] [INFO] [loss_scaler.py:181:update_scale] [deepspeed] OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 65536, reducing to 32768 epoch: 0|step: 832|ppo_ep: 1|act_loss: 0.0157928466796875|cri_loss: 0.01161956787109375|unsuper_loss: 0.0 average reward score: -1.4150390625 ------------------------------------------------------------------------------------- |E2E latency=14.00s |Gather latency=0.97s (6.95%) |Generate time=8.79s (62.76%) |Training time=2.51s (17.91%) |Others=2.71 (19.33%)|CurSamplesPerSec=2.29 |AvgSamplesPerSec=2.31 epoch: 0|step: 833|ppo_ep: 1|act_loss: -0.0007448196411132812|cri_loss: 0.01021575927734375|unsuper_loss: 0.0 average reward score: -1.6298828125 ------------------------------------------------------------------------------------- |E2E latency=13.91s |Gather latency=0.76s (5.43%) |Generate time=8.79s (63.15%) |Training time=2.54s (18.23%) |Others=2.59 (18.62%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.31 epoch: 0|step: 834|ppo_ep: 1|act_loss: 0.0009303092956542969|cri_loss: 0.035247802734375|unsuper_loss: 0.0 average reward score: -1.5390625 ------------------------------------------------------------------------------------- |E2E latency=13.78s |Gather latency=1.11s (8.08%) |Generate time=8.70s (63.14%) |Training time=2.40s (17.39%) |Others=2.68 (19.48%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.31 epoch: 0|step: 835|ppo_ep: 1|act_loss: 0.00373077392578125|cri_loss: 0.01229095458984375|unsuper_loss: 0.0 average reward score: -1.625 ------------------------------------------------------------------------------------- |E2E latency=13.88s |Gather latency=1.02s (7.33%) |Generate time=8.73s (62.94%) |Training time=2.38s (17.16%) |Others=2.76 (19.89%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.31 epoch: 0|step: 836|ppo_ep: 1|act_loss: -0.0379638671875|cri_loss: 0.0160369873046875|unsuper_loss: 0.0 average reward score: -1.650390625 ------------------------------------------------------------------------------------- |E2E latency=14.10s |Gather latency=0.97s (6.90%) |Generate time=9.13s (64.78%) |Training time=2.45s (17.41%) |Others=2.51 (17.81%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.31 epoch: 0|step: 837|ppo_ep: 1|act_loss: 0.0062713623046875|cri_loss: 0.0184783935546875|unsuper_loss: 0.0 average reward score: -1.7109375 ------------------------------------------------------------------------------------- |E2E latency=13.90s |Gather latency=0.81s (5.82%) |Generate time=8.71s (62.66%) |Training time=2.47s (17.75%) |Others=2.72 (19.58%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.31 epoch: 0|step: 838|ppo_ep: 1|act_loss: -0.07220458984375|cri_loss: 0.01800537109375|unsuper_loss: 0.0 average reward score: -1.841796875 ------------------------------------------------------------------------------------- |E2E latency=13.88s |Gather latency=0.92s (6.59%) |Generate time=8.60s (61.98%) |Training time=2.39s (17.19%) |Others=2.89 (20.83%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.31 [2023-05-17 03:15:35,812] [INFO] [logging.py:96:log_dist] [Rank 0] step=840, skipped=14, lr=[5.250879953338324e-08, 5.250879953338324e-08], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 03:15:35,813] [INFO] [timer.py:199:stop] epoch=0/micro_step=840/global_step=840, RunningAvgSamplesPerSec=28.34358432912141, CurrSamplesPerSec=29.47320378840947, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 03:15:36,557] [INFO] [logging.py:96:log_dist] [Rank 0] step=840, skipped=17, lr=[2.8471833693759525e-07, 2.8471833693759525e-07], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 839|ppo_ep: 1|act_loss: -0.016326904296875|cri_loss: 0.01430511474609375|unsuper_loss: 0.0 average reward score: -1.0390625 ------------------------------------------------------------------------------------- |E2E latency=14.21s |Gather latency=0.76s (5.37%) |Generate time=9.11s (64.06%) |Training time=2.38s (16.75%) |Others=2.73 (19.18%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.31 epoch: 0|step: 840|ppo_ep: 1|act_loss: 0.045562744140625|cri_loss: 0.0206451416015625|unsuper_loss: 0.0 average reward score: -1.244140625 ------------------------------------------------------------------------------------- |E2E latency=14.30s |Gather latency=0.89s (6.21%) |Generate time=9.06s (63.33%) |Training time=2.33s (16.33%) |Others=2.91 (20.34%)|CurSamplesPerSec=2.24 |AvgSamplesPerSec=2.31 epoch: 0|step: 841|ppo_ep: 1|act_loss: -0.0025081634521484375|cri_loss: 0.0035648345947265625|unsuper_loss: 0.0 average reward score: -1.544921875 ------------------------------------------------------------------------------------- |E2E latency=14.11s |Gather latency=0.42s (3.00%) |Generate time=9.20s (65.20%) |Training time=2.37s (16.82%) |Others=2.54 (17.98%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.31 epoch: 0|step: 842|ppo_ep: 1|act_loss: 0.0433349609375|cri_loss: 0.01409149169921875|unsuper_loss: 0.0 average reward score: -1.765625 ------------------------------------------------------------------------------------- |E2E latency=13.74s |Gather latency=0.56s (4.05%) |Generate time=8.66s (63.00%) |Training time=2.32s (16.91%) |Others=2.76 (20.08%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.31 epoch: 0|step: 843|ppo_ep: 1|act_loss: -0.0251312255859375|cri_loss: 0.017669677734375|unsuper_loss: 0.0 average reward score: -2.34375 ------------------------------------------------------------------------------------- |E2E latency=14.00s |Gather latency=0.86s (6.17%) |Generate time=9.09s (64.92%) |Training time=2.36s (16.83%) |Others=2.55 (18.25%)|CurSamplesPerSec=2.29 |AvgSamplesPerSec=2.31 epoch: 0|step: 844|ppo_ep: 1|act_loss: -0.027496337890625|cri_loss: 0.012725830078125|unsuper_loss: 0.0 average reward score: -1.8408203125 ------------------------------------------------------------------------------------- |E2E latency=13.79s |Gather latency=0.75s (5.42%) |Generate time=8.71s (63.11%) |Training time=2.38s (17.28%) |Others=2.71 (19.61%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.31 epoch: 0|step: 845|ppo_ep: 1|act_loss: 0.080078125|cri_loss: 0.0283966064453125|unsuper_loss: 0.0 average reward score: -2.16015625 ------------------------------------------------------------------------------------- |E2E latency=13.86s |Gather latency=0.76s (5.47%) |Generate time=8.74s (63.05%) |Training time=2.35s (16.95%) |Others=2.77 (20.00%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.31 epoch: 0|step: 846|ppo_ep: 1|act_loss: -0.01284027099609375|cri_loss: 0.008270263671875|unsuper_loss: 0.0 average reward score: -1.1962890625 ------------------------------------------------------------------------------------- |E2E latency=13.42s |Gather latency=0.68s (5.06%) |Generate time=8.62s (64.22%) |Training time=2.31s (17.19%) |Others=2.50 (18.59%)|CurSamplesPerSec=2.38 |AvgSamplesPerSec=2.31 epoch: 0|step: 847|ppo_ep: 1|act_loss: -0.02880859375|cri_loss: 0.0145416259765625|unsuper_loss: 0.0 average reward score: -0.94091796875 ------------------------------------------------------------------------------------- |E2E latency=13.67s |Gather latency=0.77s (5.62%) |Generate time=8.74s (63.92%) |Training time=2.35s (17.19%) |Others=2.58 (18.89%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.31 epoch: 0|step: 848|ppo_ep: 1|act_loss: 0.032958984375|cri_loss: 0.015899658203125|unsuper_loss: 0.0 average reward score: -1.8515625 ------------------------------------------------------------------------------------- |E2E latency=13.13s |Gather latency=0.84s (6.39%) |Generate time=8.11s (61.72%) |Training time=2.33s (17.71%) |Others=2.70 (20.57%)|CurSamplesPerSec=2.44 |AvgSamplesPerSec=2.31 [2023-05-17 03:17:54,087] [INFO] [logging.py:96:log_dist] [Rank 0] step=850, skipped=14, lr=[4.474918427320993e-08, 4.474918427320993e-08], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 03:17:54,087] [INFO] [timer.py:199:stop] epoch=0/micro_step=850/global_step=850, RunningAvgSamplesPerSec=28.34701244011392, CurrSamplesPerSec=29.62770880029809, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 03:17:54,843] [INFO] [logging.py:96:log_dist] [Rank 0] step=850, skipped=17, lr=[2.436027813884251e-07, 2.436027813884251e-07], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 849|ppo_ep: 1|act_loss: -0.03369140625|cri_loss: 0.01068878173828125|unsuper_loss: 0.0 average reward score: -1.240234375 ------------------------------------------------------------------------------------- |E2E latency=14.27s |Gather latency=0.91s (6.41%) |Generate time=9.14s (64.06%) |Training time=2.37s (16.58%) |Others=2.76 (19.36%)|CurSamplesPerSec=2.24 |AvgSamplesPerSec=2.31 epoch: 0|step: 850|ppo_ep: 1|act_loss: 0.027557373046875|cri_loss: 0.01224517822265625|unsuper_loss: 0.0 average reward score: -1.5205078125 ------------------------------------------------------------------------------------- |E2E latency=14.03s |Gather latency=0.88s (6.28%) |Generate time=8.84s (63.06%) |Training time=2.38s (16.98%) |Others=2.80 (19.97%)|CurSamplesPerSec=2.28 |AvgSamplesPerSec=2.31 epoch: 0|step: 851|ppo_ep: 1|act_loss: -0.045654296875|cri_loss: 0.0198974609375|unsuper_loss: 0.0 average reward score: -1.728515625 ------------------------------------------------------------------------------------- |E2E latency=13.85s |Gather latency=0.87s (6.30%) |Generate time=8.69s (62.71%) |Training time=2.43s (17.53%) |Others=2.74 (19.76%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.31 epoch: 0|step: 852|ppo_ep: 1|act_loss: 0.0014801025390625|cri_loss: 0.006710052490234375|unsuper_loss: 0.0 average reward score: -2.017578125 ------------------------------------------------------------------------------------- |E2E latency=13.47s |Gather latency=0.70s (5.19%) |Generate time=8.42s (62.49%) |Training time=2.44s (18.09%) |Others=2.62 (19.42%)|CurSamplesPerSec=2.38 |AvgSamplesPerSec=2.31 epoch: 0|step: 853|ppo_ep: 1|act_loss: -0.034423828125|cri_loss: 0.01352691650390625|unsuper_loss: 0.0 average reward score: -0.865234375 ------------------------------------------------------------------------------------- |E2E latency=13.88s |Gather latency=0.77s (5.54%) |Generate time=8.74s (62.95%) |Training time=2.37s (17.09%) |Others=2.77 (19.96%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.31 epoch: 0|step: 854|ppo_ep: 1|act_loss: 0.0384521484375|cri_loss: 0.0159454345703125|unsuper_loss: 0.0 average reward score: -2.447265625 ------------------------------------------------------------------------------------- |E2E latency=14.00s |Gather latency=0.72s (5.17%) |Generate time=9.05s (64.60%) |Training time=2.40s (17.13%) |Others=2.56 (18.27%)|CurSamplesPerSec=2.29 |AvgSamplesPerSec=2.31 epoch: 0|step: 855|ppo_ep: 1|act_loss: 0.02325439453125|cri_loss: 0.02008056640625|unsuper_loss: 0.0 average reward score: -1.2890625 ------------------------------------------------------------------------------------- |E2E latency=14.14s |Gather latency=0.72s (5.08%) |Generate time=9.12s (64.52%) |Training time=2.34s (16.58%) |Others=2.67 (18.90%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.31 epoch: 0|step: 856|ppo_ep: 1|act_loss: 0.0080413818359375|cri_loss: 0.00873565673828125|unsuper_loss: 0.0 average reward score: -1.791015625 ------------------------------------------------------------------------------------- |E2E latency=13.58s |Gather latency=0.70s (5.15%) |Generate time=8.66s (63.76%) |Training time=2.38s (17.53%) |Others=2.54 (18.72%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.31 epoch: 0|step: 857|ppo_ep: 1|act_loss: 0.0294036865234375|cri_loss: 0.01160430908203125|unsuper_loss: 0.0 average reward score: -1.271484375 ------------------------------------------------------------------------------------- |E2E latency=13.71s |Gather latency=1.15s (8.39%) |Generate time=8.65s (63.09%) |Training time=2.39s (17.42%) |Others=2.67 (19.49%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.31 epoch: 0|step: 858|ppo_ep: 1|act_loss: -0.03045654296875|cri_loss: 0.01297760009765625|unsuper_loss: 0.0 average reward score: -1.03125 ------------------------------------------------------------------------------------- |E2E latency=13.96s |Gather latency=1.09s (7.79%) |Generate time=8.94s (64.06%) |Training time=2.45s (17.58%) |Others=2.56 (18.35%)|CurSamplesPerSec=2.29 |AvgSamplesPerSec=2.31 [2023-05-17 03:20:12,436] [INFO] [logging.py:96:log_dist] [Rank 0] step=860, skipped=14, lr=[3.758189664256393e-08, 3.758189664256393e-08], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 03:20:12,437] [INFO] [timer.py:199:stop] epoch=0/micro_step=860/global_step=860, RunningAvgSamplesPerSec=28.341204295440537, CurrSamplesPerSec=27.482418387281786, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 03:20:13,167] [INFO] [logging.py:96:log_dist] [Rank 0] step=860, skipped=17, lr=[2.0554039300138384e-07, 2.0554039300138384e-07], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 859|ppo_ep: 1|act_loss: 0.00785064697265625|cri_loss: 0.00849151611328125|unsuper_loss: 0.0 average reward score: -1.09375 ------------------------------------------------------------------------------------- |E2E latency=13.71s |Gather latency=0.96s (6.99%) |Generate time=8.68s (63.33%) |Training time=2.41s (17.56%) |Others=2.62 (19.11%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.31 epoch: 0|step: 860|ppo_ep: 1|act_loss: -0.0195770263671875|cri_loss: 0.0088348388671875|unsuper_loss: 0.0 average reward score: -1.830078125 ------------------------------------------------------------------------------------- |E2E latency=13.98s |Gather latency=0.73s (5.21%) |Generate time=9.03s (64.63%) |Training time=2.35s (16.82%) |Others=2.59 (18.55%)|CurSamplesPerSec=2.29 |AvgSamplesPerSec=2.31 epoch: 0|step: 861|ppo_ep: 1|act_loss: 0.0167999267578125|cri_loss: 0.008514404296875|unsuper_loss: 0.0 average reward score: -2.349609375 ------------------------------------------------------------------------------------- |E2E latency=13.87s |Gather latency=1.28s (9.22%) |Generate time=8.76s (63.11%) |Training time=2.44s (17.61%) |Others=2.68 (19.28%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.31 epoch: 0|step: 862|ppo_ep: 1|act_loss: 0.11248779296875|cri_loss: 0.0269927978515625|unsuper_loss: 0.0 average reward score: -1.4365234375 ------------------------------------------------------------------------------------- |E2E latency=12.48s |Gather latency=1.26s (10.13%) |Generate time=7.59s (60.79%) |Training time=2.33s (18.65%) |Others=2.57 (20.56%)|CurSamplesPerSec=2.56 |AvgSamplesPerSec=2.31 epoch: 0|step: 863|ppo_ep: 1|act_loss: -0.0161895751953125|cri_loss: 0.01226043701171875|unsuper_loss: 0.0 average reward score: -1.755859375 ------------------------------------------------------------------------------------- |E2E latency=13.07s |Gather latency=0.69s (5.29%) |Generate time=7.99s (61.13%) |Training time=2.40s (18.38%) |Others=2.68 (20.49%)|CurSamplesPerSec=2.45 |AvgSamplesPerSec=2.31 epoch: 0|step: 864|ppo_ep: 1|act_loss: -0.036651611328125|cri_loss: 0.011871337890625|unsuper_loss: 0.0 average reward score: -1.2138671875 ------------------------------------------------------------------------------------- |E2E latency=13.88s |Gather latency=0.54s (3.86%) |Generate time=8.63s (62.16%) |Training time=2.36s (16.98%) |Others=2.90 (20.86%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.31 epoch: 0|step: 865|ppo_ep: 1|act_loss: -0.048431396484375|cri_loss: 0.0166168212890625|unsuper_loss: 0.0 average reward score: -1.5791015625 ------------------------------------------------------------------------------------- |E2E latency=13.60s |Gather latency=0.49s (3.59%) |Generate time=8.70s (63.94%) |Training time=2.28s (16.73%) |Others=2.63 (19.33%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.31 epoch: 0|step: 866|ppo_ep: 1|act_loss: 0.002796173095703125|cri_loss: 0.01548004150390625|unsuper_loss: 0.0 average reward score: -2.357421875 ------------------------------------------------------------------------------------- |E2E latency=14.18s |Gather latency=0.94s (6.60%) |Generate time=9.14s (64.49%) |Training time=2.38s (16.76%) |Others=2.66 (18.75%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.31 epoch: 0|step: 867|ppo_ep: 1|act_loss: 0.007083892822265625|cri_loss: 0.016357421875|unsuper_loss: 0.0 average reward score: -1.87109375 ------------------------------------------------------------------------------------- |E2E latency=14.14s |Gather latency=0.79s (5.58%) |Generate time=9.21s (65.15%) |Training time=2.37s (16.76%) |Others=2.56 (18.10%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.31 epoch: 0|step: 868|ppo_ep: 1|act_loss: -0.007015228271484375|cri_loss: 0.006687164306640625|unsuper_loss: 0.0 average reward score: -1.0419921875 ------------------------------------------------------------------------------------- |E2E latency=13.75s |Gather latency=0.79s (5.71%) |Generate time=8.63s (62.78%) |Training time=2.38s (17.33%) |Others=2.74 (19.90%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.31 [2023-05-17 03:22:29,386] [INFO] [logging.py:96:log_dist] [Rank 0] step=870, skipped=14, lr=[3.1016634812961876e-08, 3.1016634812961876e-08], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 03:22:29,386] [INFO] [timer.py:199:stop] epoch=0/micro_step=870/global_step=870, RunningAvgSamplesPerSec=28.338034106867056, CurrSamplesPerSec=27.249353877688083, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 03:22:30,191] [INFO] [logging.py:96:log_dist] [Rank 0] step=870, skipped=17, lr=[1.705826746029149e-07, 1.705826746029149e-07], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 869|ppo_ep: 1|act_loss: 0.0121002197265625|cri_loss: 0.006107330322265625|unsuper_loss: 0.0 average reward score: -1.9423828125 ------------------------------------------------------------------------------------- |E2E latency=14.07s |Gather latency=0.83s (5.93%) |Generate time=8.65s (61.49%) |Training time=2.50s (17.79%) |Others=2.92 (20.71%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.31 epoch: 0|step: 870|ppo_ep: 1|act_loss: -0.04864501953125|cri_loss: 0.018829345703125|unsuper_loss: 0.0 average reward score: -1.529296875 ------------------------------------------------------------------------------------- |E2E latency=14.15s |Gather latency=0.77s (5.44%) |Generate time=9.15s (64.65%) |Training time=2.42s (17.11%) |Others=2.58 (18.23%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.31 epoch: 0|step: 871|ppo_ep: 1|act_loss: 0.006866455078125|cri_loss: 0.015411376953125|unsuper_loss: 0.0 average reward score: -2.05078125 ------------------------------------------------------------------------------------- |E2E latency=13.91s |Gather latency=0.81s (5.79%) |Generate time=8.75s (62.91%) |Training time=2.41s (17.31%) |Others=2.75 (19.78%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.31 epoch: 0|step: 872|ppo_ep: 1|act_loss: 0.0252838134765625|cri_loss: 0.0069427490234375|unsuper_loss: 0.0 average reward score: -1.6708984375 ------------------------------------------------------------------------------------- |E2E latency=13.70s |Gather latency=0.72s (5.27%) |Generate time=8.64s (63.10%) |Training time=2.43s (17.78%) |Others=2.62 (19.13%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.31 epoch: 0|step: 873|ppo_ep: 1|act_loss: 0.0255889892578125|cri_loss: 0.00514984130859375|unsuper_loss: 0.0 average reward score: -1.560546875 ------------------------------------------------------------------------------------- |E2E latency=13.63s |Gather latency=0.71s (5.17%) |Generate time=8.68s (63.69%) |Training time=2.33s (17.10%) |Others=2.62 (19.20%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.31 epoch: 0|step: 874|ppo_ep: 1|act_loss: 0.03173828125|cri_loss: 0.0195159912109375|unsuper_loss: 0.0 average reward score: -1.1259765625 ------------------------------------------------------------------------------------- |E2E latency=14.11s |Gather latency=0.71s (5.03%) |Generate time=9.13s (64.68%) |Training time=2.33s (16.55%) |Others=2.65 (18.77%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.31 epoch: 0|step: 875|ppo_ep: 1|act_loss: 0.0030536651611328125|cri_loss: 0.01282501220703125|unsuper_loss: 0.0 average reward score: -1.4248046875 ------------------------------------------------------------------------------------- |E2E latency=13.82s |Gather latency=0.80s (5.82%) |Generate time=8.80s (63.67%) |Training time=2.36s (17.09%) |Others=2.66 (19.24%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.31 epoch: 0|step: 876|ppo_ep: 1|act_loss: 0.0004870891571044922|cri_loss: 0.00972747802734375|unsuper_loss: 0.0 average reward score: -2.44140625 ------------------------------------------------------------------------------------- |E2E latency=14.16s |Gather latency=0.99s (7.00%) |Generate time=9.17s (64.72%) |Training time=2.36s (16.69%) |Others=2.63 (18.59%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.31 epoch: 0|step: 877|ppo_ep: 1|act_loss: 0.046417236328125|cri_loss: 0.0121002197265625|unsuper_loss: 0.0 average reward score: -1.291015625 ------------------------------------------------------------------------------------- |E2E latency=14.38s |Gather latency=1.11s (7.71%) |Generate time=9.15s (63.62%) |Training time=2.44s (16.96%) |Others=2.79 (19.43%)|CurSamplesPerSec=2.23 |AvgSamplesPerSec=2.31 epoch: 0|step: 878|ppo_ep: 1|act_loss: 0.0187835693359375|cri_loss: 0.00734710693359375|unsuper_loss: 0.0 average reward score: -1.66796875 ------------------------------------------------------------------------------------- |E2E latency=13.90s |Gather latency=0.76s (5.45%) |Generate time=8.89s (63.98%) |Training time=2.37s (17.03%) |Others=2.64 (18.99%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.31 [2023-05-17 03:24:49,020] [INFO] [logging.py:96:log_dist] [Rank 0] step=880, skipped=14, lr=[2.5062282345180766e-08, 2.5062282345180766e-08], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 03:24:49,021] [INFO] [timer.py:199:stop] epoch=0/micro_step=880/global_step=880, RunningAvgSamplesPerSec=28.33655910544184, CurrSamplesPerSec=28.35713609627476, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 03:24:49,791] [INFO] [logging.py:96:log_dist] [Rank 0] step=880, skipped=17, lr=[1.3877692804084687e-07, 1.3877692804084687e-07], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 879|ppo_ep: 1|act_loss: 0.0572509765625|cri_loss: 0.0088653564453125|unsuper_loss: 0.0 average reward score: -1.3642578125 ------------------------------------------------------------------------------------- |E2E latency=13.85s |Gather latency=0.88s (6.36%) |Generate time=8.64s (62.38%) |Training time=2.39s (17.28%) |Others=2.82 (20.34%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.31 epoch: 0|step: 880|ppo_ep: 1|act_loss: -0.0189208984375|cri_loss: 0.01482391357421875|unsuper_loss: 0.0 average reward score: -1.9599609375 ------------------------------------------------------------------------------------- |E2E latency=13.61s |Gather latency=0.96s (7.05%) |Generate time=8.73s (64.11%) |Training time=2.38s (17.51%) |Others=2.50 (18.38%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.31 epoch: 0|step: 881|ppo_ep: 1|act_loss: 0.0214385986328125|cri_loss: 0.007843017578125|unsuper_loss: 0.0 average reward score: -0.92431640625 ------------------------------------------------------------------------------------- |E2E latency=13.66s |Gather latency=0.81s (5.96%) |Generate time=8.60s (62.98%) |Training time=2.38s (17.44%) |Others=2.67 (19.58%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.31 epoch: 0|step: 882|ppo_ep: 1|act_loss: -0.0255279541015625|cri_loss: 0.01201629638671875|unsuper_loss: 0.0 average reward score: -1.09765625 ------------------------------------------------------------------------------------- |E2E latency=13.90s |Gather latency=1.08s (7.79%) |Generate time=8.62s (62.04%) |Training time=2.43s (17.45%) |Others=2.85 (20.51%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.31 epoch: 0|step: 883|ppo_ep: 1|act_loss: -0.045440673828125|cri_loss: 0.0124359130859375|unsuper_loss: 0.0 average reward score: -0.9560546875 ------------------------------------------------------------------------------------- |E2E latency=13.54s |Gather latency=1.00s (7.41%) |Generate time=8.63s (63.76%) |Training time=2.29s (16.94%) |Others=2.61 (19.30%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.31 epoch: 0|step: 884|ppo_ep: 1|act_loss: 0.033721923828125|cri_loss: 0.01009368896484375|unsuper_loss: 0.0 average reward score: -1.4140625 ------------------------------------------------------------------------------------- |E2E latency=13.78s |Gather latency=0.73s (5.30%) |Generate time=8.71s (63.19%) |Training time=2.41s (17.49%) |Others=2.66 (19.32%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.31 epoch: 0|step: 885|ppo_ep: 1|act_loss: 0.004947662353515625|cri_loss: 0.0081024169921875|unsuper_loss: 0.0 average reward score: -1.2216796875 ------------------------------------------------------------------------------------- |E2E latency=13.59s |Gather latency=0.97s (7.17%) |Generate time=8.64s (63.62%) |Training time=2.37s (17.45%) |Others=2.57 (18.94%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.31 epoch: 0|step: 886|ppo_ep: 1|act_loss: -0.0095367431640625|cri_loss: 0.01119232177734375|unsuper_loss: 0.0 average reward score: -1.9296875 ------------------------------------------------------------------------------------- |E2E latency=13.75s |Gather latency=1.22s (8.89%) |Generate time=8.87s (64.53%) |Training time=2.30s (16.71%) |Others=2.58 (18.76%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.31 [2023-05-17 03:26:38,957] [INFO] [loss_scaler.py:188:update_scale] [deepspeed] OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 65536, but hysteresis is 2. Reducing hysteresis to 1 epoch: 0|step: 887|ppo_ep: 1|act_loss: -0.0198516845703125|cri_loss: 0.015350341796875|unsuper_loss: 0.0 average reward score: -1.107421875 ------------------------------------------------------------------------------------- |E2E latency=14.07s |Gather latency=0.79s (5.59%) |Generate time=9.10s (64.69%) |Training time=2.31s (16.40%) |Others=2.66 (18.92%)|CurSamplesPerSec=2.28 |AvgSamplesPerSec=2.31 [2023-05-17 03:26:52,537] [INFO] [loss_scaler.py:181:update_scale] [deepspeed] OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 65536, reducing to 32768 epoch: 0|step: 888|ppo_ep: 1|act_loss: 0.01552581787109375|cri_loss: 0.0145416259765625|unsuper_loss: 0.0 average reward score: -2.0390625 ------------------------------------------------------------------------------------- |E2E latency=13.60s |Gather latency=0.81s (5.98%) |Generate time=8.67s (63.79%) |Training time=2.33s (17.16%) |Others=2.59 (19.05%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.31 [2023-05-17 03:27:06,278] [INFO] [logging.py:96:log_dist] [Rank 0] step=890, skipped=16, lr=[2.0744097427091748e-08, 2.0744097427091748e-08], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 03:27:06,279] [INFO] [timer.py:199:stop] epoch=0/micro_step=890/global_step=890, RunningAvgSamplesPerSec=28.34510403111303, CurrSamplesPerSec=28.191724616253556, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 03:27:07,032] [INFO] [logging.py:96:log_dist] [Rank 0] step=890, skipped=17, lr=[1.1016619017950952e-07, 1.1016619017950952e-07], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 889|ppo_ep: 1|act_loss: 0.039276123046875|cri_loss: 0.0109100341796875|unsuper_loss: 0.0 average reward score: -1.4423828125 ------------------------------------------------------------------------------------- |E2E latency=13.75s |Gather latency=0.71s (5.16%) |Generate time=8.72s (63.46%) |Training time=2.46s (17.91%) |Others=2.56 (18.64%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.31 epoch: 0|step: 890|ppo_ep: 1|act_loss: 0.03717041015625|cri_loss: 0.034393310546875|unsuper_loss: 0.0 average reward score: -1.40234375 ------------------------------------------------------------------------------------- |E2E latency=13.69s |Gather latency=1.20s (8.77%) |Generate time=8.71s (63.59%) |Training time=2.27s (16.57%) |Others=2.72 (19.85%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.31 epoch: 0|step: 891|ppo_ep: 1|act_loss: 0.0229339599609375|cri_loss: 0.0092620849609375|unsuper_loss: 0.0 average reward score: -2.19140625 ------------------------------------------------------------------------------------- |E2E latency=14.06s |Gather latency=1.16s (8.22%) |Generate time=9.15s (65.06%) |Training time=2.42s (17.20%) |Others=2.49 (17.73%)|CurSamplesPerSec=2.28 |AvgSamplesPerSec=2.31 epoch: 0|step: 892|ppo_ep: 1|act_loss: -0.036773681640625|cri_loss: 0.01483154296875|unsuper_loss: 0.0 average reward score: -1.9775390625 ------------------------------------------------------------------------------------- |E2E latency=13.79s |Gather latency=0.90s (6.51%) |Generate time=8.67s (62.86%) |Training time=2.36s (17.13%) |Others=2.76 (20.01%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.31 epoch: 0|step: 893|ppo_ep: 1|act_loss: 0.03326416015625|cri_loss: 0.015472412109375|unsuper_loss: 0.0 average reward score: -0.4775390625 ------------------------------------------------------------------------------------- |E2E latency=13.82s |Gather latency=0.97s (7.01%) |Generate time=8.85s (64.05%) |Training time=2.36s (17.09%) |Others=2.61 (18.87%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.31 epoch: 0|step: 894|ppo_ep: 1|act_loss: -0.040863037109375|cri_loss: 0.009033203125|unsuper_loss: 0.0 average reward score: -0.356689453125 ------------------------------------------------------------------------------------- |E2E latency=13.74s |Gather latency=0.51s (3.69%) |Generate time=8.64s (62.87%) |Training time=2.43s (17.68%) |Others=2.67 (19.45%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.31 epoch: 0|step: 895|ppo_ep: 1|act_loss: -0.04754638671875|cri_loss: 0.01168060302734375|unsuper_loss: 0.0 average reward score: -1.111328125 ------------------------------------------------------------------------------------- |E2E latency=13.78s |Gather latency=0.74s (5.37%) |Generate time=8.62s (62.55%) |Training time=2.36s (17.15%) |Others=2.80 (20.30%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.31 epoch: 0|step: 896|ppo_ep: 1|act_loss: 0.0479736328125|cri_loss: 0.00943756103515625|unsuper_loss: 0.0 average reward score: -1.109375 ------------------------------------------------------------------------------------- |E2E latency=13.47s |Gather latency=0.69s (5.11%) |Generate time=8.63s (64.06%) |Training time=2.31s (17.11%) |Others=2.54 (18.82%)|CurSamplesPerSec=2.37 |AvgSamplesPerSec=2.31 epoch: 0|step: 897|ppo_ep: 1|act_loss: -0.048553466796875|cri_loss: 0.01416778564453125|unsuper_loss: 0.0 average reward score: -1.357421875 ------------------------------------------------------------------------------------- |E2E latency=13.60s |Gather latency=0.42s (3.08%) |Generate time=8.68s (63.78%) |Training time=2.40s (17.65%) |Others=2.53 (18.56%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.31 epoch: 0|step: 898|ppo_ep: 1|act_loss: 0.0097198486328125|cri_loss: 0.0101776123046875|unsuper_loss: 0.0 average reward score: -1.603515625 ------------------------------------------------------------------------------------- |E2E latency=13.89s |Gather latency=0.80s (5.76%) |Generate time=8.63s (62.17%) |Training time=2.45s (17.66%) |Others=2.80 (20.17%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.31 [2023-05-17 03:29:23,864] [INFO] [logging.py:96:log_dist] [Rank 0] step=900, skipped=16, lr=[1.590912278818792e-08, 1.590912278818792e-08], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 03:29:23,864] [INFO] [timer.py:199:stop] epoch=0/micro_step=900/global_step=900, RunningAvgSamplesPerSec=28.34173562864329, CurrSamplesPerSec=27.215637800001623, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 03:29:24,599] [INFO] [logging.py:96:log_dist] [Rank 0] step=900, skipped=17, lr=[8.478917466586217e-08, 8.478917466586217e-08], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 899|ppo_ep: 1|act_loss: 0.03521728515625|cri_loss: 0.00933837890625|unsuper_loss: 0.0 average reward score: -1.7236328125 ------------------------------------------------------------------------------------- |E2E latency=13.71s |Gather latency=0.66s (4.84%) |Generate time=8.70s (63.49%) |Training time=2.40s (17.50%) |Others=2.61 (19.01%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.31 epoch: 0|step: 900|ppo_ep: 1|act_loss: -0.019073486328125|cri_loss: 0.018218994140625|unsuper_loss: 0.0 average reward score: -1.0986328125 ------------------------------------------------------------------------------------- |E2E latency=14.21s |Gather latency=0.92s (6.47%) |Generate time=9.09s (63.96%) |Training time=2.42s (17.06%) |Others=2.70 (18.97%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.31 epoch: 0|step: 901|ppo_ep: 1|act_loss: 0.02960205078125|cri_loss: 0.0105438232421875|unsuper_loss: 0.0 average reward score: -2.388671875 ------------------------------------------------------------------------------------- |E2E latency=14.16s |Gather latency=0.93s (6.59%) |Generate time=9.14s (64.57%) |Training time=2.36s (16.66%) |Others=2.66 (18.78%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.31 epoch: 0|step: 902|ppo_ep: 1|act_loss: -0.0218353271484375|cri_loss: 0.01242828369140625|unsuper_loss: 0.0 average reward score: -1.7841796875 ------------------------------------------------------------------------------------- |E2E latency=13.89s |Gather latency=0.84s (6.07%) |Generate time=8.73s (62.84%) |Training time=2.45s (17.65%) |Others=2.71 (19.51%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.31 epoch: 0|step: 903|ppo_ep: 1|act_loss: 0.00757598876953125|cri_loss: 0.005523681640625|unsuper_loss: 0.0 average reward score: -2.4765625 ------------------------------------------------------------------------------------- |E2E latency=13.76s |Gather latency=0.92s (6.70%) |Generate time=8.65s (62.88%) |Training time=2.31s (16.81%) |Others=2.79 (20.31%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.31 epoch: 0|step: 904|ppo_ep: 1|act_loss: -0.0208282470703125|cri_loss: 0.01739501953125|unsuper_loss: 0.0 average reward score: -1.4033203125 ------------------------------------------------------------------------------------- |E2E latency=13.54s |Gather latency=0.69s (5.07%) |Generate time=8.69s (64.21%) |Training time=2.30s (16.98%) |Others=2.55 (18.81%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.31 epoch: 0|step: 905|ppo_ep: 1|act_loss: 0.0057525634765625|cri_loss: 0.0168304443359375|unsuper_loss: 0.0 average reward score: -2.109375 ------------------------------------------------------------------------------------- |E2E latency=14.09s |Gather latency=1.21s (8.62%) |Generate time=9.11s (64.68%) |Training time=2.34s (16.64%) |Others=2.63 (18.68%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.31 epoch: 0|step: 906|ppo_ep: 1|act_loss: 0.03509521484375|cri_loss: 0.0108489990234375|unsuper_loss: 0.0 average reward score: -1.861328125 ------------------------------------------------------------------------------------- |E2E latency=14.09s |Gather latency=1.05s (7.46%) |Generate time=9.17s (65.07%) |Training time=2.38s (16.91%) |Others=2.54 (18.02%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.31 epoch: 0|step: 907|ppo_ep: 1|act_loss: -0.01715087890625|cri_loss: 0.016326904296875|unsuper_loss: 0.0 average reward score: -2.298828125 ------------------------------------------------------------------------------------- |E2E latency=14.29s |Gather latency=0.92s (6.46%) |Generate time=9.10s (63.68%) |Training time=2.45s (17.17%) |Others=2.74 (19.15%)|CurSamplesPerSec=2.24 |AvgSamplesPerSec=2.31 epoch: 0|step: 908|ppo_ep: 1|act_loss: 0.047119140625|cri_loss: 0.00989532470703125|unsuper_loss: 0.0 average reward score: -1.34765625 ------------------------------------------------------------------------------------- |E2E latency=13.72s |Gather latency=0.99s (7.18%) |Generate time=8.63s (62.89%) |Training time=2.29s (16.68%) |Others=2.80 (20.43%)|CurSamplesPerSec=2.33 |AvgSamplesPerSec=2.31 [2023-05-17 03:31:43,127] [INFO] [logging.py:96:log_dist] [Rank 0] step=910, skipped=16, lr=[1.170549972723399e-08, 1.170549972723399e-08], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 03:31:43,128] [INFO] [timer.py:199:stop] epoch=0/micro_step=910/global_step=910, RunningAvgSamplesPerSec=28.34424545438174, CurrSamplesPerSec=28.856289076886263, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 03:31:43,945] [INFO] [logging.py:96:log_dist] [Rank 0] step=910, skipped=17, lr=[6.268021954544095e-08, 6.268021954544095e-08], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 909|ppo_ep: 1|act_loss: 0.0220947265625|cri_loss: 0.00872802734375|unsuper_loss: 0.0 average reward score: -1.3837890625 ------------------------------------------------------------------------------------- |E2E latency=13.62s |Gather latency=0.99s (7.29%) |Generate time=8.68s (63.78%) |Training time=2.30s (16.91%) |Others=2.63 (19.31%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.31 epoch: 0|step: 910|ppo_ep: 1|act_loss: 0.01088714599609375|cri_loss: 0.0085906982421875|unsuper_loss: 0.0 average reward score: -2.00390625 ------------------------------------------------------------------------------------- |E2E latency=14.06s |Gather latency=0.81s (5.78%) |Generate time=9.17s (65.21%) |Training time=2.37s (16.83%) |Others=2.52 (17.95%)|CurSamplesPerSec=2.28 |AvgSamplesPerSec=2.31 epoch: 0|step: 911|ppo_ep: 1|act_loss: 0.0189971923828125|cri_loss: 0.01078033447265625|unsuper_loss: 0.0 average reward score: -1.6416015625 ------------------------------------------------------------------------------------- |E2E latency=14.07s |Gather latency=0.88s (6.23%) |Generate time=9.13s (64.91%) |Training time=2.40s (17.07%) |Others=2.53 (18.02%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.31 epoch: 0|step: 912|ppo_ep: 1|act_loss: -0.07598876953125|cri_loss: 0.0225067138671875|unsuper_loss: 0.0 average reward score: -1.05859375 ------------------------------------------------------------------------------------- |E2E latency=13.66s |Gather latency=0.50s (3.68%) |Generate time=8.65s (63.32%) |Training time=2.33s (17.06%) |Others=2.68 (19.62%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.31 epoch: 0|step: 913|ppo_ep: 1|act_loss: -0.0197906494140625|cri_loss: 0.01155853271484375|unsuper_loss: 0.0 average reward score: -1.0634765625 ------------------------------------------------------------------------------------- |E2E latency=14.27s |Gather latency=0.77s (5.41%) |Generate time=9.12s (63.91%) |Training time=2.38s (16.69%) |Others=2.77 (19.40%)|CurSamplesPerSec=2.24 |AvgSamplesPerSec=2.31 epoch: 0|step: 914|ppo_ep: 1|act_loss: 0.0096893310546875|cri_loss: 0.005191802978515625|unsuper_loss: 0.0 average reward score: -1.923828125 ------------------------------------------------------------------------------------- |E2E latency=14.17s |Gather latency=0.71s (4.98%) |Generate time=9.14s (64.48%) |Training time=2.35s (16.59%) |Others=2.68 (18.93%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.31 epoch: 0|step: 915|ppo_ep: 1|act_loss: 0.0154266357421875|cri_loss: 0.015625|unsuper_loss: 0.0 average reward score: -1.4375 ------------------------------------------------------------------------------------- |E2E latency=13.81s |Gather latency=1.12s (8.14%) |Generate time=8.72s (63.13%) |Training time=2.32s (16.77%) |Others=2.78 (20.10%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.31 epoch: 0|step: 916|ppo_ep: 1|act_loss: 0.0069732666015625|cri_loss: 0.01488494873046875|unsuper_loss: 0.0 average reward score: -1.87890625 ------------------------------------------------------------------------------------- |E2E latency=14.06s |Gather latency=0.89s (6.36%) |Generate time=9.09s (64.65%) |Training time=2.37s (16.85%) |Others=2.60 (18.50%)|CurSamplesPerSec=2.28 |AvgSamplesPerSec=2.31 epoch: 0|step: 917|ppo_ep: 1|act_loss: -0.04608154296875|cri_loss: 0.01708984375|unsuper_loss: 0.0 average reward score: -0.8701171875 ------------------------------------------------------------------------------------- |E2E latency=13.61s |Gather latency=0.78s (5.76%) |Generate time=8.65s (63.59%) |Training time=2.34s (17.18%) |Others=2.62 (19.23%)|CurSamplesPerSec=2.35 |AvgSamplesPerSec=2.31 epoch: 0|step: 918|ppo_ep: 1|act_loss: 0.0030765533447265625|cri_loss: 0.011749267578125|unsuper_loss: 0.0 average reward score: -1.5986328125 ------------------------------------------------------------------------------------- |E2E latency=14.23s |Gather latency=0.94s (6.63%) |Generate time=9.22s (64.79%) |Training time=2.31s (16.20%) |Others=2.71 (19.01%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.31 [2023-05-17 03:34:03,582] [INFO] [logging.py:96:log_dist] [Rank 0] step=920, skipped=16, lr=[8.13891623382061e-09, 8.13891623382061e-09], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 03:34:03,582] [INFO] [timer.py:199:stop] epoch=0/micro_step=920/global_step=920, RunningAvgSamplesPerSec=28.344656559364658, CurrSamplesPerSec=26.433014452537638, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 03:34:04,331] [INFO] [logging.py:96:log_dist] [Rank 0] step=920, skipped=17, lr=[4.3869240798998236e-08, 4.3869240798998236e-08], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 919|ppo_ep: 1|act_loss: -0.043243408203125|cri_loss: 0.01220703125|unsuper_loss: 0.0 average reward score: 0.0916748046875 ------------------------------------------------------------------------------------- |E2E latency=14.45s |Gather latency=0.94s (6.52%) |Generate time=9.24s (63.96%) |Training time=2.54s (17.60%) |Others=2.66 (18.44%)|CurSamplesPerSec=2.21 |AvgSamplesPerSec=2.31 epoch: 0|step: 920|ppo_ep: 1|act_loss: 0.024505615234375|cri_loss: 0.01044464111328125|unsuper_loss: 0.0 average reward score: -2.20703125 ------------------------------------------------------------------------------------- |E2E latency=14.09s |Gather latency=0.77s (5.43%) |Generate time=9.10s (64.60%) |Training time=2.35s (16.69%) |Others=2.63 (18.70%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.31 epoch: 0|step: 921|ppo_ep: 1|act_loss: 0.033721923828125|cri_loss: 0.0254974365234375|unsuper_loss: 0.0 average reward score: -1.9638671875 ------------------------------------------------------------------------------------- |E2E latency=13.52s |Gather latency=0.97s (7.19%) |Generate time=8.62s (63.73%) |Training time=2.30s (17.03%) |Others=2.60 (19.24%)|CurSamplesPerSec=2.37 |AvgSamplesPerSec=2.31 epoch: 0|step: 922|ppo_ep: 1|act_loss: -0.03826904296875|cri_loss: 0.011993408203125|unsuper_loss: 0.0 average reward score: -1.8076171875 ------------------------------------------------------------------------------------- |E2E latency=12.72s |Gather latency=1.13s (8.87%) |Generate time=7.77s (61.11%) |Training time=2.35s (18.43%) |Others=2.60 (20.46%)|CurSamplesPerSec=2.52 |AvgSamplesPerSec=2.31 epoch: 0|step: 923|ppo_ep: 1|act_loss: -0.0164031982421875|cri_loss: 0.00814056396484375|unsuper_loss: 0.0 average reward score: -1.23046875 ------------------------------------------------------------------------------------- |E2E latency=14.07s |Gather latency=0.52s (3.73%) |Generate time=8.90s (63.23%) |Training time=2.37s (16.81%) |Others=2.81 (19.96%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.31 epoch: 0|step: 924|ppo_ep: 1|act_loss: -0.0036640167236328125|cri_loss: 0.0066986083984375|unsuper_loss: 0.0 average reward score: -2.3046875 ------------------------------------------------------------------------------------- |E2E latency=15.55s |Gather latency=0.59s (3.80%) |Generate time=9.03s (58.06%) |Training time=3.86s (24.85%) |Others=2.66 (17.09%)|CurSamplesPerSec=2.06 |AvgSamplesPerSec=2.31 epoch: 0|step: 925|ppo_ep: 1|act_loss: 0.02392578125|cri_loss: 0.006450653076171875|unsuper_loss: 0.0 average reward score: -0.916015625 ------------------------------------------------------------------------------------- |E2E latency=14.45s |Gather latency=0.94s (6.51%) |Generate time=9.45s (65.42%) |Training time=2.39s (16.53%) |Others=2.61 (18.05%)|CurSamplesPerSec=2.21 |AvgSamplesPerSec=2.31 epoch: 0|step: 926|ppo_ep: 1|act_loss: -0.00482940673828125|cri_loss: 0.006954193115234375|unsuper_loss: 0.0 average reward score: -1.791015625 ------------------------------------------------------------------------------------- |E2E latency=13.52s |Gather latency=0.83s (6.11%) |Generate time=8.50s (62.86%) |Training time=2.33s (17.26%) |Others=2.69 (19.89%)|CurSamplesPerSec=2.37 |AvgSamplesPerSec=2.31 epoch: 0|step: 927|ppo_ep: 1|act_loss: 0.0283203125|cri_loss: 0.0233154296875|unsuper_loss: 0.0 average reward score: -1.8583984375 ------------------------------------------------------------------------------------- |E2E latency=13.53s |Gather latency=0.93s (6.86%) |Generate time=8.64s (63.91%) |Training time=2.35s (17.39%) |Others=2.53 (18.70%)|CurSamplesPerSec=2.37 |AvgSamplesPerSec=2.31 epoch: 0|step: 928|ppo_ep: 1|act_loss: 0.015106201171875|cri_loss: 0.0098419189453125|unsuper_loss: 0.0 average reward score: -1.7177734375 ------------------------------------------------------------------------------------- |E2E latency=13.88s |Gather latency=1.03s (7.45%) |Generate time=8.69s (62.62%) |Training time=2.35s (16.93%) |Others=2.84 (20.45%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.31 [2023-05-17 03:36:22,365] [INFO] [logging.py:96:log_dist] [Rank 0] step=930, skipped=16, lr=[5.214198309111556e-09, 5.214198309111556e-09], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 03:36:22,366] [INFO] [timer.py:199:stop] epoch=0/micro_step=930/global_step=930, RunningAvgSamplesPerSec=28.332142114278987, CurrSamplesPerSec=29.977595265684343, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 03:36:23,141] [INFO] [logging.py:96:log_dist] [Rank 0] step=930, skipped=17, lr=[2.8381691862712503e-08, 2.8381691862712503e-08], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 929|ppo_ep: 1|act_loss: 0.0246734619140625|cri_loss: 0.00975799560546875|unsuper_loss: 0.0 average reward score: -1.7314453125 ------------------------------------------------------------------------------------- |E2E latency=13.49s |Gather latency=0.86s (6.36%) |Generate time=8.65s (64.14%) |Training time=2.28s (16.89%) |Others=2.56 (18.97%)|CurSamplesPerSec=2.37 |AvgSamplesPerSec=2.31 epoch: 0|step: 930|ppo_ep: 1|act_loss: -0.0098724365234375|cri_loss: 0.01068115234375|unsuper_loss: 0.0 average reward score: -0.8564453125 ------------------------------------------------------------------------------------- |E2E latency=13.52s |Gather latency=0.76s (5.59%) |Generate time=8.69s (64.26%) |Training time=2.32s (17.18%) |Others=2.51 (18.56%)|CurSamplesPerSec=2.37 |AvgSamplesPerSec=2.31 epoch: 0|step: 931|ppo_ep: 1|act_loss: 0.003376007080078125|cri_loss: 0.0199127197265625|unsuper_loss: 0.0 average reward score: -1.587890625 ------------------------------------------------------------------------------------- |E2E latency=13.22s |Gather latency=0.76s (5.71%) |Generate time=8.12s (61.44%) |Training time=2.37s (17.94%) |Others=2.73 (20.62%)|CurSamplesPerSec=2.42 |AvgSamplesPerSec=2.31 epoch: 0|step: 932|ppo_ep: 1|act_loss: 0.00041031837463378906|cri_loss: 0.0031948089599609375|unsuper_loss: 0.0 average reward score: -1.5537109375 ------------------------------------------------------------------------------------- |E2E latency=13.87s |Gather latency=0.64s (4.65%) |Generate time=8.85s (63.83%) |Training time=2.32s (16.73%) |Others=2.70 (19.45%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.31 [2023-05-17 03:37:17,416] [INFO] [loss_scaler.py:188:update_scale] [deepspeed] OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 65536, but hysteresis is 2. Reducing hysteresis to 1 epoch: 0|step: 933|ppo_ep: 1|act_loss: -0.0255889892578125|cri_loss: 0.0113372802734375|unsuper_loss: 0.0 average reward score: -1.611328125 ------------------------------------------------------------------------------------- |E2E latency=13.66s |Gather latency=0.79s (5.79%) |Generate time=8.64s (63.27%) |Training time=2.38s (17.43%) |Others=2.64 (19.30%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.31 [2023-05-17 03:37:31,081] [INFO] [loss_scaler.py:181:update_scale] [deepspeed] OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 65536, reducing to 32768 epoch: 0|step: 934|ppo_ep: 1|act_loss: -0.0030231475830078125|cri_loss: 0.01226806640625|unsuper_loss: 0.0 average reward score: -1.013671875 ------------------------------------------------------------------------------------- |E2E latency=13.66s |Gather latency=0.93s (6.79%) |Generate time=8.63s (63.19%) |Training time=2.45s (17.96%) |Others=2.58 (18.85%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.31 epoch: 0|step: 935|ppo_ep: 1|act_loss: -0.03753662109375|cri_loss: 0.0121002197265625|unsuper_loss: 0.0 average reward score: -1.451171875 ------------------------------------------------------------------------------------- |E2E latency=13.77s |Gather latency=0.77s (5.61%) |Generate time=8.60s (62.46%) |Training time=2.52s (18.31%) |Others=2.65 (19.23%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.31 epoch: 0|step: 936|ppo_ep: 1|act_loss: -0.020294189453125|cri_loss: 0.0105438232421875|unsuper_loss: 0.0 average reward score: -0.91943359375 ------------------------------------------------------------------------------------- |E2E latency=13.87s |Gather latency=1.10s (7.95%) |Generate time=8.76s (63.14%) |Training time=2.43s (17.54%) |Others=2.68 (19.33%)|CurSamplesPerSec=2.31 |AvgSamplesPerSec=2.31 epoch: 0|step: 937|ppo_ep: 1|act_loss: 0.0124969482421875|cri_loss: 0.007556915283203125|unsuper_loss: 0.0 average reward score: -1.5849609375 ------------------------------------------------------------------------------------- |E2E latency=13.56s |Gather latency=1.14s (8.42%) |Generate time=8.63s (63.66%) |Training time=2.35s (17.36%) |Others=2.57 (18.98%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.31 epoch: 0|step: 938|ppo_ep: 1|act_loss: 0.024383544921875|cri_loss: 0.00984954833984375|unsuper_loss: 0.0 average reward score: -1.759765625 ------------------------------------------------------------------------------------- |E2E latency=14.12s |Gather latency=1.11s (7.86%) |Generate time=9.11s (64.53%) |Training time=2.41s (17.04%) |Others=2.60 (18.43%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.31 [2023-05-17 03:38:40,018] [INFO] [logging.py:96:log_dist] [Rank 0] step=940, skipped=16, lr=[2.9353034357045694e-09, 2.9353034357045694e-09], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 03:38:40,019] [INFO] [timer.py:199:stop] epoch=0/micro_step=940/global_step=940, RunningAvgSamplesPerSec=28.330219906572946, CurrSamplesPerSec=27.82160868914516, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 03:38:40,772] [INFO] [logging.py:96:log_dist] [Rank 0] step=940, skipped=19, lr=[1.8398756765560243e-08, 1.8398756765560243e-08], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 939|ppo_ep: 1|act_loss: 0.020263671875|cri_loss: 0.00838470458984375|unsuper_loss: 0.0 average reward score: -1.85546875 ------------------------------------------------------------------------------------- |E2E latency=14.38s |Gather latency=0.96s (6.66%) |Generate time=9.14s (63.56%) |Training time=2.43s (16.91%) |Others=2.81 (19.54%)|CurSamplesPerSec=2.23 |AvgSamplesPerSec=2.31 epoch: 0|step: 940|ppo_ep: 1|act_loss: 0.029266357421875|cri_loss: 0.007587432861328125|unsuper_loss: 0.0 average reward score: -1.2626953125 ------------------------------------------------------------------------------------- |E2E latency=13.91s |Gather latency=1.12s (8.07%) |Generate time=8.87s (63.81%) |Training time=2.43s (17.47%) |Others=2.60 (18.72%)|CurSamplesPerSec=2.30 |AvgSamplesPerSec=2.31 epoch: 0|step: 941|ppo_ep: 1|act_loss: 0.00974273681640625|cri_loss: 0.01485443115234375|unsuper_loss: 0.0 average reward score: -1.283203125 ------------------------------------------------------------------------------------- |E2E latency=14.47s |Gather latency=1.00s (6.91%) |Generate time=9.25s (63.93%) |Training time=2.38s (16.46%) |Others=2.84 (19.61%)|CurSamplesPerSec=2.21 |AvgSamplesPerSec=2.31 epoch: 0|step: 942|ppo_ep: 1|act_loss: -0.006732940673828125|cri_loss: 0.00782012939453125|unsuper_loss: 0.0 average reward score: -1.435546875 ------------------------------------------------------------------------------------- |E2E latency=13.66s |Gather latency=1.07s (7.85%) |Generate time=8.65s (63.28%) |Training time=2.34s (17.10%) |Others=2.68 (19.62%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.31 epoch: 0|step: 943|ppo_ep: 1|act_loss: -0.018646240234375|cri_loss: 0.01128387451171875|unsuper_loss: 0.0 average reward score: -2.087890625 ------------------------------------------------------------------------------------- |E2E latency=14.12s |Gather latency=0.40s (2.84%) |Generate time=9.17s (64.95%) |Training time=2.34s (16.60%) |Others=2.60 (18.45%)|CurSamplesPerSec=2.27 |AvgSamplesPerSec=2.31 epoch: 0|step: 944|ppo_ep: 1|act_loss: -0.0224151611328125|cri_loss: 0.010772705078125|unsuper_loss: 0.0 average reward score: -2.0390625 ------------------------------------------------------------------------------------- |E2E latency=14.21s |Gather latency=0.55s (3.88%) |Generate time=9.13s (64.26%) |Training time=2.39s (16.80%) |Others=2.69 (18.93%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.31 epoch: 0|step: 945|ppo_ep: 1|act_loss: 0.0094146728515625|cri_loss: 0.0113067626953125|unsuper_loss: 0.0 average reward score: -2.513671875 ------------------------------------------------------------------------------------- |E2E latency=12.65s |Gather latency=1.04s (8.19%) |Generate time=7.63s (60.34%) |Training time=2.37s (18.72%) |Others=2.65 (20.94%)|CurSamplesPerSec=2.53 |AvgSamplesPerSec=2.31 epoch: 0|step: 946|ppo_ep: 1|act_loss: -0.0003783702850341797|cri_loss: 0.006916046142578125|unsuper_loss: 0.0 average reward score: -1.806640625 ------------------------------------------------------------------------------------- |E2E latency=13.77s |Gather latency=0.81s (5.86%) |Generate time=8.65s (62.87%) |Training time=2.38s (17.26%) |Others=2.74 (19.87%)|CurSamplesPerSec=2.32 |AvgSamplesPerSec=2.31 epoch: 0|step: 947|ppo_ep: 1|act_loss: -0.0295562744140625|cri_loss: 0.00982666015625|unsuper_loss: 0.0 average reward score: -1.2177734375 ------------------------------------------------------------------------------------- |E2E latency=14.14s |Gather latency=0.76s (5.41%) |Generate time=9.03s (63.84%) |Training time=2.44s (17.24%) |Others=2.68 (18.92%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.31 epoch: 0|step: 948|ppo_ep: 1|act_loss: -0.0150604248046875|cri_loss: 0.01324462890625|unsuper_loss: 0.0 average reward score: -1.1494140625 ------------------------------------------------------------------------------------- |E2E latency=13.67s |Gather latency=0.68s (4.98%) |Generate time=8.67s (63.41%) |Training time=2.36s (17.25%) |Others=2.64 (19.33%)|CurSamplesPerSec=2.34 |AvgSamplesPerSec=2.31 [2023-05-17 03:40:58,540] [INFO] [logging.py:96:log_dist] [Rank 0] step=950, skipped=16, lr=[1.3053152226982065e-09, 1.3053152226982065e-09], mom=[(0.9, 0.95), (0.9, 0.95)] [2023-05-17 03:40:58,541] [INFO] [timer.py:199:stop] epoch=0/micro_step=950/global_step=950, RunningAvgSamplesPerSec=28.33070724582498, CurrSamplesPerSec=28.234501369255554, MemAllocated=17.53GB, MaxMemAllocated=29.22GB [2023-05-17 03:40:59,327] [INFO] [logging.py:96:log_dist] [Rank 0] step=950, skipped=19, lr=[8.943151508336301e-09, 8.943151508336301e-09], mom=[(0.9, 0.95), (0.9, 0.95)] epoch: 0|step: 949|ppo_ep: 1|act_loss: 0.029205322265625|cri_loss: 0.0078887939453125|unsuper_loss: 0.0 average reward score: -1.9638671875 ------------------------------------------------------------------------------------- |E2E latency=13.96s |Gather latency=1.10s (7.88%) |Generate time=8.77s (62.82%) |Training time=2.43s (17.41%) |Others=2.76 (19.77%)|CurSamplesPerSec=2.29 |AvgSamplesPerSec=2.31 epoch: 0|step: 950|ppo_ep: 1|act_loss: 0.0082244873046875|cri_loss: 0.0160064697265625|unsuper_loss: 0.0 average reward score: -1.552734375 ------------------------------------------------------------------------------------- |E2E latency=14.24s |Gather latency=0.87s (6.12%) |Generate time=9.21s (64.67%) |Training time=2.39s (16.79%) |Others=2.64 (18.54%)|CurSamplesPerSec=2.25 |AvgSamplesPerSec=2.31 epoch: 0|step: 951|ppo_ep: 1|act_loss: 0.0005750656127929688|cri_loss: 0.01099395751953125|unsuper_loss: 0.0 average reward score: -1.904296875 ------------------------------------------------------------------------------------- |E2E latency=14.17s |Gather latency=0.72s (5.12%) |Generate time=9.14s (64.52%) |Training time=2.39s (16.88%) |Others=2.63 (18.60%)|CurSamplesPerSec=2.26 |AvgSamplesPerSec=2.31 epoch: 0|step: 952|ppo_ep: 1|act_loss: -0.0635986328125|cri_loss: 0.016326904296875|unsuper_loss: 0.0 average reward score: -1.1689453125 ------------------------------------------------------------------------------------- |E2E latency=13.57s |Gather latency=1.17s (8.64%) |Generate time=8.61s (63.42%) |Training time=2.30s (16.91%) |Others=2.67 (19.67%)|CurSamplesPerSec=2.36 |AvgSamplesPerSec=2.31 epoch: 0|step: 953|ppo_ep: 1|act_loss: 0.03826904296875|cri_loss: 0.01067352294921875|unsuper_loss: 0.0 average reward score: -2.228515625 ------------------------------------------------------------------------------------- saving model ... saving model ... saving model ... saving model ...saving model ... saving model ... saving model ... saving model ... [2023-05-17 03:42:24,597] [INFO] [launch.py:460:main] Process 77431 exits successfully. [2023-05-17 03:42:24,600] [INFO] [launch.py:460:main] Process 77437 exits successfully. [2023-05-17 03:42:24,601] [INFO] [launch.py:460:main] Process 77436 exits successfully. [2023-05-17 03:42:25,601] [INFO] [launch.py:460:main] Process 77434 exits successfully. [2023-05-17 03:42:25,602] [INFO] [launch.py:460:main] Process 77432 exits successfully. [2023-05-17 03:42:25,603] [INFO] [launch.py:460:main] Process 77433 exits successfully. [2023-05-17 03:42:26,604] [INFO] [launch.py:460:main] Process 77430 exits successfully. [2023-05-17 03:42:26,604] [INFO] [launch.py:460:main] Process 77435 exits successfully.