| [2023-12-26 21:54:56,521][15743] Saving configuration to /home/cybertron/Desktop/rl_units/train_dir/default_experiment/config.json... | |
| [2023-12-26 21:54:56,521][15743] Rollout worker 0 uses device cpu | |
| [2023-12-26 21:54:56,522][15743] Rollout worker 1 uses device cpu | |
| [2023-12-26 21:54:56,522][15743] Rollout worker 2 uses device cpu | |
| [2023-12-26 21:54:56,522][15743] Rollout worker 3 uses device cpu | |
| [2023-12-26 21:54:56,522][15743] Rollout worker 4 uses device cpu | |
| [2023-12-26 21:54:56,522][15743] Rollout worker 5 uses device cpu | |
| [2023-12-26 21:54:56,522][15743] Rollout worker 6 uses device cpu | |
| [2023-12-26 21:54:56,522][15743] Rollout worker 7 uses device cpu | |
| [2023-12-26 21:54:56,569][15743] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-12-26 21:54:56,569][15743] InferenceWorker_p0-w0: min num requests: 2 | |
| [2023-12-26 21:54:56,589][15743] Starting all processes... | |
| [2023-12-26 21:54:56,589][15743] Starting process learner_proc0 | |
| [2023-12-26 21:54:57,788][15743] Starting all processes... | |
| [2023-12-26 21:54:57,791][15743] Starting process inference_proc0-0 | |
| [2023-12-26 21:54:57,791][15743] Starting process rollout_proc0 | |
| [2023-12-26 21:54:57,791][15743] Starting process rollout_proc1 | |
| [2023-12-26 21:54:57,793][15787] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-12-26 21:54:57,793][15787] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 | |
| [2023-12-26 21:54:57,791][15743] Starting process rollout_proc2 | |
| [2023-12-26 21:54:57,791][15743] Starting process rollout_proc3 | |
| [2023-12-26 21:54:57,791][15743] Starting process rollout_proc4 | |
| [2023-12-26 21:54:57,806][15787] Num visible devices: 1 | |
| [2023-12-26 21:54:57,791][15743] Starting process rollout_proc5 | |
| [2023-12-26 21:54:57,792][15743] Starting process rollout_proc6 | |
| [2023-12-26 21:54:57,793][15743] Starting process rollout_proc7 | |
| [2023-12-26 21:54:57,844][15787] Starting seed is not provided | |
| [2023-12-26 21:54:57,844][15787] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-12-26 21:54:57,844][15787] Initializing actor-critic model on device cuda:0 | |
| [2023-12-26 21:54:57,844][15787] RunningMeanStd input shape: (3, 72, 128) | |
| [2023-12-26 21:54:57,845][15787] RunningMeanStd input shape: (1,) | |
| [2023-12-26 21:54:57,855][15787] ConvEncoder: input_channels=3 | |
| [2023-12-26 21:54:57,970][15787] Conv encoder output size: 512 | |
| [2023-12-26 21:54:57,970][15787] Policy head output size: 512 | |
| [2023-12-26 21:54:57,980][15787] Created Actor Critic model with architecture: | |
| [2023-12-26 21:54:57,980][15787] ActorCriticSharedWeights( | |
| (obs_normalizer): ObservationNormalizer( | |
| (running_mean_std): RunningMeanStdDictInPlace( | |
| (running_mean_std): ModuleDict( | |
| (obs): RunningMeanStdInPlace() | |
| ) | |
| ) | |
| ) | |
| (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) | |
| (encoder): VizdoomEncoder( | |
| (basic_encoder): ConvEncoder( | |
| (enc): RecursiveScriptModule( | |
| original_name=ConvEncoderImpl | |
| (conv_head): RecursiveScriptModule( | |
| original_name=Sequential | |
| (0): RecursiveScriptModule(original_name=Conv2d) | |
| (1): RecursiveScriptModule(original_name=ELU) | |
| (2): RecursiveScriptModule(original_name=Conv2d) | |
| (3): RecursiveScriptModule(original_name=ELU) | |
| (4): RecursiveScriptModule(original_name=Conv2d) | |
| (5): RecursiveScriptModule(original_name=ELU) | |
| ) | |
| (mlp_layers): RecursiveScriptModule( | |
| original_name=Sequential | |
| (0): RecursiveScriptModule(original_name=Linear) | |
| (1): RecursiveScriptModule(original_name=ELU) | |
| ) | |
| ) | |
| ) | |
| ) | |
| (core): ModelCoreRNN( | |
| (core): GRU(512, 512) | |
| ) | |
| (decoder): MlpDecoder( | |
| (mlp): Identity() | |
| ) | |
| (critic_linear): Linear(in_features=512, out_features=1, bias=True) | |
| (action_parameterization): ActionParameterizationDefault( | |
| (distribution_linear): Linear(in_features=512, out_features=5, bias=True) | |
| ) | |
| ) | |
| [2023-12-26 21:54:59,805][15812] Worker 0 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] | |
| [2023-12-26 21:54:59,854][15815] Worker 2 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] | |
| [2023-12-26 21:55:00,319][15813] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-12-26 21:55:00,320][15813] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 | |
| [2023-12-26 21:55:00,336][15813] Num visible devices: 1 | |
| [2023-12-26 21:55:00,384][15832] Worker 4 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] | |
| [2023-12-26 21:55:00,387][15816] Worker 3 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] | |
| [2023-12-26 21:55:00,388][15828] Worker 7 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] | |
| [2023-12-26 21:55:00,392][15830] Worker 6 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] | |
| [2023-12-26 21:55:00,408][15787] Using optimizer <class 'torch.optim.adam.Adam'> | |
| [2023-12-26 21:55:00,461][15787] EvtLoop [learner_proc0_evt_loop, process=learner_proc0] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Runner_EvtLoop', signal_name='start'), args=() | |
| Traceback (most recent call last): | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/signal_slot/signal_slot.py", line 355, in _process_signal | |
| slot_callable(*args) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sample_factory/algo/learning/learner_worker.py", line 139, in init | |
| init_model_data = self.learner.init() | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sample_factory/algo/learning/learner.py", line 243, in init | |
| self.optimizer = optimizer_cls(params, **optimizer_kwargs) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/torch/optim/adam.py", line 45, in __init__ | |
| super().__init__(params, defaults) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/torch/optim/optimizer.py", line 266, in __init__ | |
| self.add_param_group(cast(dict, param_group)) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/torch/_compile.py", line 22, in inner | |
| import torch._dynamo | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/torch/_dynamo/__init__.py", line 2, in <module> | |
| from . import allowed_functions, convert_frame, eval_frame, resume_execution | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/torch/_dynamo/allowed_functions.py", line 26, in <module> | |
| from . import config | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/torch/_dynamo/config.py", line 49, in <module> | |
| torch.onnx.is_in_onnx_export: False, | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/torch/__init__.py", line 1831, in __getattr__ | |
| return importlib.import_module(f".{name}", __name__) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/importlib/__init__.py", line 126, in import_module | |
| return _bootstrap._gcd_import(name[level:], package, level) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/torch/onnx/__init__.py", line 57, in <module> | |
| from ._internal.onnxruntime import ( | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/torch/onnx/_internal/onnxruntime.py", line 34, in <module> | |
| import onnx | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/onnx/__init__.py", line 6, in <module> | |
| from onnx.external_data_helper import load_external_data_for_model, write_external_data_tensors, convert_model_to_external_data | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/onnx/external_data_helper.py", line 9, in <module> | |
| from .onnx_pb import TensorProto, ModelProto, AttributeProto, GraphProto | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/onnx/onnx_pb.py", line 4, in <module> | |
| from .onnx_ml_pb2 import * # noqa | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/onnx/onnx_ml_pb2.py", line 33, in <module> | |
| _descriptor.EnumValueDescriptor( | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/google/protobuf/descriptor.py", line 796, in __new__ | |
| _message.Message._CheckCalledFromGeneratedFile() | |
| TypeError: Descriptors cannot not be created directly. | |
| If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0. | |
| If you cannot immediately regenerate your protos, some other possible workarounds are: | |
| 1. Downgrade the protobuf package to 3.20.x or lower. | |
| 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower). | |
| More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates | |
| [2023-12-26 21:55:00,462][15787] Unhandled exception Descriptors cannot not be created directly. | |
| If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0. | |
| If you cannot immediately regenerate your protos, some other possible workarounds are: | |
| 1. Downgrade the protobuf package to 3.20.x or lower. | |
| 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower). | |
| More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates in evt loop learner_proc0_evt_loop | |
| [2023-12-26 21:55:00,867][15814] Worker 1 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] | |
| [2023-12-26 21:55:01,468][15829] Worker 5 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] | |
| [2023-12-26 21:55:16,564][15743] Heartbeat connected on Batcher_0 | |
| [2023-12-26 21:55:16,570][15743] Heartbeat connected on InferenceWorker_p0-w0 | |
| [2023-12-26 21:55:16,573][15743] Heartbeat connected on RolloutWorker_w0 | |
| [2023-12-26 21:55:16,575][15743] Heartbeat connected on RolloutWorker_w1 | |
| [2023-12-26 21:55:16,577][15743] Heartbeat connected on RolloutWorker_w2 | |
| [2023-12-26 21:55:16,580][15743] Heartbeat connected on RolloutWorker_w3 | |
| [2023-12-26 21:55:16,582][15743] Heartbeat connected on RolloutWorker_w4 | |
| [2023-12-26 21:55:16,584][15743] Heartbeat connected on RolloutWorker_w5 | |
| [2023-12-26 21:55:16,587][15743] Heartbeat connected on RolloutWorker_w6 | |
| [2023-12-26 21:55:16,590][15743] Heartbeat connected on RolloutWorker_w7 | |
| [2023-12-26 21:55:41,089][15743] Keyboard interrupt detected in the event loop EvtLoop [Runner_EvtLoop, process=main process 15743], exiting... | |
| [2023-12-26 21:55:41,091][15813] Stopping InferenceWorker_p0-w0... | |
| [2023-12-26 21:55:41,091][15832] Stopping RolloutWorker_w4... | |
| [2023-12-26 21:55:41,091][15828] Stopping RolloutWorker_w7... | |
| [2023-12-26 21:55:41,091][15812] Stopping RolloutWorker_w0... | |
| [2023-12-26 21:55:41,091][15829] Stopping RolloutWorker_w5... | |
| [2023-12-26 21:55:41,091][15743] Runner profile tree view: | |
| main_loop: 44.5019 | |
| [2023-12-26 21:55:41,091][15814] Stopping RolloutWorker_w1... | |
| [2023-12-26 21:55:41,091][15816] Stopping RolloutWorker_w3... | |
| [2023-12-26 21:55:41,092][15743] Collected {}, FPS: 0.0 | |
| [2023-12-26 21:55:41,092][15813] Loop inference_proc0-0_evt_loop terminating... | |
| [2023-12-26 21:55:41,092][15787] Stopping Batcher_0... | |
| [2023-12-26 21:55:41,091][15815] Stopping RolloutWorker_w2... | |
| [2023-12-26 21:55:41,092][15829] Loop rollout_proc5_evt_loop terminating... | |
| [2023-12-26 21:55:41,092][15812] Loop rollout_proc0_evt_loop terminating... | |
| [2023-12-26 21:55:41,092][15814] Loop rollout_proc1_evt_loop terminating... | |
| [2023-12-26 21:55:41,092][15828] Loop rollout_proc7_evt_loop terminating... | |
| [2023-12-26 21:55:41,092][15832] Loop rollout_proc4_evt_loop terminating... | |
| [2023-12-26 21:55:41,093][15815] Loop rollout_proc2_evt_loop terminating... | |
| [2023-12-26 21:55:41,093][15787] Loop batcher_evt_loop terminating... | |
| [2023-12-26 21:55:41,093][15816] Loop rollout_proc3_evt_loop terminating... | |
| [2023-12-26 21:55:41,096][15830] Stopping RolloutWorker_w6... | |
| [2023-12-26 21:55:41,097][15830] Loop rollout_proc6_evt_loop terminating... | |
| [2023-12-26 21:57:14,410][16123] Saving configuration to /home/cybertron/Desktop/rl_units/train_dir/default_experiment/config.json... | |
| [2023-12-26 21:57:14,411][16123] Rollout worker 0 uses device cpu | |
| [2023-12-26 21:57:14,411][16123] Rollout worker 1 uses device cpu | |
| [2023-12-26 21:57:14,411][16123] Rollout worker 2 uses device cpu | |
| [2023-12-26 21:57:14,411][16123] Rollout worker 3 uses device cpu | |
| [2023-12-26 21:57:14,411][16123] Rollout worker 4 uses device cpu | |
| [2023-12-26 21:57:14,411][16123] Rollout worker 5 uses device cpu | |
| [2023-12-26 21:57:14,411][16123] Rollout worker 6 uses device cpu | |
| [2023-12-26 21:57:14,411][16123] Rollout worker 7 uses device cpu | |
| [2023-12-26 21:57:14,462][16123] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-12-26 21:57:14,462][16123] InferenceWorker_p0-w0: min num requests: 2 | |
| [2023-12-26 21:57:14,482][16123] Starting all processes... | |
| [2023-12-26 21:57:14,483][16123] Starting process learner_proc0 | |
| [2023-12-26 21:57:15,674][16123] Starting all processes... | |
| [2023-12-26 21:57:15,676][16123] Starting process inference_proc0-0 | |
| [2023-12-26 21:57:15,677][16123] Starting process rollout_proc0 | |
| [2023-12-26 21:57:15,678][16167] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-12-26 21:57:15,678][16167] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 | |
| [2023-12-26 21:57:15,677][16123] Starting process rollout_proc1 | |
| [2023-12-26 21:57:15,677][16123] Starting process rollout_proc2 | |
| [2023-12-26 21:57:15,677][16123] Starting process rollout_proc3 | |
| [2023-12-26 21:57:15,677][16123] Starting process rollout_proc4 | |
| [2023-12-26 21:57:15,679][16123] Starting process rollout_proc5 | |
| [2023-12-26 21:57:15,692][16167] Num visible devices: 1 | |
| [2023-12-26 21:57:15,680][16123] Starting process rollout_proc6 | |
| [2023-12-26 21:57:15,718][16167] Starting seed is not provided | |
| [2023-12-26 21:57:15,719][16167] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-12-26 21:57:15,719][16167] Initializing actor-critic model on device cuda:0 | |
| [2023-12-26 21:57:15,719][16167] RunningMeanStd input shape: (3, 72, 128) | |
| [2023-12-26 21:57:15,720][16167] RunningMeanStd input shape: (1,) | |
| [2023-12-26 21:57:15,680][16123] Starting process rollout_proc7 | |
| [2023-12-26 21:57:15,735][16167] ConvEncoder: input_channels=3 | |
| [2023-12-26 21:57:15,867][16167] Conv encoder output size: 512 | |
| [2023-12-26 21:57:15,867][16167] Policy head output size: 512 | |
| [2023-12-26 21:57:15,884][16167] Created Actor Critic model with architecture: | |
| [2023-12-26 21:57:15,884][16167] ActorCriticSharedWeights( | |
| (obs_normalizer): ObservationNormalizer( | |
| (running_mean_std): RunningMeanStdDictInPlace( | |
| (running_mean_std): ModuleDict( | |
| (obs): RunningMeanStdInPlace() | |
| ) | |
| ) | |
| ) | |
| (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) | |
| (encoder): VizdoomEncoder( | |
| (basic_encoder): ConvEncoder( | |
| (enc): RecursiveScriptModule( | |
| original_name=ConvEncoderImpl | |
| (conv_head): RecursiveScriptModule( | |
| original_name=Sequential | |
| (0): RecursiveScriptModule(original_name=Conv2d) | |
| (1): RecursiveScriptModule(original_name=ELU) | |
| (2): RecursiveScriptModule(original_name=Conv2d) | |
| (3): RecursiveScriptModule(original_name=ELU) | |
| (4): RecursiveScriptModule(original_name=Conv2d) | |
| (5): RecursiveScriptModule(original_name=ELU) | |
| ) | |
| (mlp_layers): RecursiveScriptModule( | |
| original_name=Sequential | |
| (0): RecursiveScriptModule(original_name=Linear) | |
| (1): RecursiveScriptModule(original_name=ELU) | |
| ) | |
| ) | |
| ) | |
| ) | |
| (core): ModelCoreRNN( | |
| (core): GRU(512, 512) | |
| ) | |
| (decoder): MlpDecoder( | |
| (mlp): Identity() | |
| ) | |
| (critic_linear): Linear(in_features=512, out_features=1, bias=True) | |
| (action_parameterization): ActionParameterizationDefault( | |
| (distribution_linear): Linear(in_features=512, out_features=5, bias=True) | |
| ) | |
| ) | |
| [2023-12-26 21:57:17,658][16193] Worker 1 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] | |
| [2023-12-26 21:57:17,700][16206] Worker 2 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] | |
| [2023-12-26 21:57:18,052][16209] Worker 5 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] | |
| [2023-12-26 21:57:18,080][16191] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-12-26 21:57:18,080][16191] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 | |
| [2023-12-26 21:57:18,094][16191] Num visible devices: 1 | |
| [2023-12-26 21:57:18,101][16167] Using optimizer <class 'torch.optim.adam.Adam'> | |
| [2023-12-26 21:57:18,109][16210] Worker 7 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] | |
| [2023-12-26 21:57:18,161][16194] Worker 3 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] | |
| [2023-12-26 21:57:18,205][16192] Worker 0 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] | |
| [2023-12-26 21:57:18,243][16207] Worker 4 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] | |
| [2023-12-26 21:57:18,249][16211] Worker 6 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] | |
| [2023-12-26 21:57:18,341][16167] No checkpoints found | |
| [2023-12-26 21:57:18,341][16167] Did not load from checkpoint, starting from scratch! | |
| [2023-12-26 21:57:18,341][16167] Initialized policy 0 weights for model version 0 | |
| [2023-12-26 21:57:18,342][16167] LearnerWorker_p0 finished initialization! | |
| [2023-12-26 21:57:18,342][16167] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-12-26 21:57:19,229][16191] RunningMeanStd input shape: (3, 72, 128) | |
| [2023-12-26 21:57:19,229][16191] RunningMeanStd input shape: (1,) | |
| [2023-12-26 21:57:19,236][16191] ConvEncoder: input_channels=3 | |
| [2023-12-26 21:57:19,309][16191] Conv encoder output size: 512 | |
| [2023-12-26 21:57:19,309][16191] Policy head output size: 512 | |
| [2023-12-26 21:57:19,604][16123] Inference worker 0-0 is ready! | |
| [2023-12-26 21:57:19,604][16123] All inference workers are ready! Signal rollout workers to start! | |
| [2023-12-26 21:57:19,638][16194] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-12-26 21:57:19,638][16210] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-12-26 21:57:19,639][16206] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-12-26 21:57:19,646][16192] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-12-26 21:57:19,651][16207] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-12-26 21:57:19,655][16209] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-12-26 21:57:19,664][16193] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-12-26 21:57:19,664][16211] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-12-26 21:57:20,137][16210] Decorrelating experience for 0 frames... | |
| [2023-12-26 21:57:20,141][16194] Decorrelating experience for 0 frames... | |
| [2023-12-26 21:57:20,147][16207] Decorrelating experience for 0 frames... | |
| [2023-12-26 21:57:20,150][16192] Decorrelating experience for 0 frames... | |
| [2023-12-26 21:57:20,152][16193] Decorrelating experience for 0 frames... | |
| [2023-12-26 21:57:20,366][16209] Decorrelating experience for 0 frames... | |
| [2023-12-26 21:57:20,367][16194] Decorrelating experience for 32 frames... | |
| [2023-12-26 21:57:20,369][16211] Decorrelating experience for 0 frames... | |
| [2023-12-26 21:57:20,370][16192] Decorrelating experience for 32 frames... | |
| [2023-12-26 21:57:20,398][16210] Decorrelating experience for 32 frames... | |
| [2023-12-26 21:57:20,592][16209] Decorrelating experience for 32 frames... | |
| [2023-12-26 21:57:20,593][16211] Decorrelating experience for 32 frames... | |
| [2023-12-26 21:57:20,638][16193] Decorrelating experience for 32 frames... | |
| [2023-12-26 21:57:20,665][16210] Decorrelating experience for 64 frames... | |
| [2023-12-26 21:57:20,819][16207] Decorrelating experience for 32 frames... | |
| [2023-12-26 21:57:20,842][16192] Decorrelating experience for 64 frames... | |
| [2023-12-26 21:57:20,890][16194] Decorrelating experience for 64 frames... | |
| [2023-12-26 21:57:20,896][16209] Decorrelating experience for 64 frames... | |
| [2023-12-26 21:57:20,918][16193] Decorrelating experience for 64 frames... | |
| [2023-12-26 21:57:21,053][16206] Decorrelating experience for 0 frames... | |
| [2023-12-26 21:57:21,087][16192] Decorrelating experience for 96 frames... | |
| [2023-12-26 21:57:21,137][16209] Decorrelating experience for 96 frames... | |
| [2023-12-26 21:57:21,137][16194] Decorrelating experience for 96 frames... | |
| [2023-12-26 21:57:21,275][16206] Decorrelating experience for 32 frames... | |
| [2023-12-26 21:57:21,307][16207] Decorrelating experience for 64 frames... | |
| [2023-12-26 21:57:21,368][16193] Decorrelating experience for 96 frames... | |
| [2023-12-26 21:57:21,545][16206] Decorrelating experience for 64 frames... | |
| [2023-12-26 21:57:21,548][16207] Decorrelating experience for 96 frames... | |
| [2023-12-26 21:57:21,580][16211] Decorrelating experience for 64 frames... | |
| [2023-12-26 21:57:21,754][16123] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2023-12-26 21:57:21,754][16123] Avg episode reward: [(0, '0.320')] | |
| [2023-12-26 21:57:21,835][16206] Decorrelating experience for 96 frames... | |
| [2023-12-26 21:57:21,889][16210] Decorrelating experience for 96 frames... | |
| [2023-12-26 21:57:21,891][16211] Decorrelating experience for 96 frames... | |
| [2023-12-26 21:57:22,244][16167] Signal inference workers to stop experience collection... | |
| [2023-12-26 21:57:22,261][16191] InferenceWorker_p0-w0: stopping experience collection | |
| [2023-12-26 21:57:23,752][16167] Signal inference workers to resume experience collection... | |
| [2023-12-26 21:57:23,753][16191] InferenceWorker_p0-w0: resuming experience collection | |
| [2023-12-26 21:57:25,439][16191] Updated weights for policy 0, policy_version 10 (0.0133) | |
| [2023-12-26 21:57:26,754][16123] Fps is (10 sec: 13926.4, 60 sec: 13926.4, 300 sec: 13926.4). Total num frames: 69632. Throughput: 0: 2332.0. Samples: 11660. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2023-12-26 21:57:26,754][16123] Avg episode reward: [(0, '4.582')] | |
| [2023-12-26 21:57:27,239][16191] Updated weights for policy 0, policy_version 20 (0.0010) | |
| [2023-12-26 21:57:29,033][16191] Updated weights for policy 0, policy_version 30 (0.0010) | |
| [2023-12-26 21:57:30,847][16191] Updated weights for policy 0, policy_version 40 (0.0010) | |
| [2023-12-26 21:57:31,754][16123] Fps is (10 sec: 18022.5, 60 sec: 18022.5, 300 sec: 18022.5). Total num frames: 180224. Throughput: 0: 4577.0. Samples: 45770. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2023-12-26 21:57:31,754][16123] Avg episode reward: [(0, '4.499')] | |
| [2023-12-26 21:57:31,757][16167] Saving new best policy, reward=4.499! | |
| [2023-12-26 21:57:32,677][16191] Updated weights for policy 0, policy_version 50 (0.0010) | |
| [2023-12-26 21:57:34,457][16123] Heartbeat connected on Batcher_0 | |
| [2023-12-26 21:57:34,469][16123] Heartbeat connected on InferenceWorker_p0-w0 | |
| [2023-12-26 21:57:34,469][16123] Heartbeat connected on RolloutWorker_w0 | |
| [2023-12-26 21:57:34,470][16123] Heartbeat connected on RolloutWorker_w2 | |
| [2023-12-26 21:57:34,472][16123] Heartbeat connected on RolloutWorker_w1 | |
| [2023-12-26 21:57:34,474][16123] Heartbeat connected on RolloutWorker_w3 | |
| [2023-12-26 21:57:34,475][16123] Heartbeat connected on RolloutWorker_w4 | |
| [2023-12-26 21:57:34,477][16123] Heartbeat connected on RolloutWorker_w5 | |
| [2023-12-26 21:57:34,482][16123] Heartbeat connected on RolloutWorker_w7 | |
| [2023-12-26 21:57:34,484][16191] Updated weights for policy 0, policy_version 60 (0.0010) | |
| [2023-12-26 21:57:34,490][16123] Heartbeat connected on LearnerWorker_p0 | |
| [2023-12-26 21:57:34,490][16123] Heartbeat connected on RolloutWorker_w6 | |
| [2023-12-26 21:57:36,332][16191] Updated weights for policy 0, policy_version 70 (0.0010) | |
| [2023-12-26 21:57:36,754][16123] Fps is (10 sec: 22528.1, 60 sec: 19660.9, 300 sec: 19660.9). Total num frames: 294912. Throughput: 0: 4181.7. Samples: 62726. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-12-26 21:57:36,754][16123] Avg episode reward: [(0, '4.436')] | |
| [2023-12-26 21:57:38,176][16191] Updated weights for policy 0, policy_version 80 (0.0010) | |
| [2023-12-26 21:57:40,031][16191] Updated weights for policy 0, policy_version 90 (0.0009) | |
| [2023-12-26 21:57:41,753][16123] Fps is (10 sec: 22528.0, 60 sec: 20275.3, 300 sec: 20275.3). Total num frames: 405504. Throughput: 0: 4798.0. Samples: 95960. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-12-26 21:57:41,754][16123] Avg episode reward: [(0, '4.625')] | |
| [2023-12-26 21:57:41,757][16167] Saving new best policy, reward=4.625! | |
| [2023-12-26 21:57:41,948][16191] Updated weights for policy 0, policy_version 100 (0.0010) | |
| [2023-12-26 21:57:43,852][16191] Updated weights for policy 0, policy_version 110 (0.0010) | |
| [2023-12-26 21:57:45,848][16191] Updated weights for policy 0, policy_version 120 (0.0010) | |
| [2023-12-26 21:57:46,754][16123] Fps is (10 sec: 21299.1, 60 sec: 20316.2, 300 sec: 20316.2). Total num frames: 507904. Throughput: 0: 5102.9. Samples: 127572. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-12-26 21:57:46,754][16123] Avg episode reward: [(0, '4.560')] | |
| [2023-12-26 21:57:47,771][16191] Updated weights for policy 0, policy_version 130 (0.0010) | |
| [2023-12-26 21:57:49,675][16191] Updated weights for policy 0, policy_version 140 (0.0009) | |
| [2023-12-26 21:57:51,602][16191] Updated weights for policy 0, policy_version 150 (0.0010) | |
| [2023-12-26 21:57:51,754][16123] Fps is (10 sec: 20889.5, 60 sec: 20480.0, 300 sec: 20480.0). Total num frames: 614400. Throughput: 0: 4789.0. Samples: 143670. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-12-26 21:57:51,754][16123] Avg episode reward: [(0, '4.664')] | |
| [2023-12-26 21:57:51,758][16167] Saving new best policy, reward=4.664! | |
| [2023-12-26 21:57:53,555][16191] Updated weights for policy 0, policy_version 160 (0.0010) | |
| [2023-12-26 21:57:55,496][16191] Updated weights for policy 0, policy_version 170 (0.0010) | |
| [2023-12-26 21:57:56,754][16123] Fps is (10 sec: 21299.3, 60 sec: 20597.1, 300 sec: 20597.1). Total num frames: 720896. Throughput: 0: 5011.4. Samples: 175400. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-12-26 21:57:56,754][16123] Avg episode reward: [(0, '4.467')] | |
| [2023-12-26 21:57:57,377][16191] Updated weights for policy 0, policy_version 180 (0.0010) | |
| [2023-12-26 21:57:59,269][16191] Updated weights for policy 0, policy_version 190 (0.0010) | |
| [2023-12-26 21:58:01,206][16191] Updated weights for policy 0, policy_version 200 (0.0010) | |
| [2023-12-26 21:58:01,754][16123] Fps is (10 sec: 21299.2, 60 sec: 20684.8, 300 sec: 20684.8). Total num frames: 827392. Throughput: 0: 5189.0. Samples: 207560. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2023-12-26 21:58:01,754][16123] Avg episode reward: [(0, '4.731')] | |
| [2023-12-26 21:58:01,767][16167] Saving new best policy, reward=4.731! | |
| [2023-12-26 21:58:03,075][16191] Updated weights for policy 0, policy_version 210 (0.0010) | |
| [2023-12-26 21:58:04,974][16191] Updated weights for policy 0, policy_version 220 (0.0010) | |
| [2023-12-26 21:58:06,754][16123] Fps is (10 sec: 21708.7, 60 sec: 20844.1, 300 sec: 20844.1). Total num frames: 937984. Throughput: 0: 4976.1. Samples: 223924. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2023-12-26 21:58:06,754][16123] Avg episode reward: [(0, '4.930')] | |
| [2023-12-26 21:58:06,754][16167] Saving new best policy, reward=4.930! | |
| [2023-12-26 21:58:06,893][16191] Updated weights for policy 0, policy_version 230 (0.0010) | |
| [2023-12-26 21:58:08,853][16191] Updated weights for policy 0, policy_version 240 (0.0010) | |
| [2023-12-26 21:58:10,757][16191] Updated weights for policy 0, policy_version 250 (0.0010) | |
| [2023-12-26 21:58:11,754][16123] Fps is (10 sec: 21708.9, 60 sec: 20889.6, 300 sec: 20889.6). Total num frames: 1044480. Throughput: 0: 5424.6. Samples: 255768. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-12-26 21:58:11,754][16123] Avg episode reward: [(0, '5.146')] | |
| [2023-12-26 21:58:11,758][16167] Saving new best policy, reward=5.146! | |
| [2023-12-26 21:58:12,722][16191] Updated weights for policy 0, policy_version 260 (0.0010) | |
| [2023-12-26 21:58:14,688][16191] Updated weights for policy 0, policy_version 270 (0.0010) | |
| [2023-12-26 21:58:16,582][16191] Updated weights for policy 0, policy_version 280 (0.0010) | |
| [2023-12-26 21:58:16,754][16123] Fps is (10 sec: 20889.5, 60 sec: 20852.3, 300 sec: 20852.3). Total num frames: 1146880. Throughput: 0: 5374.3. Samples: 287614. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-12-26 21:58:16,754][16123] Avg episode reward: [(0, '5.436')] | |
| [2023-12-26 21:58:16,764][16167] Saving new best policy, reward=5.436! | |
| [2023-12-26 21:58:18,508][16191] Updated weights for policy 0, policy_version 290 (0.0011) | |
| [2023-12-26 21:58:20,396][16191] Updated weights for policy 0, policy_version 300 (0.0010) | |
| [2023-12-26 21:58:21,754][16123] Fps is (10 sec: 21299.1, 60 sec: 20957.9, 300 sec: 20957.9). Total num frames: 1257472. Throughput: 0: 5354.6. Samples: 303682. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) | |
| [2023-12-26 21:58:21,754][16123] Avg episode reward: [(0, '5.672')] | |
| [2023-12-26 21:58:21,758][16167] Saving new best policy, reward=5.672! | |
| [2023-12-26 21:58:22,300][16191] Updated weights for policy 0, policy_version 310 (0.0010) | |
| [2023-12-26 21:58:24,212][16191] Updated weights for policy 0, policy_version 320 (0.0011) | |
| [2023-12-26 21:58:24,990][16211] EvtLoop [rollout_proc6_evt_loop, process=rollout_proc6] unhandled exception in slot='advance_rollouts' connected to emitter=Emitter(object_id='InferenceWorker_p0-w0', signal_name='advance6'), args=(1, 0) | |
| Traceback (most recent call last): | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/signal_slot/signal_slot.py", line 355, in _process_signal | |
| slot_callable(*args) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sample_factory/algo/sampling/rollout_worker.py", line 241, in advance_rollouts | |
| complete_rollouts, episodic_stats = runner.advance_rollouts(policy_id, self.timing) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 634, in advance_rollouts | |
| new_obs, rewards, terminated, truncated, infos = e.step(actions) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/gymnasium/core.py", line 461, in step | |
| return self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sample_factory/algo/utils/make_env.py", line 129, in step | |
| obs, rew, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sample_factory/algo/utils/make_env.py", line 115, in step | |
| obs, rew, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sf_examples/vizdoom/doom/wrappers/scenario_wrappers/gathering_reward_shaping.py", line 33, in step | |
| observation, reward, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/gymnasium/core.py", line 522, in step | |
| observation, reward, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sample_factory/envs/env_wrappers.py", line 86, in step | |
| obs, reward, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/gymnasium/core.py", line 461, in step | |
| return self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sf_examples/vizdoom/doom/wrappers/multiplayer_stats.py", line 54, in step | |
| obs, reward, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sf_examples/vizdoom/doom/doom_gym.py", line 452, in step | |
| reward = self.game.make_action(actions_flattened, self.skip_frames) | |
| vizdoom.vizdoom.SignalException: Signal SIGINT received. ViZDoom instance has been closed. | |
| [2023-12-26 21:58:24,991][16211] Unhandled exception Signal SIGINT received. ViZDoom instance has been closed. in evt loop rollout_proc6_evt_loop | |
| [2023-12-26 21:58:24,991][16206] EvtLoop [rollout_proc2_evt_loop, process=rollout_proc2] unhandled exception in slot='advance_rollouts' connected to emitter=Emitter(object_id='InferenceWorker_p0-w0', signal_name='advance2'), args=(1, 0) | |
| Traceback (most recent call last): | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/signal_slot/signal_slot.py", line 355, in _process_signal | |
| slot_callable(*args) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sample_factory/algo/sampling/rollout_worker.py", line 241, in advance_rollouts | |
| complete_rollouts, episodic_stats = runner.advance_rollouts(policy_id, self.timing) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 634, in advance_rollouts | |
| new_obs, rewards, terminated, truncated, infos = e.step(actions) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/gymnasium/core.py", line 461, in step | |
| return self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sample_factory/algo/utils/make_env.py", line 129, in step | |
| obs, rew, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sample_factory/algo/utils/make_env.py", line 115, in step | |
| obs, rew, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sf_examples/vizdoom/doom/wrappers/scenario_wrappers/gathering_reward_shaping.py", line 33, in step | |
| observation, reward, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/gymnasium/core.py", line 522, in step | |
| observation, reward, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sample_factory/envs/env_wrappers.py", line 86, in step | |
| obs, reward, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/gymnasium/core.py", line 461, in step | |
| return self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sf_examples/vizdoom/doom/wrappers/multiplayer_stats.py", line 54, in step | |
| obs, reward, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sf_examples/vizdoom/doom/doom_gym.py", line 452, in step | |
| reward = self.game.make_action(actions_flattened, self.skip_frames) | |
| vizdoom.vizdoom.SignalException: Signal SIGINT received. ViZDoom instance has been closed. | |
| [2023-12-26 21:58:24,993][16206] Unhandled exception Signal SIGINT received. ViZDoom instance has been closed. in evt loop rollout_proc2_evt_loop | |
| [2023-12-26 21:58:24,993][16207] EvtLoop [rollout_proc4_evt_loop, process=rollout_proc4] unhandled exception in slot='advance_rollouts' connected to emitter=Emitter(object_id='InferenceWorker_p0-w0', signal_name='advance4'), args=(1, 0) | |
| Traceback (most recent call last): | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/signal_slot/signal_slot.py", line 355, in _process_signal | |
| slot_callable(*args) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sample_factory/algo/sampling/rollout_worker.py", line 241, in advance_rollouts | |
| complete_rollouts, episodic_stats = runner.advance_rollouts(policy_id, self.timing) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 634, in advance_rollouts | |
| new_obs, rewards, terminated, truncated, infos = e.step(actions) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/gymnasium/core.py", line 461, in step | |
| return self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sample_factory/algo/utils/make_env.py", line 129, in step | |
| obs, rew, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sample_factory/algo/utils/make_env.py", line 115, in step | |
| obs, rew, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sf_examples/vizdoom/doom/wrappers/scenario_wrappers/gathering_reward_shaping.py", line 33, in step | |
| observation, reward, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/gymnasium/core.py", line 522, in step | |
| observation, reward, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sample_factory/envs/env_wrappers.py", line 86, in step | |
| obs, reward, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/gymnasium/core.py", line 461, in step | |
| return self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sf_examples/vizdoom/doom/wrappers/multiplayer_stats.py", line 54, in step | |
| obs, reward, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sf_examples/vizdoom/doom/doom_gym.py", line 452, in step | |
| reward = self.game.make_action(actions_flattened, self.skip_frames) | |
| vizdoom.vizdoom.SignalException: Signal SIGINT received. ViZDoom instance has been closed. | |
| [2023-12-26 21:58:24,993][16194] EvtLoop [rollout_proc3_evt_loop, process=rollout_proc3] unhandled exception in slot='advance_rollouts' connected to emitter=Emitter(object_id='InferenceWorker_p0-w0', signal_name='advance3'), args=(0, 0) | |
| Traceback (most recent call last): | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/signal_slot/signal_slot.py", line 355, in _process_signal | |
| slot_callable(*args) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sample_factory/algo/sampling/rollout_worker.py", line 241, in advance_rollouts | |
| complete_rollouts, episodic_stats = runner.advance_rollouts(policy_id, self.timing) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 634, in advance_rollouts | |
| new_obs, rewards, terminated, truncated, infos = e.step(actions) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/gymnasium/core.py", line 461, in step | |
| return self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sample_factory/algo/utils/make_env.py", line 129, in step | |
| obs, rew, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sample_factory/algo/utils/make_env.py", line 115, in step | |
| obs, rew, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sf_examples/vizdoom/doom/wrappers/scenario_wrappers/gathering_reward_shaping.py", line 33, in step | |
| observation, reward, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/gymnasium/core.py", line 522, in step | |
| observation, reward, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sample_factory/envs/env_wrappers.py", line 86, in step | |
| obs, reward, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/gymnasium/core.py", line 461, in step | |
| return self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sf_examples/vizdoom/doom/wrappers/multiplayer_stats.py", line 54, in step | |
| obs, reward, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sf_examples/vizdoom/doom/doom_gym.py", line 452, in step | |
| reward = self.game.make_action(actions_flattened, self.skip_frames) | |
| vizdoom.vizdoom.SignalException: Signal SIGINT received. ViZDoom instance has been closed. | |
| [2023-12-26 21:58:24,995][16194] Unhandled exception Signal SIGINT received. ViZDoom instance has been closed. in evt loop rollout_proc3_evt_loop | |
| [2023-12-26 21:58:24,993][16193] EvtLoop [rollout_proc1_evt_loop, process=rollout_proc1] unhandled exception in slot='advance_rollouts' connected to emitter=Emitter(object_id='InferenceWorker_p0-w0', signal_name='advance1'), args=(0, 0) | |
| Traceback (most recent call last): | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/signal_slot/signal_slot.py", line 355, in _process_signal | |
| slot_callable(*args) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sample_factory/algo/sampling/rollout_worker.py", line 241, in advance_rollouts | |
| complete_rollouts, episodic_stats = runner.advance_rollouts(policy_id, self.timing) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 634, in advance_rollouts | |
| new_obs, rewards, terminated, truncated, infos = e.step(actions) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/gymnasium/core.py", line 461, in step | |
| return self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sample_factory/algo/utils/make_env.py", line 129, in step | |
| obs, rew, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sample_factory/algo/utils/make_env.py", line 115, in step | |
| obs, rew, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sf_examples/vizdoom/doom/wrappers/scenario_wrappers/gathering_reward_shaping.py", line 33, in step | |
| observation, reward, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/gymnasium/core.py", line 522, in step | |
| observation, reward, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sample_factory/envs/env_wrappers.py", line 86, in step | |
| obs, reward, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/gymnasium/core.py", line 461, in step | |
| return self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sf_examples/vizdoom/doom/wrappers/multiplayer_stats.py", line 54, in step | |
| obs, reward, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sf_examples/vizdoom/doom/doom_gym.py", line 452, in step | |
| reward = self.game.make_action(actions_flattened, self.skip_frames) | |
| vizdoom.vizdoom.SignalException: Signal SIGINT received. ViZDoom instance has been closed. | |
| [2023-12-26 21:58:24,995][16193] Unhandled exception Signal SIGINT received. ViZDoom instance has been closed. in evt loop rollout_proc1_evt_loop | |
| [2023-12-26 21:58:24,996][16192] EvtLoop [rollout_proc0_evt_loop, process=rollout_proc0] unhandled exception in slot='advance_rollouts' connected to emitter=Emitter(object_id='InferenceWorker_p0-w0', signal_name='advance0'), args=(0, 0) | |
| Traceback (most recent call last): | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/signal_slot/signal_slot.py", line 355, in _process_signal | |
| slot_callable(*args) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sample_factory/algo/sampling/rollout_worker.py", line 241, in advance_rollouts | |
| complete_rollouts, episodic_stats = runner.advance_rollouts(policy_id, self.timing) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 634, in advance_rollouts | |
| new_obs, rewards, terminated, truncated, infos = e.step(actions) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/gymnasium/core.py", line 461, in step | |
| return self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sample_factory/algo/utils/make_env.py", line 129, in step | |
| obs, rew, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sample_factory/algo/utils/make_env.py", line 115, in step | |
| obs, rew, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sf_examples/vizdoom/doom/wrappers/scenario_wrappers/gathering_reward_shaping.py", line 33, in step | |
| observation, reward, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/gymnasium/core.py", line 522, in step | |
| observation, reward, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sample_factory/envs/env_wrappers.py", line 86, in step | |
| obs, reward, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/gymnasium/core.py", line 461, in step | |
| return self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sf_examples/vizdoom/doom/wrappers/multiplayer_stats.py", line 54, in step | |
| obs, reward, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sf_examples/vizdoom/doom/doom_gym.py", line 452, in step | |
| reward = self.game.make_action(actions_flattened, self.skip_frames) | |
| vizdoom.vizdoom.SignalException: Signal SIGINT received. ViZDoom instance has been closed. | |
| [2023-12-26 21:58:24,997][16192] Unhandled exception Signal SIGINT received. ViZDoom instance has been closed. in evt loop rollout_proc0_evt_loop | |
| [2023-12-26 21:58:24,994][16207] Unhandled exception Signal SIGINT received. ViZDoom instance has been closed. in evt loop rollout_proc4_evt_loop | |
| [2023-12-26 21:58:24,997][16210] EvtLoop [rollout_proc7_evt_loop, process=rollout_proc7] unhandled exception in slot='advance_rollouts' connected to emitter=Emitter(object_id='InferenceWorker_p0-w0', signal_name='advance7'), args=(0, 0) | |
| Traceback (most recent call last): | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/signal_slot/signal_slot.py", line 355, in _process_signal | |
| slot_callable(*args) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sample_factory/algo/sampling/rollout_worker.py", line 241, in advance_rollouts | |
| complete_rollouts, episodic_stats = runner.advance_rollouts(policy_id, self.timing) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 634, in advance_rollouts | |
| new_obs, rewards, terminated, truncated, infos = e.step(actions) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/gymnasium/core.py", line 461, in step | |
| return self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sample_factory/algo/utils/make_env.py", line 129, in step | |
| obs, rew, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sample_factory/algo/utils/make_env.py", line 115, in step | |
| obs, rew, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sf_examples/vizdoom/doom/wrappers/scenario_wrappers/gathering_reward_shaping.py", line 33, in step | |
| observation, reward, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/gymnasium/core.py", line 522, in step | |
| observation, reward, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sample_factory/envs/env_wrappers.py", line 86, in step | |
| obs, reward, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/gymnasium/core.py", line 461, in step | |
| return self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sf_examples/vizdoom/doom/wrappers/multiplayer_stats.py", line 54, in step | |
| obs, reward, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sf_examples/vizdoom/doom/doom_gym.py", line 452, in step | |
| reward = self.game.make_action(actions_flattened, self.skip_frames) | |
| vizdoom.vizdoom.SignalException: Signal SIGINT received. ViZDoom instance has been closed. | |
| [2023-12-26 21:58:25,006][16210] Unhandled exception Signal SIGINT received. ViZDoom instance has been closed. in evt loop rollout_proc7_evt_loop | |
| [2023-12-26 21:58:25,000][16209] EvtLoop [rollout_proc5_evt_loop, process=rollout_proc5] unhandled exception in slot='advance_rollouts' connected to emitter=Emitter(object_id='InferenceWorker_p0-w0', signal_name='advance5'), args=(0, 0) | |
| Traceback (most recent call last): | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/signal_slot/signal_slot.py", line 355, in _process_signal | |
| slot_callable(*args) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sample_factory/algo/sampling/rollout_worker.py", line 241, in advance_rollouts | |
| complete_rollouts, episodic_stats = runner.advance_rollouts(policy_id, self.timing) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 634, in advance_rollouts | |
| new_obs, rewards, terminated, truncated, infos = e.step(actions) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/gymnasium/core.py", line 461, in step | |
| return self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sample_factory/algo/utils/make_env.py", line 129, in step | |
| obs, rew, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sample_factory/algo/utils/make_env.py", line 115, in step | |
| obs, rew, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sf_examples/vizdoom/doom/wrappers/scenario_wrappers/gathering_reward_shaping.py", line 33, in step | |
| observation, reward, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/gymnasium/core.py", line 522, in step | |
| observation, reward, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sample_factory/envs/env_wrappers.py", line 86, in step | |
| obs, reward, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/gymnasium/core.py", line 461, in step | |
| return self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sf_examples/vizdoom/doom/wrappers/multiplayer_stats.py", line 54, in step | |
| obs, reward, terminated, truncated, info = self.env.step(action) | |
| File "/home/cybertron/anaconda3/envs/rl/lib/python3.10/site-packages/sf_examples/vizdoom/doom/doom_gym.py", line 452, in step | |
| reward = self.game.make_action(actions_flattened, self.skip_frames) | |
| vizdoom.vizdoom.SignalException: Signal SIGINT received. ViZDoom instance has been closed. | |
| [2023-12-26 21:58:25,008][16209] Unhandled exception Signal SIGINT received. ViZDoom instance has been closed. in evt loop rollout_proc5_evt_loop | |
| [2023-12-26 21:58:25,032][16123] Keyboard interrupt detected in the event loop EvtLoop [Runner_EvtLoop, process=main process 16123], exiting... | |
| [2023-12-26 21:58:25,033][16123] Runner profile tree view: | |
| main_loop: 70.5508 | |
| [2023-12-26 21:58:25,033][16123] Collected {0: 1327104}, FPS: 18810.6 | |
| [2023-12-26 21:58:25,034][16167] Stopping Batcher_0... | |
| [2023-12-26 21:58:25,035][16167] Loop batcher_evt_loop terminating... | |
| [2023-12-26 21:58:25,040][16167] Saving /home/cybertron/Desktop/rl_units/train_dir/default_experiment/checkpoint_p0/checkpoint_000000324_1327104.pth... | |
| [2023-12-26 21:58:25,088][16191] Weights refcount: 2 0 | |
| [2023-12-26 21:58:25,090][16191] Stopping InferenceWorker_p0-w0... | |
| [2023-12-26 21:58:25,090][16191] Loop inference_proc0-0_evt_loop terminating... | |
| [2023-12-26 21:58:25,108][16167] Stopping LearnerWorker_p0... | |
| [2023-12-26 21:58:25,108][16167] Loop learner_proc0_evt_loop terminating... | |
| [2023-12-26 21:58:31,979][17486] Saving configuration to /home/cybertron/Desktop/rl_units/train_dir/default_experiment/config.json... | |
| [2023-12-26 21:58:31,980][17486] Rollout worker 0 uses device cpu | |
| [2023-12-26 21:58:31,980][17486] Rollout worker 1 uses device cpu | |
| [2023-12-26 21:58:31,980][17486] Rollout worker 2 uses device cpu | |
| [2023-12-26 21:58:31,980][17486] Rollout worker 3 uses device cpu | |
| [2023-12-26 21:58:31,980][17486] Rollout worker 4 uses device cpu | |
| [2023-12-26 21:58:31,980][17486] Rollout worker 5 uses device cpu | |
| [2023-12-26 21:58:31,980][17486] Rollout worker 6 uses device cpu | |
| [2023-12-26 21:58:31,980][17486] Rollout worker 7 uses device cpu | |
| [2023-12-26 21:58:32,034][17486] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-12-26 21:58:32,035][17486] InferenceWorker_p0-w0: min num requests: 2 | |
| [2023-12-26 21:58:32,055][17486] Starting all processes... | |
| [2023-12-26 21:58:32,056][17486] Starting process learner_proc0 | |
| [2023-12-26 21:58:33,379][17486] Starting all processes... | |
| [2023-12-26 21:58:33,382][17486] Starting process inference_proc0-0 | |
| [2023-12-26 21:58:33,383][17486] Starting process rollout_proc0 | |
| [2023-12-26 21:58:33,383][17532] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-12-26 21:58:33,383][17532] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 | |
| [2023-12-26 21:58:33,383][17486] Starting process rollout_proc1 | |
| [2023-12-26 21:58:33,384][17486] Starting process rollout_proc2 | |
| [2023-12-26 21:58:33,384][17486] Starting process rollout_proc3 | |
| [2023-12-26 21:58:33,384][17486] Starting process rollout_proc4 | |
| [2023-12-26 21:58:33,384][17486] Starting process rollout_proc5 | |
| [2023-12-26 21:58:33,384][17486] Starting process rollout_proc6 | |
| [2023-12-26 21:58:33,396][17532] Num visible devices: 1 | |
| [2023-12-26 21:58:33,384][17486] Starting process rollout_proc7 | |
| [2023-12-26 21:58:33,425][17532] Starting seed is not provided | |
| [2023-12-26 21:58:33,425][17532] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-12-26 21:58:33,426][17532] Initializing actor-critic model on device cuda:0 | |
| [2023-12-26 21:58:33,426][17532] RunningMeanStd input shape: (3, 72, 128) | |
| [2023-12-26 21:58:33,427][17532] RunningMeanStd input shape: (1,) | |
| [2023-12-26 21:58:33,442][17532] ConvEncoder: input_channels=3 | |
| [2023-12-26 21:58:33,579][17532] Conv encoder output size: 512 | |
| [2023-12-26 21:58:33,579][17532] Policy head output size: 512 | |
| [2023-12-26 21:58:33,598][17532] Created Actor Critic model with architecture: | |
| [2023-12-26 21:58:33,598][17532] ActorCriticSharedWeights( | |
| (obs_normalizer): ObservationNormalizer( | |
| (running_mean_std): RunningMeanStdDictInPlace( | |
| (running_mean_std): ModuleDict( | |
| (obs): RunningMeanStdInPlace() | |
| ) | |
| ) | |
| ) | |
| (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) | |
| (encoder): VizdoomEncoder( | |
| (basic_encoder): ConvEncoder( | |
| (enc): RecursiveScriptModule( | |
| original_name=ConvEncoderImpl | |
| (conv_head): RecursiveScriptModule( | |
| original_name=Sequential | |
| (0): RecursiveScriptModule(original_name=Conv2d) | |
| (1): RecursiveScriptModule(original_name=ELU) | |
| (2): RecursiveScriptModule(original_name=Conv2d) | |
| (3): RecursiveScriptModule(original_name=ELU) | |
| (4): RecursiveScriptModule(original_name=Conv2d) | |
| (5): RecursiveScriptModule(original_name=ELU) | |
| ) | |
| (mlp_layers): RecursiveScriptModule( | |
| original_name=Sequential | |
| (0): RecursiveScriptModule(original_name=Linear) | |
| (1): RecursiveScriptModule(original_name=ELU) | |
| ) | |
| ) | |
| ) | |
| ) | |
| (core): ModelCoreRNN( | |
| (core): GRU(512, 512) | |
| ) | |
| (decoder): MlpDecoder( | |
| (mlp): Identity() | |
| ) | |
| (critic_linear): Linear(in_features=512, out_features=1, bias=True) | |
| (action_parameterization): ActionParameterizationDefault( | |
| (distribution_linear): Linear(in_features=512, out_features=5, bias=True) | |
| ) | |
| ) | |
| [2023-12-26 21:58:35,720][17561] Worker 3 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] | |
| [2023-12-26 21:58:35,808][17577] Worker 6 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] | |
| [2023-12-26 21:58:35,980][17557] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-12-26 21:58:35,981][17557] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 | |
| [2023-12-26 21:58:35,991][17560] Worker 0 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] | |
| [2023-12-26 21:58:35,994][17557] Num visible devices: 1 | |
| [2023-12-26 21:58:36,065][17532] Using optimizer <class 'torch.optim.adam.Adam'> | |
| [2023-12-26 21:58:36,089][17574] Worker 7 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] | |
| [2023-12-26 21:58:36,094][17558] Worker 2 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] | |
| [2023-12-26 21:58:36,128][17562] Worker 4 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] | |
| [2023-12-26 21:58:36,165][17559] Worker 1 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] | |
| [2023-12-26 21:58:36,171][17575] Worker 5 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] | |
| [2023-12-26 21:58:36,307][17532] Loading state from checkpoint /home/cybertron/Desktop/rl_units/train_dir/default_experiment/checkpoint_p0/checkpoint_000000324_1327104.pth... | |
| [2023-12-26 21:58:36,329][17532] Loading model from checkpoint | |
| [2023-12-26 21:58:36,330][17532] Loaded experiment state at self.train_step=324, self.env_steps=1327104 | |
| [2023-12-26 21:58:36,330][17532] Initialized policy 0 weights for model version 324 | |
| [2023-12-26 21:58:36,331][17532] LearnerWorker_p0 finished initialization! | |
| [2023-12-26 21:58:36,331][17532] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-12-26 21:58:37,295][17557] RunningMeanStd input shape: (3, 72, 128) | |
| [2023-12-26 21:58:37,296][17557] RunningMeanStd input shape: (1,) | |
| [2023-12-26 21:58:37,303][17557] ConvEncoder: input_channels=3 | |
| [2023-12-26 21:58:37,374][17557] Conv encoder output size: 512 | |
| [2023-12-26 21:58:37,375][17557] Policy head output size: 512 | |
| [2023-12-26 21:58:37,697][17486] Inference worker 0-0 is ready! | |
| [2023-12-26 21:58:37,697][17486] All inference workers are ready! Signal rollout workers to start! | |
| [2023-12-26 21:58:37,733][17577] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-12-26 21:58:37,733][17574] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-12-26 21:58:37,743][17559] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-12-26 21:58:37,743][17558] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-12-26 21:58:37,746][17561] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-12-26 21:58:37,749][17575] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-12-26 21:58:37,750][17560] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-12-26 21:58:37,758][17562] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-12-26 21:58:38,192][17559] Decorrelating experience for 0 frames... | |
| [2023-12-26 21:58:38,195][17561] Decorrelating experience for 0 frames... | |
| [2023-12-26 21:58:38,200][17577] Decorrelating experience for 0 frames... | |
| [2023-12-26 21:58:38,200][17575] Decorrelating experience for 0 frames... | |
| [2023-12-26 21:58:38,200][17560] Decorrelating experience for 0 frames... | |
| [2023-12-26 21:58:38,201][17574] Decorrelating experience for 0 frames... | |
| [2023-12-26 21:58:38,460][17561] Decorrelating experience for 32 frames... | |
| [2023-12-26 21:58:38,464][17574] Decorrelating experience for 32 frames... | |
| [2023-12-26 21:58:38,480][17577] Decorrelating experience for 32 frames... | |
| [2023-12-26 21:58:38,521][17558] Decorrelating experience for 0 frames... | |
| [2023-12-26 21:58:38,528][17559] Decorrelating experience for 32 frames... | |
| [2023-12-26 21:58:38,574][17562] Decorrelating experience for 0 frames... | |
| [2023-12-26 21:58:38,583][17560] Decorrelating experience for 32 frames... | |
| [2023-12-26 21:58:38,761][17575] Decorrelating experience for 32 frames... | |
| [2023-12-26 21:58:38,785][17574] Decorrelating experience for 64 frames... | |
| [2023-12-26 21:58:38,796][17558] Decorrelating experience for 32 frames... | |
| [2023-12-26 21:58:38,806][17561] Decorrelating experience for 64 frames... | |
| [2023-12-26 21:58:39,012][17562] Decorrelating experience for 32 frames... | |
| [2023-12-26 21:58:39,054][17575] Decorrelating experience for 64 frames... | |
| [2023-12-26 21:58:39,065][17577] Decorrelating experience for 64 frames... | |
| [2023-12-26 21:58:39,094][17561] Decorrelating experience for 96 frames... | |
| [2023-12-26 21:58:39,107][17558] Decorrelating experience for 64 frames... | |
| [2023-12-26 21:58:39,134][17486] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 1327104. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2023-12-26 21:58:39,277][17574] Decorrelating experience for 96 frames... | |
| [2023-12-26 21:58:39,327][17562] Decorrelating experience for 64 frames... | |
| [2023-12-26 21:58:39,333][17575] Decorrelating experience for 96 frames... | |
| [2023-12-26 21:58:39,378][17577] Decorrelating experience for 96 frames... | |
| [2023-12-26 21:58:39,542][17560] Decorrelating experience for 64 frames... | |
| [2023-12-26 21:58:39,588][17562] Decorrelating experience for 96 frames... | |
| [2023-12-26 21:58:39,772][17559] Decorrelating experience for 64 frames... | |
| [2023-12-26 21:58:40,091][17560] Decorrelating experience for 96 frames... | |
| [2023-12-26 21:58:40,139][17559] Decorrelating experience for 96 frames... | |
| [2023-12-26 21:58:40,399][17532] Signal inference workers to stop experience collection... | |
| [2023-12-26 21:58:40,404][17557] InferenceWorker_p0-w0: stopping experience collection | |
| [2023-12-26 21:58:40,435][17558] Decorrelating experience for 96 frames... | |
| [2023-12-26 21:58:41,834][17532] Signal inference workers to resume experience collection... | |
| [2023-12-26 21:58:41,835][17557] InferenceWorker_p0-w0: resuming experience collection | |
| [2023-12-26 21:58:43,658][17557] Updated weights for policy 0, policy_version 334 (0.0136) | |
| [2023-12-26 21:58:44,134][17486] Fps is (10 sec: 9830.6, 60 sec: 9830.6, 300 sec: 9830.6). Total num frames: 1376256. Throughput: 0: 2267.6. Samples: 11338. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) | |
| [2023-12-26 21:58:44,134][17486] Avg episode reward: [(0, '5.931')] | |
| [2023-12-26 21:58:44,135][17532] Saving new best policy, reward=5.931! | |
| [2023-12-26 21:58:45,587][17557] Updated weights for policy 0, policy_version 344 (0.0011) | |
| [2023-12-26 21:58:47,541][17557] Updated weights for policy 0, policy_version 354 (0.0010) | |
| [2023-12-26 21:58:49,134][17486] Fps is (10 sec: 15564.9, 60 sec: 15564.9, 300 sec: 15564.9). Total num frames: 1482752. Throughput: 0: 2727.0. Samples: 27270. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-12-26 21:58:49,134][17486] Avg episode reward: [(0, '6.774')] | |
| [2023-12-26 21:58:49,138][17532] Saving new best policy, reward=6.774! | |
| [2023-12-26 21:58:49,451][17557] Updated weights for policy 0, policy_version 364 (0.0010) | |
| [2023-12-26 21:58:51,427][17557] Updated weights for policy 0, policy_version 374 (0.0011) | |
| [2023-12-26 21:58:52,029][17486] Heartbeat connected on Batcher_0 | |
| [2023-12-26 21:58:52,031][17486] Heartbeat connected on LearnerWorker_p0 | |
| [2023-12-26 21:58:52,039][17486] Heartbeat connected on RolloutWorker_w0 | |
| [2023-12-26 21:58:52,039][17486] Heartbeat connected on InferenceWorker_p0-w0 | |
| [2023-12-26 21:58:52,042][17486] Heartbeat connected on RolloutWorker_w1 | |
| [2023-12-26 21:58:52,044][17486] Heartbeat connected on RolloutWorker_w2 | |
| [2023-12-26 21:58:52,048][17486] Heartbeat connected on RolloutWorker_w4 | |
| [2023-12-26 21:58:52,050][17486] Heartbeat connected on RolloutWorker_w5 | |
| [2023-12-26 21:58:52,050][17486] Heartbeat connected on RolloutWorker_w3 | |
| [2023-12-26 21:58:52,052][17486] Heartbeat connected on RolloutWorker_w6 | |
| [2023-12-26 21:58:52,055][17486] Heartbeat connected on RolloutWorker_w7 | |
| [2023-12-26 21:58:53,331][17557] Updated weights for policy 0, policy_version 384 (0.0010) | |
| [2023-12-26 21:58:54,134][17486] Fps is (10 sec: 21299.2, 60 sec: 17476.4, 300 sec: 17476.4). Total num frames: 1589248. Throughput: 0: 3927.0. Samples: 58904. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-12-26 21:58:54,134][17486] Avg episode reward: [(0, '6.314')] | |
| [2023-12-26 21:58:55,263][17557] Updated weights for policy 0, policy_version 394 (0.0011) | |
| [2023-12-26 21:58:57,183][17557] Updated weights for policy 0, policy_version 404 (0.0010) | |
| [2023-12-26 21:58:59,114][17557] Updated weights for policy 0, policy_version 414 (0.0010) | |
| [2023-12-26 21:58:59,134][17486] Fps is (10 sec: 21299.2, 60 sec: 18432.1, 300 sec: 18432.1). Total num frames: 1695744. Throughput: 0: 4545.4. Samples: 90908. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-12-26 21:58:59,134][17486] Avg episode reward: [(0, '7.559')] | |
| [2023-12-26 21:58:59,138][17532] Saving new best policy, reward=7.559! | |
| [2023-12-26 21:59:01,051][17557] Updated weights for policy 0, policy_version 424 (0.0011) | |
| [2023-12-26 21:59:03,046][17557] Updated weights for policy 0, policy_version 434 (0.0010) | |
| [2023-12-26 21:59:04,134][17486] Fps is (10 sec: 20889.5, 60 sec: 18841.7, 300 sec: 18841.7). Total num frames: 1798144. Throughput: 0: 4269.0. Samples: 106724. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-12-26 21:59:04,134][17486] Avg episode reward: [(0, '10.002')] | |
| [2023-12-26 21:59:04,135][17532] Saving new best policy, reward=10.002! | |
| [2023-12-26 21:59:05,018][17557] Updated weights for policy 0, policy_version 444 (0.0011) | |
| [2023-12-26 21:59:06,911][17557] Updated weights for policy 0, policy_version 454 (0.0010) | |
| [2023-12-26 21:59:08,831][17557] Updated weights for policy 0, policy_version 464 (0.0011) | |
| [2023-12-26 21:59:09,134][17486] Fps is (10 sec: 20889.6, 60 sec: 19251.2, 300 sec: 19251.2). Total num frames: 1904640. Throughput: 0: 4605.6. Samples: 138168. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-12-26 21:59:09,134][17486] Avg episode reward: [(0, '9.454')] | |
| [2023-12-26 21:59:10,824][17557] Updated weights for policy 0, policy_version 474 (0.0010) | |
| [2023-12-26 21:59:12,731][17557] Updated weights for policy 0, policy_version 484 (0.0011) | |
| [2023-12-26 21:59:14,134][17486] Fps is (10 sec: 21299.1, 60 sec: 19543.8, 300 sec: 19543.8). Total num frames: 2011136. Throughput: 0: 4852.2. Samples: 169826. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-12-26 21:59:14,134][17486] Avg episode reward: [(0, '11.666')] | |
| [2023-12-26 21:59:14,135][17532] Saving new best policy, reward=11.666! | |
| [2023-12-26 21:59:14,692][17557] Updated weights for policy 0, policy_version 494 (0.0011) | |
| [2023-12-26 21:59:16,626][17557] Updated weights for policy 0, policy_version 504 (0.0010) | |
| [2023-12-26 21:59:18,610][17557] Updated weights for policy 0, policy_version 514 (0.0010) | |
| [2023-12-26 21:59:19,134][17486] Fps is (10 sec: 20889.6, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 2113536. Throughput: 0: 4639.2. Samples: 185568. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-12-26 21:59:19,135][17486] Avg episode reward: [(0, '12.029')] | |
| [2023-12-26 21:59:19,139][17532] Saving new best policy, reward=12.029! | |
| [2023-12-26 21:59:20,633][17557] Updated weights for policy 0, policy_version 524 (0.0011) | |
| [2023-12-26 21:59:22,690][17557] Updated weights for policy 0, policy_version 534 (0.0011) | |
| [2023-12-26 21:59:24,134][17486] Fps is (10 sec: 20480.2, 60 sec: 19751.9, 300 sec: 19751.9). Total num frames: 2215936. Throughput: 0: 4797.9. Samples: 215906. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-12-26 21:59:24,134][17486] Avg episode reward: [(0, '12.973')] | |
| [2023-12-26 21:59:24,135][17532] Saving new best policy, reward=12.973! | |
| [2023-12-26 21:59:24,656][17557] Updated weights for policy 0, policy_version 544 (0.0010) | |
| [2023-12-26 21:59:26,569][17557] Updated weights for policy 0, policy_version 554 (0.0010) | |
| [2023-12-26 21:59:28,518][17557] Updated weights for policy 0, policy_version 564 (0.0011) | |
| [2023-12-26 21:59:29,134][17486] Fps is (10 sec: 20889.6, 60 sec: 19906.6, 300 sec: 19906.6). Total num frames: 2322432. Throughput: 0: 5247.9. Samples: 247496. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-12-26 21:59:29,134][17486] Avg episode reward: [(0, '15.120')] | |
| [2023-12-26 21:59:29,138][17532] Saving new best policy, reward=15.120! | |
| [2023-12-26 21:59:30,494][17557] Updated weights for policy 0, policy_version 574 (0.0011) | |
| [2023-12-26 21:59:32,524][17557] Updated weights for policy 0, policy_version 584 (0.0011) | |
| [2023-12-26 21:59:34,134][17486] Fps is (10 sec: 20889.4, 60 sec: 19958.7, 300 sec: 19958.7). Total num frames: 2424832. Throughput: 0: 5232.9. Samples: 262752. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) | |
| [2023-12-26 21:59:34,134][17486] Avg episode reward: [(0, '16.742')] | |
| [2023-12-26 21:59:34,135][17532] Saving new best policy, reward=16.742! | |
| [2023-12-26 21:59:34,418][17557] Updated weights for policy 0, policy_version 594 (0.0011) | |
| [2023-12-26 21:59:36,372][17557] Updated weights for policy 0, policy_version 604 (0.0011) | |
| [2023-12-26 21:59:38,472][17557] Updated weights for policy 0, policy_version 614 (0.0011) | |
| [2023-12-26 21:59:39,134][17486] Fps is (10 sec: 20480.0, 60 sec: 20002.1, 300 sec: 20002.1). Total num frames: 2527232. Throughput: 0: 5223.8. Samples: 293974. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-12-26 21:59:39,135][17486] Avg episode reward: [(0, '20.766')] | |
| [2023-12-26 21:59:39,138][17532] Saving new best policy, reward=20.766! | |
| [2023-12-26 21:59:40,543][17557] Updated weights for policy 0, policy_version 624 (0.0011) | |
| [2023-12-26 21:59:42,475][17557] Updated weights for policy 0, policy_version 634 (0.0010) | |
| [2023-12-26 21:59:44,134][17486] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20038.9). Total num frames: 2629632. Throughput: 0: 5192.6. Samples: 324576. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-12-26 21:59:44,134][17486] Avg episode reward: [(0, '18.192')] | |
| [2023-12-26 21:59:44,473][17557] Updated weights for policy 0, policy_version 644 (0.0010) | |
| [2023-12-26 21:59:46,416][17557] Updated weights for policy 0, policy_version 654 (0.0010) | |
| [2023-12-26 21:59:48,369][17557] Updated weights for policy 0, policy_version 664 (0.0011) | |
| [2023-12-26 21:59:49,134][17486] Fps is (10 sec: 20479.9, 60 sec: 20821.3, 300 sec: 20070.4). Total num frames: 2732032. Throughput: 0: 5188.5. Samples: 340208. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) | |
| [2023-12-26 21:59:49,135][17486] Avg episode reward: [(0, '20.043')] | |
| [2023-12-26 21:59:50,345][17557] Updated weights for policy 0, policy_version 674 (0.0011) | |
| [2023-12-26 21:59:52,371][17557] Updated weights for policy 0, policy_version 684 (0.0011) | |
| [2023-12-26 21:59:54,134][17486] Fps is (10 sec: 20889.5, 60 sec: 20821.3, 300 sec: 20152.3). Total num frames: 2838528. Throughput: 0: 5181.1. Samples: 371318. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-12-26 21:59:54,134][17486] Avg episode reward: [(0, '20.133')] | |
| [2023-12-26 21:59:54,305][17557] Updated weights for policy 0, policy_version 694 (0.0010) | |
| [2023-12-26 21:59:56,453][17557] Updated weights for policy 0, policy_version 704 (0.0012) | |
| [2023-12-26 21:59:58,508][17557] Updated weights for policy 0, policy_version 714 (0.0011) | |
| [2023-12-26 21:59:59,134][17486] Fps is (10 sec: 20070.6, 60 sec: 20616.5, 300 sec: 20070.4). Total num frames: 2932736. Throughput: 0: 5142.0. Samples: 401218. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-12-26 21:59:59,134][17486] Avg episode reward: [(0, '20.781')] | |
| [2023-12-26 21:59:59,139][17532] Saving new best policy, reward=20.781! | |
| [2023-12-26 22:00:00,651][17557] Updated weights for policy 0, policy_version 724 (0.0011) | |
| [2023-12-26 22:00:02,623][17557] Updated weights for policy 0, policy_version 734 (0.0011) | |
| [2023-12-26 22:00:04,134][17486] Fps is (10 sec: 19660.9, 60 sec: 20616.5, 300 sec: 20094.5). Total num frames: 3035136. Throughput: 0: 5117.3. Samples: 415844. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) | |
| [2023-12-26 22:00:04,134][17486] Avg episode reward: [(0, '20.092')] | |
| [2023-12-26 22:00:04,652][17557] Updated weights for policy 0, policy_version 744 (0.0010) | |
| [2023-12-26 22:00:06,688][17557] Updated weights for policy 0, policy_version 754 (0.0011) | |
| [2023-12-26 22:00:08,642][17557] Updated weights for policy 0, policy_version 764 (0.0011) | |
| [2023-12-26 22:00:09,134][17486] Fps is (10 sec: 20479.9, 60 sec: 20548.3, 300 sec: 20115.9). Total num frames: 3137536. Throughput: 0: 5123.9. Samples: 446480. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) | |
| [2023-12-26 22:00:09,134][17486] Avg episode reward: [(0, '22.233')] | |
| [2023-12-26 22:00:09,139][17532] Saving new best policy, reward=22.233! | |
| [2023-12-26 22:00:10,676][17557] Updated weights for policy 0, policy_version 774 (0.0011) | |
| [2023-12-26 22:00:12,706][17557] Updated weights for policy 0, policy_version 784 (0.0011) | |
| [2023-12-26 22:00:14,134][17486] Fps is (10 sec: 20070.4, 60 sec: 20411.7, 300 sec: 20092.0). Total num frames: 3235840. Throughput: 0: 5087.8. Samples: 476448. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-12-26 22:00:14,134][17486] Avg episode reward: [(0, '20.157')] | |
| [2023-12-26 22:00:14,748][17557] Updated weights for policy 0, policy_version 794 (0.0011) | |
| [2023-12-26 22:00:16,655][17557] Updated weights for policy 0, policy_version 804 (0.0010) | |
| [2023-12-26 22:00:18,543][17557] Updated weights for policy 0, policy_version 814 (0.0010) | |
| [2023-12-26 22:00:19,134][17486] Fps is (10 sec: 20889.6, 60 sec: 20548.3, 300 sec: 20193.3). Total num frames: 3346432. Throughput: 0: 5109.5. Samples: 492678. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-12-26 22:00:19,134][17486] Avg episode reward: [(0, '26.068')] | |
| [2023-12-26 22:00:19,138][17532] Saving new best policy, reward=26.068! | |
| [2023-12-26 22:00:20,502][17557] Updated weights for policy 0, policy_version 824 (0.0010) | |
| [2023-12-26 22:00:22,403][17557] Updated weights for policy 0, policy_version 834 (0.0011) | |
| [2023-12-26 22:00:24,134][17486] Fps is (10 sec: 21299.1, 60 sec: 20548.2, 300 sec: 20206.9). Total num frames: 3448832. Throughput: 0: 5129.2. Samples: 524786. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-12-26 22:00:24,134][17486] Avg episode reward: [(0, '22.497')] | |
| [2023-12-26 22:00:24,437][17557] Updated weights for policy 0, policy_version 844 (0.0011) | |
| [2023-12-26 22:00:26,531][17557] Updated weights for policy 0, policy_version 854 (0.0011) | |
| [2023-12-26 22:00:28,616][17557] Updated weights for policy 0, policy_version 864 (0.0011) | |
| [2023-12-26 22:00:29,134][17486] Fps is (10 sec: 20070.5, 60 sec: 20411.7, 300 sec: 20182.1). Total num frames: 3547136. Throughput: 0: 5104.4. Samples: 554274. Policy #0 lag: (min: 0.0, avg: 0.8, max: 1.0) | |
| [2023-12-26 22:00:29,134][17486] Avg episode reward: [(0, '24.875')] | |
| [2023-12-26 22:00:29,139][17532] Saving /home/cybertron/Desktop/rl_units/train_dir/default_experiment/checkpoint_p0/checkpoint_000000866_3547136.pth... | |
| [2023-12-26 22:00:30,612][17557] Updated weights for policy 0, policy_version 874 (0.0010) | |
| [2023-12-26 22:00:32,646][17557] Updated weights for policy 0, policy_version 884 (0.0010) | |
| [2023-12-26 22:00:34,134][17486] Fps is (10 sec: 19660.9, 60 sec: 20343.5, 300 sec: 20159.5). Total num frames: 3645440. Throughput: 0: 5095.3. Samples: 569496. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-12-26 22:00:34,134][17486] Avg episode reward: [(0, '27.716')] | |
| [2023-12-26 22:00:34,135][17532] Saving new best policy, reward=27.716! | |
| [2023-12-26 22:00:34,853][17557] Updated weights for policy 0, policy_version 894 (0.0011) | |
| [2023-12-26 22:00:36,921][17557] Updated weights for policy 0, policy_version 904 (0.0011) | |
| [2023-12-26 22:00:39,055][17557] Updated weights for policy 0, policy_version 914 (0.0011) | |
| [2023-12-26 22:00:39,134][17486] Fps is (10 sec: 19660.7, 60 sec: 20275.2, 300 sec: 20138.7). Total num frames: 3743744. Throughput: 0: 5053.8. Samples: 598738. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-12-26 22:00:39,135][17486] Avg episode reward: [(0, '24.333')] | |
| [2023-12-26 22:00:41,206][17557] Updated weights for policy 0, policy_version 924 (0.0012) | |
| [2023-12-26 22:00:43,272][17557] Updated weights for policy 0, policy_version 934 (0.0012) | |
| [2023-12-26 22:00:44,134][17486] Fps is (10 sec: 19660.8, 60 sec: 20206.9, 300 sec: 20119.6). Total num frames: 3842048. Throughput: 0: 5035.3. Samples: 627808. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-12-26 22:00:44,134][17486] Avg episode reward: [(0, '23.879')] | |
| [2023-12-26 22:00:45,205][17557] Updated weights for policy 0, policy_version 944 (0.0010) | |
| [2023-12-26 22:00:47,134][17557] Updated weights for policy 0, policy_version 954 (0.0010) | |
| [2023-12-26 22:00:49,093][17557] Updated weights for policy 0, policy_version 964 (0.0010) | |
| [2023-12-26 22:00:49,134][17486] Fps is (10 sec: 20480.2, 60 sec: 20275.2, 300 sec: 20164.9). Total num frames: 3948544. Throughput: 0: 5064.1. Samples: 643728. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-12-26 22:00:49,134][17486] Avg episode reward: [(0, '25.699')] | |
| [2023-12-26 22:00:51,092][17557] Updated weights for policy 0, policy_version 974 (0.0011) | |
| [2023-12-26 22:00:51,906][17486] Component Batcher_0 stopped! | |
| [2023-12-26 22:00:51,906][17532] Stopping Batcher_0... | |
| [2023-12-26 22:00:51,907][17532] Loop batcher_evt_loop terminating... | |
| [2023-12-26 22:00:51,907][17532] Saving /home/cybertron/Desktop/rl_units/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... | |
| [2023-12-26 22:00:51,915][17558] Stopping RolloutWorker_w2... | |
| [2023-12-26 22:00:51,916][17486] Component RolloutWorker_w2 stopped! | |
| [2023-12-26 22:00:51,916][17486] Component RolloutWorker_w5 stopped! | |
| [2023-12-26 22:00:51,916][17575] Stopping RolloutWorker_w5... | |
| [2023-12-26 22:00:51,916][17558] Loop rollout_proc2_evt_loop terminating... | |
| [2023-12-26 22:00:51,916][17575] Loop rollout_proc5_evt_loop terminating... | |
| [2023-12-26 22:00:51,916][17574] Stopping RolloutWorker_w7... | |
| [2023-12-26 22:00:51,916][17486] Component RolloutWorker_w7 stopped! | |
| [2023-12-26 22:00:51,916][17560] Stopping RolloutWorker_w0... | |
| [2023-12-26 22:00:51,916][17561] Stopping RolloutWorker_w3... | |
| [2023-12-26 22:00:51,917][17486] Component RolloutWorker_w0 stopped! | |
| [2023-12-26 22:00:51,917][17574] Loop rollout_proc7_evt_loop terminating... | |
| [2023-12-26 22:00:51,917][17486] Component RolloutWorker_w3 stopped! | |
| [2023-12-26 22:00:51,917][17560] Loop rollout_proc0_evt_loop terminating... | |
| [2023-12-26 22:00:51,917][17561] Loop rollout_proc3_evt_loop terminating... | |
| [2023-12-26 22:00:51,917][17486] Component RolloutWorker_w4 stopped! | |
| [2023-12-26 22:00:51,917][17562] Stopping RolloutWorker_w4... | |
| [2023-12-26 22:00:51,917][17562] Loop rollout_proc4_evt_loop terminating... | |
| [2023-12-26 22:00:51,920][17577] Stopping RolloutWorker_w6... | |
| [2023-12-26 22:00:51,920][17486] Component RolloutWorker_w6 stopped! | |
| [2023-12-26 22:00:51,920][17577] Loop rollout_proc6_evt_loop terminating... | |
| [2023-12-26 22:00:51,922][17559] Stopping RolloutWorker_w1... | |
| [2023-12-26 22:00:51,922][17486] Component RolloutWorker_w1 stopped! | |
| [2023-12-26 22:00:51,922][17559] Loop rollout_proc1_evt_loop terminating... | |
| [2023-12-26 22:00:51,933][17557] Weights refcount: 2 0 | |
| [2023-12-26 22:00:51,935][17557] Stopping InferenceWorker_p0-w0... | |
| [2023-12-26 22:00:51,935][17486] Component InferenceWorker_p0-w0 stopped! | |
| [2023-12-26 22:00:51,935][17557] Loop inference_proc0-0_evt_loop terminating... | |
| [2023-12-26 22:00:51,986][17532] Removing /home/cybertron/Desktop/rl_units/train_dir/default_experiment/checkpoint_p0/checkpoint_000000324_1327104.pth | |
| [2023-12-26 22:00:51,994][17532] Saving /home/cybertron/Desktop/rl_units/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... | |
| [2023-12-26 22:00:52,201][17532] Stopping LearnerWorker_p0... | |
| [2023-12-26 22:00:52,201][17486] Component LearnerWorker_p0 stopped! | |
| [2023-12-26 22:00:52,201][17532] Loop learner_proc0_evt_loop terminating... | |
| [2023-12-26 22:00:52,201][17486] Waiting for process learner_proc0 to stop... | |
| [2023-12-26 22:00:53,018][17486] Waiting for process inference_proc0-0 to join... | |
| [2023-12-26 22:00:53,018][17486] Waiting for process rollout_proc0 to join... | |
| [2023-12-26 22:00:53,018][17486] Waiting for process rollout_proc1 to join... | |
| [2023-12-26 22:00:53,018][17486] Waiting for process rollout_proc2 to join... | |
| [2023-12-26 22:00:53,018][17486] Waiting for process rollout_proc3 to join... | |
| [2023-12-26 22:00:53,018][17486] Waiting for process rollout_proc4 to join... | |
| [2023-12-26 22:00:53,018][17486] Waiting for process rollout_proc5 to join... | |
| [2023-12-26 22:00:53,019][17486] Waiting for process rollout_proc6 to join... | |
| [2023-12-26 22:00:53,019][17486] Waiting for process rollout_proc7 to join... | |
| [2023-12-26 22:00:53,019][17486] Batcher 0 profile tree view: | |
| batching: 7.5571, releasing_batches: 0.0129 | |
| [2023-12-26 22:00:53,019][17486] InferenceWorker_p0-w0 profile tree view: | |
| wait_policy: 0.0001 | |
| wait_policy_total: 3.1212 | |
| update_model: 2.0881 | |
| weight_update: 0.0012 | |
| one_step: 0.0044 | |
| handle_policy_step: 121.1792 | |
| deserialize: 5.3333, stack: 0.7067, obs_to_device_normalize: 26.5060, forward: 60.5218, send_messages: 8.9535 | |
| prepare_outputs: 13.9331 | |
| to_cpu: 7.7252 | |
| [2023-12-26 22:00:53,019][17486] Learner 0 profile tree view: | |
| misc: 0.0024, prepare_batch: 4.9472 | |
| train: 16.7261 | |
| epoch_init: 0.0027, minibatch_init: 0.0027, losses_postprocess: 0.1095, kl_divergence: 0.1305, after_optimizer: 0.3384 | |
| calculate_losses: 5.4462 | |
| losses_init: 0.0018, forward_head: 0.3834, bptt_initial: 3.5046, tail: 0.2963, advantages_returns: 0.0809, losses: 0.5075 | |
| bptt: 0.5792 | |
| bptt_forward_core: 0.5490 | |
| update: 10.5090 | |
| clip: 0.4330 | |
| [2023-12-26 22:00:53,019][17486] RolloutWorker_w0 profile tree view: | |
| wait_for_trajectories: 0.0873, enqueue_policy_requests: 5.7973, env_step: 72.4011, overhead: 4.2257, complete_rollouts: 0.1368 | |
| save_policy_outputs: 6.3596 | |
| split_output_tensors: 2.1919 | |
| [2023-12-26 22:00:53,019][17486] RolloutWorker_w7 profile tree view: | |
| wait_for_trajectories: 0.0862, enqueue_policy_requests: 5.8328, env_step: 72.5999, overhead: 4.2315, complete_rollouts: 0.1394 | |
| save_policy_outputs: 6.4396 | |
| split_output_tensors: 2.2297 | |
| [2023-12-26 22:00:53,020][17486] Loop Runner_EvtLoop terminating... | |
| [2023-12-26 22:00:53,020][17486] Runner profile tree view: | |
| main_loop: 140.9646 | |
| [2023-12-26 22:00:53,020][17486] Collected {0: 4005888}, FPS: 19003.2 | |
| [2023-12-26 22:00:53,115][17486] Loading existing experiment configuration from /home/cybertron/Desktop/rl_units/train_dir/default_experiment/config.json | |
| [2023-12-26 22:00:53,115][17486] Overriding arg 'num_workers' with value 1 passed from command line | |
| [2023-12-26 22:00:53,115][17486] Adding new argument 'no_render'=True that is not in the saved config file! | |
| [2023-12-26 22:00:53,115][17486] Adding new argument 'save_video'=True that is not in the saved config file! | |
| [2023-12-26 22:00:53,115][17486] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! | |
| [2023-12-26 22:00:53,115][17486] Adding new argument 'video_name'=None that is not in the saved config file! | |
| [2023-12-26 22:00:53,115][17486] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! | |
| [2023-12-26 22:00:53,115][17486] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! | |
| [2023-12-26 22:00:53,115][17486] Adding new argument 'push_to_hub'=True that is not in the saved config file! | |
| [2023-12-26 22:00:53,116][17486] Adding new argument 'hf_repository'='soonchang/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! | |
| [2023-12-26 22:00:53,116][17486] Adding new argument 'policy_index'=0 that is not in the saved config file! | |
| [2023-12-26 22:00:53,116][17486] Adding new argument 'eval_deterministic'=False that is not in the saved config file! | |
| [2023-12-26 22:00:53,116][17486] Adding new argument 'train_script'=None that is not in the saved config file! | |
| [2023-12-26 22:00:53,116][17486] Adding new argument 'enjoy_script'=None that is not in the saved config file! | |
| [2023-12-26 22:00:53,116][17486] Using frameskip 1 and render_action_repeat=4 for evaluation | |
| [2023-12-26 22:00:53,132][17486] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-12-26 22:00:53,133][17486] RunningMeanStd input shape: (3, 72, 128) | |
| [2023-12-26 22:00:53,134][17486] RunningMeanStd input shape: (1,) | |
| [2023-12-26 22:00:53,183][17486] ConvEncoder: input_channels=3 | |
| [2023-12-26 22:00:53,257][17486] Conv encoder output size: 512 | |
| [2023-12-26 22:00:53,257][17486] Policy head output size: 512 | |
| [2023-12-26 22:00:54,625][17486] Loading state from checkpoint /home/cybertron/Desktop/rl_units/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... | |
| [2023-12-26 22:00:55,641][17486] Num frames 100... | |
| [2023-12-26 22:00:55,719][17486] Num frames 200... | |
| [2023-12-26 22:00:55,800][17486] Num frames 300... | |
| [2023-12-26 22:00:55,879][17486] Num frames 400... | |
| [2023-12-26 22:00:55,962][17486] Num frames 500... | |
| [2023-12-26 22:00:56,048][17486] Num frames 600... | |
| [2023-12-26 22:00:56,129][17486] Num frames 700... | |
| [2023-12-26 22:00:56,209][17486] Num frames 800... | |
| [2023-12-26 22:00:56,290][17486] Num frames 900... | |
| [2023-12-26 22:00:56,407][17486] Avg episode rewards: #0: 22.800, true rewards: #0: 9.800 | |
| [2023-12-26 22:00:56,407][17486] Avg episode reward: 22.800, avg true_objective: 9.800 | |
| [2023-12-26 22:00:56,437][17486] Num frames 1000... | |
| [2023-12-26 22:00:56,525][17486] Num frames 1100... | |
| [2023-12-26 22:00:56,603][17486] Num frames 1200... | |
| [2023-12-26 22:00:56,680][17486] Num frames 1300... | |
| [2023-12-26 22:00:56,758][17486] Num frames 1400... | |
| [2023-12-26 22:00:56,839][17486] Num frames 1500... | |
| [2023-12-26 22:00:56,922][17486] Num frames 1600... | |
| [2023-12-26 22:00:57,007][17486] Num frames 1700... | |
| [2023-12-26 22:00:57,095][17486] Num frames 1800... | |
| [2023-12-26 22:00:57,183][17486] Num frames 1900... | |
| [2023-12-26 22:00:57,269][17486] Num frames 2000... | |
| [2023-12-26 22:00:57,353][17486] Num frames 2100... | |
| [2023-12-26 22:00:57,436][17486] Num frames 2200... | |
| [2023-12-26 22:00:57,525][17486] Num frames 2300... | |
| [2023-12-26 22:00:57,634][17486] Num frames 2400... | |
| [2023-12-26 22:00:57,732][17486] Num frames 2500... | |
| [2023-12-26 22:00:57,854][17486] Avg episode rewards: #0: 30.400, true rewards: #0: 12.900 | |
| [2023-12-26 22:00:57,854][17486] Avg episode reward: 30.400, avg true_objective: 12.900 | |
| [2023-12-26 22:00:57,899][17486] Num frames 2600... | |
| [2023-12-26 22:00:57,998][17486] Num frames 2700... | |
| [2023-12-26 22:00:58,079][17486] Num frames 2800... | |
| [2023-12-26 22:00:58,159][17486] Num frames 2900... | |
| [2023-12-26 22:00:58,237][17486] Num frames 3000... | |
| [2023-12-26 22:00:58,319][17486] Num frames 3100... | |
| [2023-12-26 22:00:58,403][17486] Num frames 3200... | |
| [2023-12-26 22:00:58,484][17486] Num frames 3300... | |
| [2023-12-26 22:00:58,549][17486] Avg episode rewards: #0: 24.720, true rewards: #0: 11.053 | |
| [2023-12-26 22:00:58,549][17486] Avg episode reward: 24.720, avg true_objective: 11.053 | |
| [2023-12-26 22:00:58,635][17486] Num frames 3400... | |
| [2023-12-26 22:00:58,717][17486] Num frames 3500... | |
| [2023-12-26 22:00:58,797][17486] Num frames 3600... | |
| [2023-12-26 22:00:58,879][17486] Num frames 3700... | |
| [2023-12-26 22:00:58,959][17486] Num frames 3800... | |
| [2023-12-26 22:00:59,045][17486] Num frames 3900... | |
| [2023-12-26 22:00:59,126][17486] Num frames 4000... | |
| [2023-12-26 22:00:59,209][17486] Num frames 4100... | |
| [2023-12-26 22:00:59,290][17486] Num frames 4200... | |
| [2023-12-26 22:00:59,380][17486] Avg episode rewards: #0: 22.610, true rewards: #0: 10.610 | |
| [2023-12-26 22:00:59,380][17486] Avg episode reward: 22.610, avg true_objective: 10.610 | |
| [2023-12-26 22:00:59,449][17486] Num frames 4300... | |
| [2023-12-26 22:00:59,531][17486] Num frames 4400... | |
| [2023-12-26 22:00:59,612][17486] Num frames 4500... | |
| [2023-12-26 22:00:59,694][17486] Num frames 4600... | |
| [2023-12-26 22:00:59,775][17486] Num frames 4700... | |
| [2023-12-26 22:00:59,857][17486] Num frames 4800... | |
| [2023-12-26 22:00:59,947][17486] Num frames 4900... | |
| [2023-12-26 22:01:00,030][17486] Num frames 5000... | |
| [2023-12-26 22:01:00,112][17486] Num frames 5100... | |
| [2023-12-26 22:01:00,193][17486] Num frames 5200... | |
| [2023-12-26 22:01:00,276][17486] Num frames 5300... | |
| [2023-12-26 22:01:00,372][17486] Num frames 5400... | |
| [2023-12-26 22:01:00,465][17486] Num frames 5500... | |
| [2023-12-26 22:01:00,548][17486] Num frames 5600... | |
| [2023-12-26 22:01:00,631][17486] Num frames 5700... | |
| [2023-12-26 22:01:00,759][17486] Avg episode rewards: #0: 25.780, true rewards: #0: 11.580 | |
| [2023-12-26 22:01:00,760][17486] Avg episode reward: 25.780, avg true_objective: 11.580 | |
| [2023-12-26 22:01:00,773][17486] Num frames 5800... | |
| [2023-12-26 22:01:00,869][17486] Num frames 5900... | |
| [2023-12-26 22:01:00,953][17486] Num frames 6000... | |
| [2023-12-26 22:01:01,033][17486] Num frames 6100... | |
| [2023-12-26 22:01:01,111][17486] Num frames 6200... | |
| [2023-12-26 22:01:01,191][17486] Num frames 6300... | |
| [2023-12-26 22:01:01,274][17486] Num frames 6400... | |
| [2023-12-26 22:01:01,356][17486] Num frames 6500... | |
| [2023-12-26 22:01:01,440][17486] Num frames 6600... | |
| [2023-12-26 22:01:01,523][17486] Num frames 6700... | |
| [2023-12-26 22:01:01,603][17486] Num frames 6800... | |
| [2023-12-26 22:01:01,685][17486] Num frames 6900... | |
| [2023-12-26 22:01:01,799][17486] Avg episode rewards: #0: 25.457, true rewards: #0: 11.623 | |
| [2023-12-26 22:01:01,799][17486] Avg episode reward: 25.457, avg true_objective: 11.623 | |
| [2023-12-26 22:01:01,840][17486] Num frames 7000... | |
| [2023-12-26 22:01:01,930][17486] Num frames 7100... | |
| [2023-12-26 22:01:02,016][17486] Num frames 7200... | |
| [2023-12-26 22:01:02,098][17486] Num frames 7300... | |
| [2023-12-26 22:01:02,180][17486] Num frames 7400... | |
| [2023-12-26 22:01:02,277][17486] Avg episode rewards: #0: 23.220, true rewards: #0: 10.649 | |
| [2023-12-26 22:01:02,278][17486] Avg episode reward: 23.220, avg true_objective: 10.649 | |
| [2023-12-26 22:01:02,335][17486] Num frames 7500... | |
| [2023-12-26 22:01:02,418][17486] Num frames 7600... | |
| [2023-12-26 22:01:02,501][17486] Num frames 7700... | |
| [2023-12-26 22:01:02,582][17486] Num frames 7800... | |
| [2023-12-26 22:01:02,665][17486] Num frames 7900... | |
| [2023-12-26 22:01:02,752][17486] Num frames 8000... | |
| [2023-12-26 22:01:02,843][17486] Num frames 8100... | |
| [2023-12-26 22:01:02,927][17486] Num frames 8200... | |
| [2023-12-26 22:01:03,016][17486] Num frames 8300... | |
| [2023-12-26 22:01:03,107][17486] Num frames 8400... | |
| [2023-12-26 22:01:03,203][17486] Avg episode rewards: #0: 23.058, true rewards: #0: 10.557 | |
| [2023-12-26 22:01:03,204][17486] Avg episode reward: 23.058, avg true_objective: 10.557 | |
| [2023-12-26 22:01:03,286][17486] Num frames 8500... | |
| [2023-12-26 22:01:03,370][17486] Num frames 8600... | |
| [2023-12-26 22:01:03,462][17486] Num frames 8700... | |
| [2023-12-26 22:01:03,545][17486] Num frames 8800... | |
| [2023-12-26 22:01:03,637][17486] Num frames 8900... | |
| [2023-12-26 22:01:03,725][17486] Num frames 9000... | |
| [2023-12-26 22:01:03,827][17486] Num frames 9100... | |
| [2023-12-26 22:01:03,918][17486] Num frames 9200... | |
| [2023-12-26 22:01:04,008][17486] Num frames 9300... | |
| [2023-12-26 22:01:04,094][17486] Num frames 9400... | |
| [2023-12-26 22:01:04,194][17486] Avg episode rewards: #0: 22.831, true rewards: #0: 10.498 | |
| [2023-12-26 22:01:04,194][17486] Avg episode reward: 22.831, avg true_objective: 10.498 | |
| [2023-12-26 22:01:04,282][17486] Num frames 9500... | |
| [2023-12-26 22:01:04,370][17486] Num frames 9600... | |
| [2023-12-26 22:01:04,454][17486] Num frames 9700... | |
| [2023-12-26 22:01:04,535][17486] Num frames 9800... | |
| [2023-12-26 22:01:04,619][17486] Num frames 9900... | |
| [2023-12-26 22:01:04,702][17486] Num frames 10000... | |
| [2023-12-26 22:01:04,788][17486] Num frames 10100... | |
| [2023-12-26 22:01:04,872][17486] Num frames 10200... | |
| [2023-12-26 22:01:04,978][17486] Avg episode rewards: #0: 22.064, true rewards: #0: 10.264 | |
| [2023-12-26 22:01:04,978][17486] Avg episode reward: 22.064, avg true_objective: 10.264 | |
| [2023-12-26 22:01:09,429][17486] Replay video saved to /home/cybertron/Desktop/rl_units/train_dir/default_experiment/replay.mp4! | |