Trying to train more than one mlagent environment on aws

the_gamer: <p>I am trying to train mlagents on an aws instance. I have loaded up the nvidia drivers and I’m able to run one since environment of training. On my personal computer I can run multiple. When I try to run multiple environments on the aws instance I always get a different error, but something to do with it running out of memory or unable to multiprocess. here is the error for running with 2 envs. It starts training for 1 second before crashing:</p>
<p>Traceback (most recent call last):<br>
File “C:\Users\Administrator\AppData\Local\Programs\Python\Python38\lib\multiprocessing\queues.py”, line 239, in _feed<br>
obj = _ForkingPickler.dumps(obj)<br>
File “C:\Users\Administrator\AppData\Local\Programs\Python\Python38\lib\multiprocessing\reduction.py”, line 51, in dumps<br>
cls(buf, protocol).dump(obj)<br>
MemoryError<br>
Traceback (most recent call last):<br>
File “C:\Users\Administrator\Desktop\maxkcystuff\pyawsjulvenv\lib\site-packages\mlagents\trainers\trainer_controller.py”, line 176, in start_learning<br>
n_steps = self.advance(env_manager)<br>
File “C:\Users\Administrator\Desktop\maxkcystuff\pyawsjulvenv\lib\site-packages\mlagents_envs\timers.py”, line 305, in wrapped<br>
return func(*args, **kwargs)<br>
File “C:\Users\Administrator\Desktop\maxkcystuff\pyawsjulvenv\lib\site-packages\mlagents\trainers\trainer_controller.py”, line 234, in advance<br>
new_step_infos = env_manager.get_steps()<br>
File “C:\Users\Administrator\Desktop\maxkcystuff\pyawsjulvenv\lib\site-packages\mlagents\trainers\env_manager.py”, line 124, in get_steps<br>
new_step_infos = self._step()<br>
File “C:\Users\Administrator\Desktop\maxkcystuff\pyawsjulvenv\lib\site-packages\mlagents\trainers\subprocess_env_manager.py”, line 417, in _step<br>
step: EnvironmentResponse = self.step_queue.get_nowait()<br>
File “C:\Users\Administrator\AppData\Local\Programs\Python\Python38\lib\multiprocessing\queues.py”, line 129, in get_nowait<br>
return self.get(False)<br>
File “C:\Users\Administrator\AppData\Local\Programs\Python\Python38\lib\multiprocessing\queues.py”, line 111, in get<br>
res = self._recv_bytes()<br>
File “C:\Users\Administrator\AppData\Local\Programs\Python\Python38\lib\multiprocessing\connection.py”, line 216, in recv_bytes<br>
buf = self._recv_bytes(maxlength)<br>
File “C:\Users\Administrator\AppData\Local\Programs\Python\Python38\lib\multiprocessing\connection.py”, line 318, in _recv_bytes<br>
return self._get_more_data(ov, maxsize)<br>
File “C:\Users\Administrator\AppData\Local\Programs\Python\Python38\lib\multiprocessing\connection.py”, line 340, in _get_more_data<br>
ov, err = _winapi.ReadFile(self._handle, left, overlapped=True)<br>
MemoryError</p>
<p>During handling of the above exception, another exception occurred:</p>
<p>Traceback (most recent call last):<br>
File “C:\Users\Administrator\Desktop\maxkcystuff\pyawsjulvenv\lib\site-packages\mlagents\trainers\learn.py”, line 132, in run_training<br>
tc.start_learning(env_manager)<br>
File “C:\Users\Administrator\Desktop\maxkcystuff\pyawsjulvenv\lib\site-packages\mlagents_envs\timers.py”, line 305, in wrapped<br>
return func(*args, **kwargs)<br>
File “C:\Users\Administrator\Desktop\maxkcystuff\pyawsjulvenv\lib\site-packages\mlagents\trainers\trainer_controller.py”, line 201, in start_learning<br>
self._save_models()<br>
File “C:\Users\Administrator\Desktop\maxkcystuff\pyawsjulvenv\lib\site-packages\mlagents_envs\timers.py”, line 305, in wrapped<br>
return func(*args, **kwargs)<br>
File “C:\Users\Administrator\Desktop\maxkcystuff\pyawsjulvenv\lib\site-packages\mlagents\trainers\trainer_controller.py”, line 80, in _save_models<br>
self.trainers[brain_name].save_model()<br>
File “C:\Users\Administrator\Desktop\maxkcystuff\pyawsjulvenv\lib\site-packages\mlagents\trainers\ghost\trainer.py”, line 333, in save_model<br>
self.trainer.save_model()<br>
File “C:\Users\Administrator\Desktop\maxkcystuff\pyawsjulvenv\lib\site-packages\mlagents\trainers\trainer\rl_trainer.py”, line 185, in save_model<br>
model_checkpoint = self._checkpoint()<br>
File “C:\Users\Administrator\Desktop\maxkcystuff\pyawsjulvenv\lib\site-packages\mlagents_envs\timers.py”, line 305, in wrapped<br>
return func(*args, **kwargs)<br>
File “C:\Users\Administrator\Desktop\maxkcystuff\pyawsjulvenv\lib\site-packages\mlagents\trainers\trainer\rl_trainer.py”, line 157, in <em>checkpoint<br>
export_path, auxillary_paths = self.model_saver.save_checkpoint(<br>
File “C:\Users\Administrator\Desktop\maxkcystuff\pyawsjulvenv\lib\site-packages\mlagents\trainers\model_saver\torch_model_saver.py”, line 60, in save_checkpoint<br>
self.export(checkpoint_path, behavior_name)<br>
File “C:\Users\Administrator\Desktop\maxkcystuff\pyawsjulvenv\lib\site-packages\mlagents\trainers\model_saver\torch_model_saver.py”, line 65, in export<br>
self.exporter.export_policy_model(output_filepath)<br>
File “C:\Users\Administrator\Desktop\maxkcystuff\pyawsjulvenv\lib\site-packages\mlagents\trainers\torch\model_serialization.py”, line 164, in export_policy_model<br>
torch.onnx.export(<br>
File "C:\Users\Administrator\Desktop\maxkcystuff\pyawsjulvenv\lib\site-packages\torch\onnx_<em>init</em></em>.py", line 271, in export<br>
return utils.export(model, args, f, export_params, verbose, training,<br>
File “C:\Users\Administrator\Desktop\maxkcystuff\pyawsjulvenv\lib\site-packages\torch\onnx\utils.py”, line 88, in export<br>
_export(model, args, f, export_params, verbose, training, input_names, output_names,<br>
File “C:\Users\Administrator\Desktop\maxkcystuff\pyawsjulvenv\lib\site-packages\torch\onnx\utils.py”, line 709, in _export<br>
proto, export_map = graph._export_onnx(<br>
MemoryError: bad allocation</p>
<p>During handling of the above exception, another exception occurred:</p>
<p>Traceback (most recent call last):<br>
File “C:\Users\Administrator\AppData\Local\Programs\Python\Python38\lib\runpy.py”, line 193, in _run_module_as_main<br>
return _run_code(code, main_globals, None,<br>
File “C:\Users\Administrator\AppData\Local\Programs\Python\Python38\lib\runpy.py”, line 86, in <em>run_code<br>
exec(code, run_globals)<br>
File "C:\Users\Administrator\Desktop\maxkcystuff\pyawsjulvenv\Scripts\mlagents-learn.exe_<em>main</em></em>.py", line 7, in <br>
File “C:\Users\Administrator\Desktop\maxkcystuff\pyawsjulvenv\lib\site-packages\mlagents\trainers\learn.py”, line 260, in main<br>
run_cli(parse_command_line())<br>
File “C:\Users\Administrator\Desktop\maxkcystuff\pyawsjulvenv\lib\site-packages\mlagents\trainers\learn.py”, line 256, in run_cli<br>
run_training(run_seed, options, num_areas)<br>
File “C:\Users\Administrator\Desktop\maxkcystuff\pyawsjulvenv\lib\site-packages\mlagents\trainers\learn.py”, line 134, in run_training<br>
env_manager.close()<br>
File “C:\Users\Administrator\Desktop\maxkcystuff\pyawsjulvenv\lib\site-packages\mlagents\trainers\subprocess_env_manager.py”, line 489, in close<br>
step: EnvironmentResponse = self.step_queue.get_nowait()<br>
File “C:\Users\Administrator\AppData\Local\Programs\Python\Python38\lib\multiprocessing\queues.py”, line 129, in get_nowait<br>
return self.get(False)<br>
File “C:\Users\Administrator\AppData\Local\Programs\Python\Python38\lib\multiprocessing\queues.py”, line 111, in get<br>
res = self._recv_bytes()<br>
File “C:\Users\Administrator\AppData\Local\Programs\Python\Python38\lib\multiprocessing\connection.py”, line 216, in recv_bytes<br>
buf = self._recv_bytes(maxlength)<br>
File “C:\Users\Administrator\AppData\Local\Programs\Python\Python38\lib\multiprocessing\connection.py”, line 318, in _recv_bytes<br>
return self._get_more_data(ov, maxsize)<br>
File “C:\Users\Administrator\AppData\Local\Programs\Python\Python38\lib\multiprocessing\connection.py”, line 340, in _get_more_data<br>
ov, err = _winapi.ReadFile(self._handle, left, overlapped=True)<br>
MemoryError</p>