cea-hpc/clustershell — Issue #153: subprocess creation optimizations
Author: thiell
Created: 2012-03-03 16:48:52+00:00
State: open
Labels: Lib

We have noticed that child process creation performed with {{{subprocess.Popen()}}} is very slow for our needs. We can find several references on the web showing it is a known issue, like http://bugs.python.org/issue11314, but will unlikely be resolved.

We tried also {{{subprocess32}}} from http://code.google.com/p/python-subprocess32/, but is not much faster than standard 2.6 subprocess, although it is known to be safer.

We plan to provide a lightweight private subprocess-like module to improve ClusterShell overall performance, especially on very large clusters.