jtriley/StarCluster — pull request #233 by adgaudio (2013-03-15 21:14:40+00:00)
Title: Support multiple ipclusters via the --cluster-id option

Hi,

I'm submitting this PR and hoping that we might be able to incorporate it into StarCluster. In summary, it adds --cluster-id support to the IPython plugin.

The motivation behind this PR, as mentioned in Issue [229](https://github.com/jtriley/StarCluster/issues/229) are these:

* Support multiple concurrent instances of an IPython ipcluster
* Provide users with option to specify how to manage cluster instances (as identified by cluster-id)
* Maintain backwards compatibility

Why:

* Spin up new IPython ipclusters on same nodes without killing currently running tasks
* Continuous code deployment (you don't have to kill your jobs and spin up a whole new set of nodes all the time)
* Could encourage people (ie me) to attempt to generalize the SGE load balancer for (IPython and) other plugins, so that aws nodes can by dynamically allocated via the sge algorithm.

I'd love to get your feedback, either here or on the issue listed above.

Thank you!
Alex