JuliaLang/julia — pull request #3649 by amitmurthy (2013-07-08 10:00:46+00:00)
Title: Introduced ClusterManager. Externalized support for new cluster types.

Design
---------

- Base has only one ```addprocs```. Signature is ```function addprocs(instances::Union(AbstractVector, Integer);
tunnel=false, dir=JULIA_HOME, exename="./julia-release-basic", sshflags::Cmd=``, cman=nothing)```

- The first parameter can be an integer (number of procs) or array of hostnames as before.

- Additional keyword parameter ```cman```, an object of type ClusterManager

- The ClusterManager is responsible for starting worker processes. It should expose a callback
```cman.launch_cb(np::Integer, config::Dict)``` which launches the required number or procs.

- ```config``` parameter above has the following keys - :dir, :exename, :exeflags, :tunnel, :sshflags, :cman - the callback can use these fields to start the instance appropriately.

- The callback should return a Tuple consisting of the response type and an array of instance information

i.e.

```
(:io_only, [io1, io2,..]) ||
(:io_host, [(io1, host1), (io2, host2), ...]) ||
(:io_host_port, [(io1, host1, port1), (io2, host2, port2), ...]) ||
(:host_port, [(host1, port1), (host2, port2), ...]) ||
(:cmd, [cmd1, cmd2, ...])
```

- The regular local procs and ssh procs are supported via an internal ClusterManager called RegularCluster (internal to Base)

- code for SGE and Scyld have been moved into https://github.com/amitmurthy/ClusterManagers.jl

- NOTE : I HAVE NOT BEEN ABLE TO TEST SCYLD AND SGE SINCE I DO NOT HAVE ACCESS FOR THE SAME.

- I would like to transfer ownership of https://github.com/amitmurthy/ClusterManagers.jl to JuliaLang (or other regular users of such technologies), since probably more cluster managers will get added to it over time from folks who do use such clustering technologies on a regular basis.

- ```addprocs``` now returns a list of the new process ids instead of an ```:ok```

Closes #3549