apache/incubator-spark — pull request #91 by al-indigo (2013-10-21 15:59:55+00:00)
Title: Openstack support

This commit contains a script (spark-openstack) for launching Spark clusters on
Openstack cloud platform.

Much more details in documentation: docs/openstack.md

Features:
1) Scripts support Openstack Grizzly release and newer
2) Flexible network configuration for instances (inner and floating addresses)
3) Openstack access throttling limits mechanism support
4) Full docs and ready preconfigured images available

IMPORTANT NOTES AND LIMITATIONS:
1) Scripts have python-novaclient v1.1 as a dependency. You may install it with
"sudo pip install python-novaclient" to get script working (for Linux).
2) It's unknown if script works in Windows OS but it should.
3) You need special VM-image to be deployed on your openstack. We will distribute images for 0.7.3 and 0.8.0,
instructions for building your own images are in progress
4) You need to correct file "flavors.py" to correspond your Openstack instance types. We need it because
in typical Openstack installation we don't have any Ephemeral disk devices by default, so you must define
flavors yourself.
5) It's possible to launch instances in one zone only
6) You must have ephemeral disk in instance type defined; otherwise script will fail. HDFS etc are created there
7) Creating Cinder volumes is unsupported at this moment (but it will be)
8) Links to workers in spark webui are broken, it's Openstack issue (it doesn't update information about hostnames)