Updated gitignore

Remvoed config.py and properties from gitignore

Signed-off-by: Shashank Sahni <shredder12@gmail.com>
Updated gitignore to exclude *.conf files

Signed-off-by: Shashank Sahni <shredder12@gmail.com>
Introduced a new configuration format

* hadoopstack.conf stores the main configuration parameters
* cloud specific options can be added in 'clouds' folder. The file
extension should be .conf

Signed-off-by: Shashank Sahni <shredder12@gmail.com>
Introduced a config module to handle HS config

Introduced a single module, config, to handle the fetching and parsing
of Hadoopstack. They are fetched from a conf_dir,
/etc/hadoopstack is default.

The information is saved in a 'conf' collection.

The module also provides the ability to fetch it using read_conf().

Signed-off-by: Shashank Sahni <shredder12@gmail.com>
Rewrote the scheduler

The scheduler can now handle multicloud(>2) environments and follows
the new configuration format.

The scheduler works by apply a bunch of filters and take decision.
Currently, only quota and priority based filters are implemented.

TODO: Do someting similar to openstack's filter scheduler.

Signed-off-by: Shashank Sahni <shredder12@gmail.com>
Updated ec2.make_connection()

Updated ec2.make_connection() to fetch credentials from conf's cloud
object.

Updated the concerned files/functions accordingly.

TODO: make the function independent of the cloud object i.e. directly
accept credentials or a dict.

Signed-off-by: Shashank Sahni <shredder12@gmail.com>
Fixed flush_data_to_mongo()

Signed-off-by: Shashank Sahni <shredder12@gmail.com>
Changes to flask app object initialization

* Config is set during flask app initalization
* Setting a few flask specific params in main.py

Signed-off-by: Shashank Sahni <shredder12@gmail.com>
Minor refactor in job.py

Signed-off-by: Shashank Sahni <shredder12@gmail.com>