Bugfixes/improvements to scheduler : PR #507

Cleaned up version of PR #507 - should have all review comments incorporated.
Have synced internal svn to latest spark master : so should not have goof'ups in this one !

/cc @kayousterhout @mateiz