Uses mkdir function in HdfsAtomicWritePipe init
use amazon ses for emailing
set permissions of pidfiles to 777

This commit also features some super minor refactoring of luigi.lock
Remove deprecated code from hive.py
Merge pull request #421 from spotify/erikbern/pidfile-777

set permissions of pidfiles to 777
seperated notify commands for ses and email
Merge pull request #422 from spotify/arash/remove-deprecated-stuff

Remove deprecated code from hive.py
renamed notify to send_email, removed globals()
Refactored luigi.worker a bit to use multiprocessing

...instead of os.fork and os.wait

This commit features the following stuff
- multiprocessing instead of os.fork
- Added a bunch of unit tests to run with multiple workers
- Fixed MockFile to support multiple processes
- Multiple processes are now supported even with a local scheduler

Other things I had to fix that was marginally related:
- Added --no-lock to a bunch of tests
- Fixed an issue in lock_test.py that triggered an error on my machine

I have no idea if/how this will work on non-Unix platforms (eg. Windows). On thing that should work (in theory) is to replace multiprocessing.Process with threading.Thread. I doubt that the previous code was working since it was relying directly on os.fork.
Merge pull request #407 from Houzz/simplify_atomic_mkdir

Uses mkdir function in HdfsAtomicWritePipe init
Merge pull request #419 from spotify/erikbern/worker-multiprocessing

Refactored luigi.worker a bit to use multiprocessing
Merge pull request #416 from tresata/notifications-boto

Allow use of Amazon SES for notification Emailing
ALPHA VERSION!
Add support for dynamic requirements

With this change tasks may yield requirements from the run method.
If the requirements are present the task will continue if not the task will
be scheduled to restart later.

In order for all processes to have access to the new requirements it is required
that the task define a dynamic_requires function that return the classes it may
yield. I don't like this very much and would be happy to change it if someone have
a better idea.

This is only a prototype patch to get some feedback. I realize that the eval will fail
if arguments contain unknown types and that tests are needed etc.
Add dynamic requirements example