nvie/rq — pull request #305 by alexbw (2014-01-29 18:35:35+00:00)
Title: WIP: Allow jobs on inline functions and closures

Motivation: rapid iteration in a distributed computing environment requires the ability to quickly change and test code. Currently, in Python RQ, this workflow requires editing a codebase, and then distributing the new codebase to all computers that will consume jobs, and then testing the code on those workers. This code-syncing bottleneck is largely because functions declared in the `__main__` namespace cannot be reliably pickled and unpickled.

By using [dill](https://github.com/uqfoundation/dill) to serialize methods, we can accept inline functions, closures and instance methods trivially. Also, the worker machines code need not be completely in sync with the dispatch machine's codebase. This allows for very rapid and robust iteration. In this way, the user can know that the code on the dispatching machine will be run as-is on the worker machine.

A few tests required modification to pass. Namely, for `job(func=my_method)`, calling `job.func_name` now resolves to `my_method`, which is what is returned by `my_method.__name__`. This occurs even if it's buried in a namespace like `my_module.my_submodule.my_method`

Also, the `sentry` test does not pass, because trying to create a job with a non-existent method will produce an immediate error, as opposed to an error in the sentry.