Internal refactoring and a few other changes to support restish-ish traversal

In Mailman 3 we've been using restish for many years to provide administrative control over the core server via REST.  While this has served us well, there are two reasons we are looking to switch from restish: 1) its upstream development seems abandoned; 2) it is not Python 3 compatible, and porting has proven daunting.  I've looked around at *many* alternatives, but none so far have had as elegant an API as I consider restish to have.  I've just completed an attempt at porting MM3 to falcon and I am very impressed.  There are plenty of superficial API differences (e.g. @GET decorator in restish == def on_get() method in falcon, and statuses returned from methods in restish vs. set on the response object in falcon), but all of these are very easily ported.  Of greater concern is object traversal, i.e. translating from a url path to the resource object.  In restish, I can specify "child" methods that delegate further path processing to other object, while in falcon, I specify the end point that serves as the resource for a full url specificaion (albeit with route placeholders).  This is one API difference that I cannot port to as it would be too disruptive to my current code base, the machinery my REST infrastructure already uses, and beside I really like the restish approach! :)

Fortunately, with a little bit of internal refactoring, I was able to modify falcon's internal responder lookup routines so that my own subclass could implement the essential bits of the restish resource traversal model.  You can see the results in the MM3 branch:  http://bazaar.launchpad.net/~barry/mailman/falcon/view/head:/src/mailman/rest/wsgiapp.py#L67

This isn't a completely faithful reproduction of the restish model, but it's close enough to pass all my tests (I wasn't using many of the more advanced @child() features, and I opted for literal regexs rather than {placeholders}).  As an example of usage, take a look at this file, where I can traverse from the root of my resource tree to /3.0/system/preferences: http://bazaar.launchpad.net/~barry/mailman/falcon/view/head:/src/mailman/rest/root.py

A few caveats.  Most importantly, because this relies on the internal (non-public) structure of _get_responder(), you could end up changing falcon and breaking my code.  That would suck :).  I'd be very happy to discuss either making segment-based (i.e. @child()) traversal a first class feature of falcon (my ideal solution), or promoting the _get_responder() API to a public API specifically allowing for subclassing.  In any case, for now, this works!  (Since there's no falcon mailing list - feel free to contact me via email, or perhaps we can open a github issue to discuss further).

The other thing is that because @child traversal isn't first class, there are a few weird things I've had to adopt.  For example, in say an on_get() method I can just set the response.status = falcon.HTTP_404 to specify that a resource is not found, in @child traversal, if a segment handler must signal a 404, it has to return a special NotFound class and consume all remaining path segments.  The NotFound then implements on_{get,put,patch,delete} methods to set the 404 status regardless of the method.   You can see that here

http://bazaar.launchpad.net/~barry/mailman/falcon/view/head:/src/mailman/rest/root.py#L98
and
http://bazaar.launchpad.net/~barry/mailman/falcon/view/head:/src/mailman/rest/helpers.py#L260

There are a few other API conveniences I might suggest, but none that deeply affect the utility of falcon for MM3.  All in all, porting to MM3, with the branch under pull request, was quite a joy!  Great work.

A few other things to note.  I have to pass the full Request object to _get_responder() because @child segment handlers need the full request object. It's easy enough to dig out the path and method in _get_responder() so this seems like an uncontroversial change.

I had to add a "params" property to Request because I needed to iterate over all the parameters to say, a POST or PUT.  MM3 does this to validate those methods when e.g. updating a mailing list resource which can have a *ton* of attributes.

I also noticed what I think is a bug in falcon.  Forms can be posted with multiple values for the same key, but in unpatched falcon, the last one will win.  E.g. ant=1&ant=2&ant=3 (see the added unittest for details).  In this case, I modified the parameter to return a list of strings.  Perhaps not ideal, but simple enough to explain and it allows client code to do whatever value conversion is necessary.  I'd be happy to discuss alternatives, but I really must have support for multiple identical keys.  (Some of my attributes allow for lists of things, and they get posted as duplicate keys.)

I think that's it!  Hopefully it makes sense, but I'm happy to discuss further either here, over email, or in IRC.  With the added tests, the suite passes for me under Python 2.7 and 3.4 which are the only two versions I have atm.  I.e. $ tox -e py27,py34 passes with 100% coverage.

Cheers!
It sort of feel semantically equivalent to me, TBH. Maybe others can chime in with their thoughts...