Package: mixnmatchttp
Overviewmixnmatchttpis a modular HTTP/S server based onPython's http
serverthat lets you "mix
'n' match" various functionalities. It defines several request handlers, which
are wrappers aroundhttp.server.SimpleHTTPRequestHandler, as well as
aThreadingHTTPServerwhich can be used in place of Python'shttp.server.HTTPServerfor multi-threading support.Quick startRequest handlers define special endpoints and/or templates as class attributes.Endpoints are the RESTful API of the server. A request which does not map to an
endpoint is treated as a request for a file or directory (Python's http server
handles it, unless your class overrides thedo_(GET|POST|...)methods).
A request which does map to an endpoint will call an endpoint handler: a method
of your class by the namedo_{underscope-separated path}with "path" being
the most-specific (longest) path for which a method is defined. E.g./foo/bar/bazwill try to calldo_foo_bar_baz, thendo_foo_bar, thendo_foo, and finallydo_default.do_defaultis defined inBaseHTTPRequestHandlerbut your class may want to override it.Template pages are parametrized response bodies. Templates specify a template
page to be used with and give a dictionary of parameters and values to use with
the template. Each parameter value may also contain dynamic parameters, which
are given to theBaseHTTPRequestHandler.page_from_templatemethod, which
constructs the final page.You can inherit from one or more of the*HTTPRequestHandlers. Each parent's
endpoints/templates will be copied to, without overwriting, your child class'
endpoints/templates.Important notes:If you need to override any of the HTTP method handlers (e.g.do_GET),
you must decorate them withmixnmatchttp.handlers.base.methodhandler, as
shown in the demo below. And if you need to call any of the parent's HTTP
method handlers you must call the original wrapped method using the__wrapped__attribute, as shown in the demo.Defining endpointsEndpoints, templates and template pages constructors have the same signature as
for a dictionary. Endpoints are of typemixnmatchttp.endpoints.Endpoint,
while templates and template pages are of typemixnmatchttp.common.DictNoClobber. However, you can define them as
a dictionary, or any type which has a dictionary-like interface, andBaseHTTPRequestHandler's meta class will convert them to the appropriate
class.For example you can define endpoints like so:classMyHandler(BaseHTTPRequestHandler):_endpoints=mixnmatchttp.endpoints.Endpoint(some_sub={'$allowed_methods':{'GET','POST'},'$nargs':1,'some_sub_sub':{'$nargs':endpoints.ARGS_ANY,'$raw_args':True,# don't canonicalize rest of path}},some_other_sub={})or like so:classMyHandler(BaseHTTPRequestHandler):_endpoints=mixnmatchttp.endpoints.Endpoint({'some_sub':{'$allowed_methods':{'GET','POST'},'$nargs':1,'some_sub_sub':{'$nargs':endpoints.ARGS_ANY,'$raw_args':True,# don't canonicalize rest of path}},},some_other_sub={})or like so:classMyHandler(BaseHTTPRequestHandler):_endpoints={'some_sub':{'$allowed_methods':{'GET','POST'},'$nargs':1,'some_sub_sub':{'$nargs':endpoints.ARGS_ANY,'$raw_args':True,# don't canonicalize rest of path}},'some_other_sub':{}}Any keyword arguments or dictionary keys starting with$correspond to an
attribute (without the$) which specifies how an endpoint can be called.
All other keyword arguments/keys become child endpoints of the parent; their
value should be anotherEndpoint(or any dictionary-like object).Default endpoint attributes are:disabled=False|True# specifies if the enpoint cannot be called directly;# False for child endpoints but True for root endpointallowed_methods={'GET','HEAD'}# a set of allowed HTTP methods; HEAD is added to the# set if 'GET' is presentnargs=0# how many slash-separated arguments the endpoint can take;# can be a number of any of:#   mixnmatchttp.endpoints.ARGS_OPTIONAL for 0 or 1#   mixnmatchttp.endpoints.ARGS_ANY      for any number#   mixnmatchttp.endpoints.ARGS_REQUIRED for 1 or more# !!only reliable if raw_args is False!!raw_args=False# whether arguments should not be canonicalized,# e.g. /foo/..//bar/./baz will not be turned to /bar/bazChild endpoints are enabled by default, the root endpoint is disabled by
default; if you want it enabled, either manually change thedisabledattribute, or construct it like so:classMyHandler(BaseHTTPRequestHandler):_endpoints={'some_sub':{...},'$disabled':False,# a request for / will now call do_ or do_default# instead of do_(GET|POST|...)}Handling parsed endpointsWhen a path resolves to an endpoint,ep, the corresponding endpoint handler
(do_???) will be passed a single argument:
amixnmatchttp.endpoints.ParsedEndpoint(inherits frommixnmatchttp.endpoints.Endpoint) initialized from the originalepwith the
following additional attributes:httpreq: the instance ofBaseHTTPRequestHandlerfor this requesthandler: partial of thehttpreq's method selected as a handler, with the
first argument being theParsedEndpointitselfroot: longest path of the endpoint (with a leading/) corresponding to
a defined handler, i.e.  if the path is/foo/baranddo_foois
selected,rootwill be/foo; if usingdo_default,rootis empty ('').sub: rest of the path of the endpoint without a leading/, e.g.barorfoo/barargs: everything following the endpoint's path without a leading/argslen: the number of arguments it was called with (length of array from/separatedargs)Example: Implementing a serverSome methods that you may want to override, as well as implementing a custom
endpoint and template, are shown below forMyHandler:#  from __future__ import unicode_literalsfrom__future__importprint_functionfrom__future__importdivisionfrom__future__importabsolute_importfrombuiltinsimport*fromfutureimportstandard_librarystandard_library.install_aliases()importreimportsslfromhttp.serverimportHTTPServerfrommixnmatchttp.serversimportThreadingHTTPServerfrommixnmatchttpimportendpointsfrommixnmatchttp.handlersimportBaseHTTPRequestHandler,methodhandlerfrommixnmatchttp.commonimportDictNoClobberclassMyHandler(BaseHTTPRequestHandler):_endpoints=endpoints.Endpoint(foobar={},# will use do_default handlerrefreshme={'$nargs':endpoints.ARGS_OPTIONAL,},debug={# these are for when /debug is called'$allowed_methods':{'GET','POST'},'$nargs':1,'sub':{# will use do_debug handler# these are for when /debug/sub is called'$nargs':endpoints.ARGS_ANY,'$raw_args':True,# don't canonicalize rest of path}},)_template_pages=DictNoClobber(simpletxt={'data':'$CONTENT','type':'text/html'},simplehtml={'data':'''<!DOCTYPE html><html><head><meta charset="UTF-8" />$HEAD<title>$TITLE</title></head><body>$BODY</body></html>''','type':'text/html'},)_templates=DictNoClobber(refresh={'fields':{'HEAD':'<meta http-equiv="refresh" content="${interval}">','TITLE':'Example','BODY':'<h1>Example page, will refresh every ${interval}s.</h1>',},'page':'simplehtml',},debug={'fields':{'CONTENT':'${info}You called endpoint $root ($sub) ($args)',},'page':'simpletxt',},)defdo_refreshme(self,ep):interval=ep.argsifnotinterval:interval='30''''Handler for the endpoint /refreshme'''page=self.page_from_template(self.templates['refresh'],{'interval':interval})self.render(page)defdo_debug(self,ep):'''Handler for the endpoint /debug'''# set a header just for this requestself.headers_to_send['X-Debug']='Foo'page=self.page_from_template(self.templates['debug'],{'root':ep.root,'sub':ep.sub,'args':ep.args})self.render(page)defdo_default(self,ep):'''Default endpoints handler'''page=self.page_from_template(self.templates['debug'],{'info':'This is do_default. ','root':ep.root,'sub':ep.sub,'args':ep.args})self.render(page)# Don't forget this decorator!@methodhandlerdefdo_GET(self):# Do something here, then call parent's undecorated methodsuper().do_GET.__wrapped__()defdenied(self):'''Deny access to /forbidden'''ifre.match('^/forbidden(/|$)',self.pathname):# return args are passed to BaseHTTPRequestHandler.send_error# in that order; both messages are optionalreturn(403,None,'Access denied')returnsuper().denied()defno_cache(self):'''Only allow caching of scripts'''return(notself.pathname.endswith('.js'))orsuper().no_cache()defsend_custom_headers(self):'''Send our custom headers'''self.send_header('X-Foo','Foobar')if__name__=="__main__":use_SSL=Falsekeyfile=''# path to PEM key, if use_SSL is Truecertfile=''# path to PEM certificate, if use_SSL is Truesrv_cls=HTTPServer# srv_cls = ThreadingHTTPServer # if using multi-threadingaddress='127.0.0.1'port=58080httpd=srv_cls((address,port),MyHandler)ifuse_SSL:httpd.socket=ssl.wrap_socket(httpd.socket,keyfile=keyfile,certfile=certfile,server_side=True)try:httpd.serve_forever()exceptKeyboardInterrupt:httpd.server_close()A request for/debug/sub/..//foo/../bar/./bazwill callself.do_debugwithep, a copy ofMyHandler._endpoints['debug']['sub'].ep.rootwill be "debug",ep.subwill be "sub",ep.argswill be "..//foo/../bar/./baz".self.commandwill give the HTTP method.A request for/debug/foo/../barwill canonicalize the path to/debug/bar(sinceraw_argsfor/debugisFalse) calldo_debugwithep, a copy ofMyHandler._endpoints['debug'].ep.rootwill be "debug",ep.subwill be "",ep.argswill be "bar".A request for/debug/../barwill raisemixnmatchttp.endpoints.NotAnEndpointErrorsince the path will be canonicalized (as above) and result in/bar, andbaris not a valid endpoint.APOSTrequest for/foobarwill raisemixnmatchttp.endpoints.MethodNotAllowedErrorsince/foobaronly allows HTTPGET.A request for/debug/../foobarwill call parent'sdo_defaultwithep, a copy ofMyHandler._endpoints['foobar'].ep.rootwill be "",ep.subwill be "foobar",ep.argswill be "".A request for/debug/foo/barwill raisemixnmatchttp.endpoints.ExtraArgsErrorsince/debugexpect exactly one argument.A request for/debugwill raisemixnmatchttp.endpoints.MissingArgsErrorsince/debugexpect exactly one argument.A request for/refreshmewill render a page which refreshes every 30 seconds (default).A request for/refreshme/5will render a page which refreshes every 5 seconds.HandlersAuthHTTPRequestHandlerImplements username:password authentication via form or JSONPOSTrequest. Has configurable file paths/endpoints for which authentication is required.If no file containing username:password is set (as a_userfileclass attribute), it implements dummy authentication (all logins succeed).GET|POST /login: Issues aSESSIONcookie if username and password is validSupported URL parameters:goto: Redirect to this URLRequired body or URL parameters (unless no userfile was given, in which case it always authenticates):username: Username (duh)password: Password (duh)Response codes:200 OK: Authentication successful;SESSIONcookie is set401 Unauthorized: Username or password invalid;302 Found: Location is as requested via thegotoparameterNotes:Sessions are forgotten on the server side upon restartCookies are issued with theHttpOnlyflag, and if over SSL with theSecureflag as wellGET /logout: Clears theSESSIONcookie from the browser and the serverSupported URL parameters:goto: Redirect to this URLResponse codes:200 OK: Empty body302 Found: Location is as requested via thegotoparameterGET|POST /changepwd: Changes the password for a given usernameSupported URL parameters:goto: Redirect to this URLRequired body or URL parameters:username: Username (duh)password: Current passwordnew_password: New password (duh)Response codes:200 OK: Success; password is changed, currentSESSIONis invalidated and a newSESSIONcookie is set401 Unauthorized: Username or password invalid302 Found: Location is as requested via thegotoparameterCachingHTTPRequestHandlerAllows saving of content as a named page for later request, or displaying the encoded (URL or base64) request content as a page in response.POST /echo: Render the requested contentSupported URL parameters:data: The encoded content of the page to be rendered (required)type: The content type of the rendered page (defaults to text/plain)Supported formats:application/jsonwithbase64encoded dataapplication/x-www-form-urlencoded(with URL encoded data)Response codes:200 OK: The body andContent-Typeare as requested400 Bad Request: Cannot decode data or find the data parameterPOST /cache/{name}: Temporarily save the requested content (in memory only)Supported URL parameters and formats are the same as forPOST /echoResponse codes:204 No Content: Page cached500 Server Error: Maximum cache memory reached, or page{name}already cachedNotes:Once saved, a page cannot be overwritten (until the server is shutdown) even if it is cleared from memory (see/cache/clear)GET /cache/{name}: Retrieve a previously saved pageResponse codes:200 OK: The body andContent-Typeare as requested during caching500 Server Error: No such cached page, or page cleared from memoryGET /cache/clear/{name}: Clear a previously saved page to free memoryResponse codes:204 No Content: Page clearedGET /cache/clear: Clear all previously saved pages to free memoryResponse codes:204 No Content: All pages clearedGET /cache/new: Get a random UUIDResponse codes:200 OK: Body contains a randomly generated UUID; use inPOST /cache/{uuid}ProxyingHTTPRequestHandlerRedirects (with307) to any address given in the URL.GET /goto/{address}: Redirect to this (URI-decoded) addressResponse codes:302 Found: Location is the address which follows/goto/; if domain is not given (i.e. address does not start withschema://or//) it is taken from theReferer,Origin,X-Forwarded-Host,X-Forwarded-FororForwarded200 OK: Empty body; this happens if address was relative (no domain) and neither of the aforementioned headers was givenNotes:Unlike the address given as agotoparameter to some of the other endpoints, the address here is not URI-decodedThe{address}is not parsed at all, its path is not canonicalized unlike calls to other endpoints. I.e./login///foo.baz/../..//cachewill call/cache, but/goto///foo.baz/../..//cachewill redirect to//foo.baz/../..//cache(remote hostfoo.bazwith path../..//cache)Known issuesClearing of cache is not done safely in mutli-threaded context. You may experience issues under heavy load.Solution: Wait for fix...When running as a signle thread (default), the server sometimes hangs. It seems to be an issue whereby some browsers don't close the socket.Solution: Run the server in multi-thread mode (-toption).Occasionally aBrokenPipeErroris thrown. It happens with some browsers which close the socket abruptly.Solution: Just ignore it.Coming soonMT-safe saving and clearing of cachePossibly coming at some pointDatabase lookup of usersPassword policyDemos and sourceSource code and demo scripts which build on the handlers can be found athttps://github.com/aayla-secura/mixnmatchttp