Package: pyicap
pyicap======A Python 3 framework for writing ICAP serversInstallation------------You can use the package right away, if you copy the pyicap.py fileonto your PYTHONPATH, or the directory of your python module.A setup.py is provided with the package, run it to install thepackage:sudo ./setup.py installpip3/easy_install doesn't know about this yetYou can also install with pip or easy-install:sudo pip install pyicapWhat is ICAP?-------------For a dry and precise technical description see RFC 3507.ICAP is a protocol that is used by HTTP proxies to ask a separateservice (an ICAP server) to do modification on HTTP requests andresponses it proxies. Such proxy is an ICAP client.ICAP can be used to check permissions, scan viruses, place ads orotherwise modify the headers, content or request URL or HTTP requestsand/or responses. These can be done without modifying the proxy server'scode.The popular proxy software Squid 3.x supports the ICAP protocol, andthis framework was tested with Squid3.Design------The ICAP protocol closely resembles HTTP/1.1, so I choosed to modifyPython's stock BaseHTTPServer class for the purpose.It is important to note that ICAP _IS NOT_ an application of HTTP,neither a protocoll wrapped into it. If a relationship must be stated,I's say ICAP is a sibling of HTTP rather it's child.Because of this relationship a HTTP server or client cannot be triviallyextended (or even monkey-patched) to handle ICAP. This is why I chooseto copy, then completely rewrite the BaseHTTPServer classHow it works?-------------You can use a framework by importing stuff from the pyicap package,extending the protocol handler class and starting the server, passingyour handler to it:```python#!/bin/env python# -*- coding: utf8 -*-import randomimport SocketServerfrom pyicap import *class ThreadingSimpleServer(SocketServer.ThreadingMixIn, ICAPServer):passclass ICAPHandler(BaseICAPRequestHandler):def echo_OPTIONS(self):self.set_icap_response(200)self.set_icap_header('Methods', 'RESPMOD')self.set_icap_header('Preview', '0')self.send_headers(False)def echo_RESPMOD(self):self.no_adaptation_required()port = 13440server = ThreadingSimpleServer(('', port), ICAPHandler)try:while 1:server.handle_request()except KeyboardInterrupt:print "Finished"```The above example is a rewritten SimpleHTTPServer example withthreading. The SocketServer.ThreadingMixin can be used with ICAPServerjust like you would use it with SimpleHTTPServer.For every service endpoint there is a pair of methods. The currentexample simply does nothing by telling the ICAP client that the requestneeds no modification.ICAP defines three HTTP-like methods: OPTIONS, REQMOD and RESPMOD.OPTIONS must be handled in every case. An endpoint has to support eitherREQMOD or RESPMOD, but not both. However, this is not enforced, andaccording to the squid 3 documentation, such overloading will even workwith squid.REQMOD is called, when a HTTP request must be modified - like checkingaccess to an URL, stripping or adding query string parameters or POSTdata, modifying headers or otherwise mangling the request.RESPMOD is called when a HTTP response must be modified - such aschecking to-be-downloaded files for viruses, watermarking images oraudio files, placing ad banners, or otherwise modifying the contentand/or the headers of the request.ICAP works with URLs just like HTTP. Every ICAP service has an URL like:icap://icap.myorganization.com/place_bannersThe PyICAP framework will parse this URL, determines the ICAP method tocall (OPTIONS, REQMOD or RESPMOD), and calls one of the handler methodsthat the user (probably) provided. If the ICAP server can't find ahandler method, it returns a 404 error.In this particular example, the place_banners_*() method is called (*can be reqmod or respmod).An OPTIONS request triggers the related _options() method, in this casethe place_banners_options().Various information can be extracted from the ICAP request by examiningcertain fields of the handler object:* enc_req: encapsulated request line, list with 3 elements* enc_req_headers: encapsulated request headers, dictionary of lists* enc_res_status: encapsulated response status* enc_res_headers: encapsulated response headers* has_body: True, if the ICAP request has a body* servicename: name of the service endpoint* encapsulated: contains the "Encapsulated:" header's content as a dict* ieof: True, if read_chunk() encounters an ieof chunk extension* command: the current ICAP command* request_uri: contains the full request URI of the ICAP request* version: the version if the current ICAP request* preview: None, or an integer that arrived in the Preview header* allow: Contains a set() of Allow:-ed stuff* icap_response_code: contains the response code if set_icap_reponsewas called.There are several helper methods that can be called while serving arequets:* send_error(error_code): Sends and entire ICAP error response* no_adaptation_required(): Sends a response that means that leaves theencapsulated message unaltered. It honors the Allow header, and onlysends 204 No adaptation required if the client allowed such response.* cont(): Sends an ICAP 100 Continue response to the client. Can beused to request the client to continue sending data after a preview.* read_chunk(): Reads a chunk from the client. Be aware that this callblocks. If there is no available data on the line, and Connection:keep-alive is used, it will cause the server to hang. This methodshould only be called if it's sure there will be data availableeventually. If it returns an empty string, it means that it's thelast chunk, and no further read should be executed. It also sets theieof variable to True, if the ieof chunk extension is encountered.This extension is sent during a preview if the encapsulated messagefits in the preview entirelly. If ieof is True continue() must not becalled.* set_icap_response(code): sets the ICAP response* set_enc_status(stats): Sets the encapsulated status line* set_enc_request(request): Sets the encapsulated request line* set_enc_header(header, value): Set an encapsulated header. Multiplecalls will cause the header to be sent more than once. This is usefulfor example for Cookie: headers.* set_icap_header(header, value): Set an ICAP header. Note that this shouldnot be used normally, since all necesary ICAP headers are setautomatically by the framework (such as ISTag, Encapsulated, Date,Server, etc.)* send_headers(has_body=False): can be used after setting ICAP andencapsulated headers. The parameter has_body signals the existance ofan encapsulated message body.* send_chunk(data): writes a chunk to the client. An empty chunk mustbe written as the last chunk. Data must be sent after send appropriateheaders either with send_header() or enc_header()/icap_header() +send_headers(). The two header-sending methods must not be mixed.If sending data with send_headers, the has_body parameter must be setto properly indicate the existance or absence of an encapsulatedmessage body.