Package: pythonrv
# pythonrvpythonrv is a runtime verification framework for python. It is theimplementation of a master thesis currently being written. Please see[tgwizard.github.com/thesis](http://tgwizard.github.com/thesis) for moreinformation.[![Build Status](https://secure.travis-ci.org/tgwizard/pythonrv.png)](http://travis-ci.org/tgwizard/pythonrv)## Installingpythonrv is available as a [pip package](http://pypi.python.org/pypi/pythonrv),with setup.py and everything. Use[pip](http://www.pip-installer.org/en/latest/index.html) to install it likethis:pip install pythonrvpythonrv has been tested with Python 2.7. It does not yet work with Python 3.## CodingClone the repository:git clone git@github.com:tgwizard/pythonrv.gitcd pythonrvInitialize the environment (preferrably through[virtualenv](http://pypi.python.org/pypi/virtualenv), and possibly[virtualenvwrapper](http://www.doughellmann.com/docs/virtualenvwrapper/)):pip install -r requirements.txtRun the tests:./runtests.sh## Runtime VerificationRuntime verification is the idea of verifying the correctness of a programduring its execution. Verification is done by checking if the program conformsto its specifications.There are several approaches to runtime verification, where pythonrv is one. Itis based on the idea that the specifications should be written in the targetprogramming language, in a similar manner to unit tests, and therefore makethem both more expressive and easier to learn.The specifications written in pythonrv get "baked" into the target's sourcecode, so that whenever a function is called the specifications are executed aswell.## ExamplesHere are some simple examples showing the pythonrv API. For more realisticexamples, see the[examples](https://github.com/tgwizard/pythonrv/tree/master/examples) folder.The [unittests](https://github.com/tgwizard/pythonrv/tree/master/pythonrv/test) mightalso be useful to see what works.First, lets say we have a function which correctness we want to verify:~~~ python# factorial.pydef factorial(n):res = 1for i in range(2, n)res *= ireturn res~~~We can now write specifications for it, preferably in another file:~~~ python# rvspecs.pyfrom pythonrv import rvimport factorial@rv.monitor(fact=factorial.factorial)def input_only_spec(event):assert event.fn.fact.inputs[0] >= 0@rv.monitor(fact=factorial.factorial)@rv.spec(when=rv.POST)def simple_specification(event):assert event.fn.fact.result >= event.fn.fact.inputs[0]@rv.monitor(fact=factorial.factorial)@rv.spec(when=rv.POST, history_size=rv.INFINITE_HISTORY_SIZE)def simple_specification(event):in_out = (event.fn.fact.inputs[0], event.fn.fact.result)old_in_out = [(x.inputs[0], x.result) for x in event.fn.fact.history]for a in old_in_out:if in_out[0] > old_in_out[0]:assert in_out[1] >= old_in_out[1]~~~The first specification checks that all inputs are greater-than-or-equal-tozero.The second specification checks that all outputs are at least as big as theinputs. The third specification verifies the input/output against thehistorical data for the function; given a larger input, the output must belarger-or-equal than before. Note that the last two specs are executed afterthe factorial has been called. This gives them access to return values andoutput arguments.A specification is sent an event with information of the called function, itshistory, and the history of all functions monitored by the specification.~~~ pythonfrom pythonrv import rvimport mymodule@rv.monitor(foo=mymodule.foo, bar=mymodule.bar)@rv.spec(when=rv.POST, history_size=20)def more_specifications(event):# here are all functions monitoredevent.fn# the currently called function can be accessed like thisevent.called_function# which, if mymodule.foo was called, is the same asevent.fn.foo# we can also check if a function was calledassert event.fn.foo.called# the inputs, outputs and result can be accessed like thisevent.fn.foo.inputs        # a copy of the input argument tupleevent.fn.foo.input_kwargs  # a copy of the input key-word argument dictevent.fn.foo.outputsevent.fn.foo.output_kwargsevent.fn.foo.result# we can gain access to the previous event, and the previous function callevent.prevevent.fn.foo.prev# if two calls to mymodule.foo occurr consecutivelyassert event.prev.fn.foo == event.fn.foo.prev# but they needn't be if more than one function is monitored by a spec# we can gain access to the "entire" historyfor old_event in event.history:pass# andfor old_foo_call in event.fn.foo.history:pass# this is obviously a big drain on the memory, so by default only two events# are stored in the history (this and the previous). this can be changed,# like we do here with @rv.spec(history_size=20)# we can also say that the next time some monitored function is called,# something should happenevent.next(call_next_time)# or for just a specific monitored functionevent.fn.foo.next(call_next_time)# we can also specify which monitored function should be the next to be# called:event.next_called_should_be(event.fn.bar)# sometimes a specification can "finish" - it need not be verified againevent.success("optional message telling that everything was ok")# orevent.failure("we've failed, and there's no point continuing this verification")def call_next_time(event):# here we gain access to all the same data as in the specpass~~~## When to Execute the SpecificationThe specification is executed before the call is passed on to the activemonitoree. This can be customized:~~~ python@rv.monitor(f=func)@rv.spec(when=rv.POST)def spec_after(event):pass# this is the default, it is not needed to explicitly specify PRE@rv.monitor(f=func)@rv.spec(when=rv.PRE)def spec_before(event):pass~~~## Dealing with ErrorsSpecifications signal verifications errors by raising the `AssertionError`exception (which the `assert` statement does). When this happens, pythonrv, bydefault, lets this error propagate, and, if uncaught, the program stops.If this is not the desired behaviour, it can be changed. For a logging errorhandler, do~~~ pythonfrom pythonrv import rvrv.configure(error_handler=rv.LoggingErrorHandler())~~~and then configure logging the normal way, through the [python loggingmodule](http://docs.python.org/library/logging.html).Specifications can be marked with an error level:~~~ python@rv.monitor(f=func)@rv.spec(level=rv.DEBUG)def spec(event):pass~~~The available error levels are DEBUG, INFO, WARNING, ERROR and CRITICAL (justas in the logging module).For more on error handling, see the [sourcecode](https://github.com/tgwizard/pythonrv/blob/master/pythonrv/rv.py), and the[unittests](https://github.com/tgwizard/pythonrv/blob/master/pythonrv/test/rv_configuration_test.py).## Technical Issues### ImportingIt is recommended that you import your rv specifications among the first thingsyou do in your program. The reasons will be detailed below.When writing specs, this is the **wrong** way:~~~ python# this doesn't workfrom mymodule import myfunc@rv.monitor(f=myfunc)def spec(event):pass~~~This is the correct way:~~~ python# this worksimport mymodule@rv.monitor(f=mymodule.myfunc)def spec(event):pass~~~The first example creates a reference to myfunc and inserts it into thecurrent module (the module defining the specification). Monitoring a functionmeans adding a wrapper around it, and in this case we only add a wrapper forthe myfunc reference in the current module. We do not modify mymodule, whichall other code will use. The second example fixes this.The above reason also explains why the specifications should beimported/defined at the very beginning of the execution: Other modules mightuse the from x import y style, and if they do so before the rv specificationshave had a chance to monitor/instrument the functions, they will getunmonitored/uninstrumented references to them.### Copying argumentsWhen intercepting function calls, pythonrv copies the arguments to make surethat the history it stores doesn't get altered afterwards and/or from theoutside. This also makes it so that the input arguments really are inputarguments, and not modified by the function itself.This might sometimes be deemed unnecessary, or needlessly expensive. It mightsometimes not even work, for instance when[`cStringIO`](http://docs.python.org/library/stringio.html) is involved. Thisis the case for [Django](https://www.djangoproject.com/) requests in v1.4.x(but not on master).Copying can be turned off for all specifications:~~~ pythonfrom pythonrv import rvrv.configure(enable_copy_args=False)~~~Or for a specific specification:~~~ pythonfrom pythonrv import rv@rv.monitor(func=somemodule.somefunc)@rv.spec(enable_copy_args=False)def spec(event):pass~~~Note: Disabling argument copying for one specification actually disablesargument copying for all monitored functions for that specification. Otherspecifications that monitor the same functions won't get argument copyingeither. This is "a feature".## Licensepythonrv is released under the [MITlicense](http://opensource.org/licenses/mit-license.php).