Package: jptest
JPTestJPTest is a unit testing framework for Jupyter Notebooks and aims for fast test writing in less lines of code. It creates the possibility to score and automatically grade exams with separate notebook (.ipynb) and test (.py) files.JPTest usestestbookbynteractfor interaction with notebooks. While we mainly focus on grading, testbook might fit your use case better. So please feel free to watchthis presentation on YouTubebyRohit Sanjayif you want to know more about testbook.InstallationJPTest relies ontestbookandJupyter. If you need any other libraries for executing your notebook cells they can be installed in your environment the usual way.The preferred way to use JPTest is in a virtual environment:python-mvenvvenvsourcevenv/bin/activateUsepipto download and install JPTest:pipinstalljptestUsageYou may use the decorator@JPTestwith any function within a test file. It accepts the following parameters. Any other named parameters are copied to the JPTestBook object.nametypedescriptionnamestrname used in the outputmax_scorefloatmaximum score (can be exceeded, used to calculate total score)executeseebelowcells or code to execute prior to the testtimeoutintexecution timeout in seconds (default: 2 minutes)yieldcan be used to grant points:yieldcondition:bool,value:float,[commentifTrue:str,[commentifFalse:str]]Run thejptestmodule with the notebook and test file as parameters. Every test with the decorator is automatically discovered and executed.python-mjptestnotebook.ipynbtests.pyFor more parameters see the included help:python-mjptest--helpUseful Functionsnamedescriptionrefget a reference to an object or functiongetlikeref, but copy to a name with a random suffix beforeExamplesa simple unit test# Mark as unit test and execute the cell with the tag `task-1`# prior to this test function.@JPTest('Task 1',max_score=2.0,execute=('task-1',))deftest_task1(tb:JPTestBook):# Use `ref` to reference an object or function from# the notebook.fib_from_nb=tb.ref('fib')# Grant points if output matches.yieldfib_from_nb(0)==0,0.5yieldfib_from_nb(1)==1,0.5yieldfib_from_nb(2)==1,0.5yieldfib_from_nb(20)==6765,0.5independent grading@JPTest('Task 2',max_score=1.0,execute=[# Inject a correct `fib` function instead of executing the# cell with the tag `task-1`.'''def fib(n):if n <= 1:return nelse:return fib(n-1) + fib(n-2)''',# Execute the cell with the tag `task-2` only.('task-2',)])deftest_task2(tb:JPTestBook):yieldtb.ref('fib_list')==[0,1,1,2,3,5,8,13,21,34],1.0track function calls@JPTest('Bonus Point',max_score=2.0,execute=('task-1',))deftest_recursive(tb:JPTestBook):fib_from_nb=tb.ref('fib')# Track calls of `fib` function and store them into `params`# after the context manager's `__exit__` function is executed.withtb.track('fib')asparams:fib_from_nb(20)# Award a bonus point if the function is recursive.yieldlen(params)>1000,1.0,None,'recursion... neat!'equality functionsSome objects like pandas dataframes are neither serializable nor comparable via==. One can call functions likeequalson references instead.@JPTest('Task 3',max_score=2.0,execute=('task-3',))deftest_task3(tb:JPTestBook):# Store `df` with a random suffix and get a reference.result=tb.get('df')# Inject sample solution and get a reference.test=tb.inject('df = pd.read_csv("data.csv", ...)','df')# Call `equals` function on one of the dataframes.yieldtest.equals(result),1.0# This also works with imported modules.# yield, tb.ref('nx').could_be_isomorphic(test, result), 1.0Execute ParameterOne can control in detail what code is executed prior to a test. Therefore theexecuteparameter accepts different types, which can also be nested recursively.StringIf the parameter is of typestr, the value is considered as code and injected into the notebook.TupleIf the parameter is of typetuple, the value is considered as tags. If there is one element in the tuple, every cell with this tag is executed. If there are two elements in the tuple, every cell between the first appearence of the first tag and the first appearence of the second tag (including) is executed.DictionaryIf the parameter is of typedict, it must include a key namedexecute, which value is executed following the rules stated in this section.It may include a key namedtrack, which value is another dictionary with function names as key and a tuple or a list of tuples containing the parameters to track. The result is passed to the test function as a parameter in the order of the execution.FunctionIf the parameter is a function (Callable), it will be called with the JPTestBook object as a parameter.ListIf the parameter is of typelist, every element will be executed in the order of its appearence, following the rules stated above.