Package: grading-utils
Grading UtilsA collection of utilities for grading Python assignments.Utilities are based off some common unit testing patterns.
Overall, syntax is modelled off JavaScript's Jest syntax,
with some modifications (hencecamelCase, for example).Design Philosophy:securitysimplicityimmutabilityMatchers and ModifiersThe most common operations with this module will involve matching test values to expected values.
All matchers begin with theexpect(value)syntax, and are chained by some matcher.
For example, if we want to test $f(x) = x^2$, we write:fromgrading_utils.GradingUtilsimportexpectexpect(f(2)).toEqual(4)In this case, we could also write:expect(f(2)).toBe(4)which will check for referential equality amonst the two objects.
However,use cautionwhen using.toBe()to compare objects for equality,
as this will not useobj.__eq__orobj.__hash__to compare objects.
Matchers behave similarly to Python'sassert, but provide more context on error.Generally, matchers accept any type for which they are logically defined.
For example,.toBeApprox(2.5, eps=0.1)works for numeric types.Matchers with Callable ObjectsTypically, we want to check the output of a function,
so we call the function and pass its result to the expectation (expect(f(x))).
Occasionally, matchers need to track the way functions behave.
For example, letavg(l: iter) = sum(iter) / len(iter).
If we want to check whether[]throwsZeroDivisionError,
we can write:expect(avg).toThrow(ZeroDivisionError,[])Thus, we pass the function (uncalled) to ourexpectcall,
then we passtoThrow's custom argument (the exception toexcept),
and then all arguments to the function to the.toThrowcall.
This is an unfortunate limitation, but this works for the time being.
In the future, some decorator workaround may be implemented.List of MatchersThe following matchers are currently defined:toBetoEqualtoBeApproxtoMatchTypetoMatchtoContaintoBeGreaterThantoBeLessThantoThrowModifiersModifiers are used to modify the behavior of matchers, by modifying the underling "Expectation" object.
Most commonly, we may want to expect that a certain condition isnotmet.
We can chain modifiers for thus purpose.
Modifiers use._modifier_namesyntax, with()if arguments are needed.
There are currently only 3 modifiers:._notThis modifier negates the expectation. For $$f(x) = x^2$$:expect(f(2))._not.toBeGreaterThan(2)# throws GradingError._exception(e)If we want to throw a custom exception when a matcher fails.expect(f(2))._exception(UrAnAwfulProgrammerError).toBe(4)._maxtime(s)This is one of 3 existing options for timing.
The_maxtimemodifier should be used similarly to a matcher with callables.
To set a 2-second time limit onslow_reducer(some_set),
and expect a result ofTruewe write:expect(slow_reducer)._maxtime(2,some_set).toBe(True)For all timing semantics, see the dedicated timing section.Looking Forward_andand_orare some potential future modifiers.TimingOne critical aspect of any timer implementation is that for long running functions,
we want to keep grading time down; execution should be immediately stopped
when the time limit is reached, with a fail error.
This avoids functions from taking needlessly long times when they will fail anyways.
All times are given in seconds.Three methods are provided for time limiting functions:withtimer(5):expect(slow_reducer(some_set)).toBe(True)Or, when defining functions:@time_limit(5)defslow_reducer(some_set):pass# implementation hereexpect(slow_reducer(some_set)).toBe(True)Naturally, this can be easily used to wrap student solutions.Finally, the_maxtimemodifier can be used directly:expect(slow_reducer)._maxtime(2,some_set).toBe(True)