Package: pygradethis
pygradethisA Python package to facilitate checking code output or static code checking
using AST analysis. It can either be used with R using thelearnrpackage, as
a mirror ofgradethispackage, or as a standalone package for general Python
use in educational settings.Note: This package is in early development and will undergo rapid changes.Install pygradethispip install pygradethisInstall Dev Dependenciespip install -e .[dev]FeaturesSimple output checking based on pass / fail conditions with feedbackSimple static code checking (AST), with feedback on how student's code differs from solutionOutput checkspygradethismimics the cadence togradethis::grade_result. For e.g., we can
check that the student supplies thempgdataset like so:grade_result(pass_if_equals(mpg,"You also got the mpg dataframe!"),fail_if_equals(None,""))Internally, thesepass_if_equals(output, message)orfail_if_equals(output, message)will be checked sequentially in
the order of arguments and return on first condition we match. TheNonehere can be used
if you simply want to execute a condition if none of the other conditions matched.If we match apass_if_equalsorfail_if_equals, we will present a feedback message wrapped in a convenientdict:dict(message=str,correct=True|False,type="auto|success|info|warning|error|custom",location="append|prepend|replace")Themessageis the feedback, thecorrectis whether or not the student's solution is correct,typeis the type of feedback. When
used withlearnrthelocationfield here is useful for where the message is situated in the tutorial. However, for those using
this package as a standalone thelocationis not an important field and it can be ignored. More on the flagshere.Internally, a random praise/encouragement message will be appended before any custom message supplied.pass_if_equals(x=mpg,message="You also got the mpg dataframe!")Feedback:Bravo! You also got the mpg dataframe!fail_if_equals(x=None,message="")Feedback:Try it again. You get better each time.Code checksFor static code checking, we follow a similar cadence forgradethis::grade_code.When there is a solution code being supplied,grade_code(user_code, solution_code)can be used to check the AST of
the user and solution code, making sure to standardize function calls and producing a helpful message for the student
to diagnose their issue.Example:grade_code(student_code="2 + sqrt(log(2))",solution_code="2 + sqrt(log(1))")Feedback:I expectedlog(1), but what you wrote was interpreted aslog(2)insqrt(log(2))at line 1.Note how the feedback narrows in on the expression in which the problem occurs (sqrt(log(2)))
so that the student can focus on the most relevant outer expression of the problem. In this case, thelog(2)is the problem and the2on the left operand of
the addition is not as relevant.Similarly, here the feedback points out that the 2 within thelogfunction is incorrect, similar to thegradethisexample.Call Standardizationpygradethisalso knows how to take user's function call code and map positional arguments
to proper parameter names and set defaults if not supplied. This is so that you don't penalize
a student's code just because they did not explicitly spell out positional argument names, or
write the default arguments out.For e.g. suppose a student is calling the following silly functionfoo:deffoo(a,b=1):passGrading the code withgrade_code(student_code="foo(1)",solution_code="foo(1)")In the example above, thegrade_codedoesn't give us a feedback message since they are equivalent expressions.However, if the student suppliesfoo(2)grade_code(student_code="foo(2)",solution_code="foo(1)")we get back this feedback:I expected1, but what you wrote was interpreted as2infoo(2)at line 1.Note:Although underneath the hood we do standardize the arguments of both the student and the solution code
before checking, we don't surface this standardized form to the feedback message. This is certainly possible to
achieve but in certain cases can hinder learning by revealing too much information. For example, the builtin functions
likesumis normally called without specifying its actual formal parameters (e.g.sum(1)versussum(iterable=[1], start=0). In the future, averbosemode could be made available such that the formal parameters are pointed out.For call standardizing to work, the function definitions corresponding to function
calls must be defined  and 'live' in the environment, whether that is theglobals()/locals(),builtins, or custom module importspandas. This works if the student/solution source code also
includes the definition (likefooabove) in their own source code or it's included by instructor.Currently, common modules likemathis imported for grading withincheck_functions.py, but more modules
will be included to serve data science grading as well, such aspandasornumpyin the future.
We plan to make the code more extensible for the instructor to add them as dependencies.