Package: nbautoeval
nbautoevalnbautoevalis a very lightweight python framework for creatingauto-evaluatedexercises inside a jupyter (python) notebook.two flavours of exercises are supported at this point :code-oriented : given a text that describes the expectations, students are invited to
write their own code, and can then see the outcome on teacher-defined data samples,
compared with the results obtained through a teacher-provided solution, with a visual
(green/red) feedbackquizzes : a separate module allows to create quizzesAt this point, due to lack of knowledge/documentation about open/edx (read: the
version running at FUN), there is no available code for exporting the results as
grades or anything similar (hence theautoevalname).There indeed are provisions in the code to accumulate statistics on all
attempted corrections, as an attempt to provide feedback to teachers.Try it onmybinderClick the badge below to see a few sample demos undermybinder.org- it's all
in thedemo-notebookssubdir.NOTEthe demo notebooks ship under a.pyformat and requirejupytextto be
installed before you can open them in Jupyter.HistoryThis was initially embedded into aMOOC on
python2that ran for the first time onthe
French FUN platformin Fall 2014. It
was then duplicated into aMOOC on
bioinformaticsin Spring 2016 where it was
namednbautoevalfor the first time, but still embedded in a greater git module.The current git repo is created in June 2016 from that basis, with the intention
to be used as a git subtree from these 2 repos, and possibly others since a few
people have proved interested.Installationpip install nbautoevalOverviewcode-orientedCurrently supports the following types of exercisesExerciseFunction: the student is asked to write a functionExerciseRegexp: the student is asked to write a regular expressionExerciseGenerator: the student is asked to write a generator functionExerciseClass: tests will happen on a class implementationA teacher who wishes to implement an exercise needs to write 2 parts :One python file that defines an instance of an exercise class; this in a nutshell
typically involvesproviding one solution (let's say a function) written in Pythonproviding a set of input dataplus optionnally various tweaks for rendering resultsOne notebook that imports this exercise object, and can then take advantage of it to
write jupyter cells that typicallyinvokeexample()on  the  exercise  object to show examples of the expected outputinvite the student to write their own codeinvokecorrection()on  the  exercise  object to display the outcome.quizzesHere again there will be 2 parts at work :The recommended way is to define quizzes in YAML format :one YAML file can contain several quizzes - see examples in theyaml/subdirand each quiz contain a set of questionsgrouping questions into quizzes essentially makes sense wrt the maximal number of
attemptsmostly all the pieces can be written in markdown (currently we usemyst_parser)then one invokesrun_yaml_quiz()from a notebook to display the testthis function takes 2 arguments, one to help locate the YAML fileone to spot the quiz inside the YAML filerun withdebug=Trueto pinpoint errors in the sourceresults and storageRegardless of their type all tests have anexonamethat is used to store information
about that specific test; for quizzes it is recommended to use a different name than
the quiz name used inrun_yaml_quiz()so that students cant guess it too easily.stuff is stored in 2 separate locations :~/.nbautoeval.tracecontain one JSON line per attempt (correction or submit)~/.nbautoeval.storagefor quizzes only, preserves previous choices, number of attemptsKnown issuesseehttps://github.com/parmentelat/nbautoeval/issues