Package: unitgrade-devel
Unitgrade-develNote: This is the development version of unitgrade. If you are a student, please seehttp://gitlab.compute.dtu.dk/tuhe/unitgrade.Unitgrade is an automatic report and exam evaluation framework that enables instructors to offer automatically evaluated programming assignments.
Unitgrade is build on pythonsunittestframework so that the tests can be specified in a familiar syntax and will integrate with any modern IDE. What it offers beyondunittestis the ability to collect tests in reports (for automatic evaluation) and an easy and 100% safe mechanism for verifying the students results and creating additional, hidden tests. A powerful cache system allows instructors to automatically create test-answers based on a working solution.100% PythonunittestcompatibleNo configuration filesNo limitations: If you canunittestit, it worksTests are quick to run and will integrate with your IDECache and hint-system makes tests easy to developGranular security model:Students get publicunittestsfor easy development of solutionsStudents use a tamper-resistant file to create submissions which are uploadedInstructors can automatically verify the students solution using a Docker VM and run hidden testsAutomatic Moss anti-plagiarism detectionCMU Autolab integration (Experimental)A live dashboard which shows the outcome of the testsInstallSimply usepippip install unitgrade-develThis will installunitgrade-devel(this package) and all dependencies to get you started.OverviewThe figure shows an overview of the workflow.You write exercises and a suite of unittests.They are then compiled to a version of the exercises without solutions.The students solve the exercises using the tests and when they are happy, they run an automatically generated_grade.py-script to produce a.token-file with the number of points they obtain. This file is then uploaded for further verification/evaluation.The students can see their progress and review hints using the dashboard (see below)VideosVideos where I try to talk and code my way through the examples can be found on youtube:First test:https://youtu.be/jC9AzZA5FcQFramework and hints:https://youtu.be/xyY9Qan1b1QMOSS plagiarism check:https://youtu.be/Cp4PvOnYozoHidden tests and Docker:https://youtu.be/vP6ZqeDwC5UJupyter notebooks:https://youtu.be/B6nzVuFTEsAAutolab:https://youtu.be/h5mqR8iNMwMInstructions and examples of useThe examples can be found in the/examplesdirectory:https://gitlab.compute.dtu.dk/tuhe/unitgrade_private/-/tree/master/examplesA simple exampleUnitgrade makes the following assumptions:Your code is in pythonWhatever you want to do can be specified as aunittestAlthough not required, it is recommended you maintain two version of the code:A fully-working version (i.e. all tests pass)A public version distributed to students (some code removed))I usecodesnipper(seehttp://gitlab.compute.dtu.dk/tuhe/snipper) to synchronize the two versions automatically.Let's look at an example. Suppose our course is calledcs101, in which case we make three files in our private folderinstructor:instructor/cs101/homework.py # This contains the students homework
instructor/cs101/report1.py  # This contains the tests
instructor/cs101/deploy.py   # A private file to deploy the testsThe homeworkThe homework is just any old python code you would give to the students. For instance:# autolab_example_py_upload/instructor/cs102_autolab/homework1.pydefreverse_list(mylist):#!f"""Given a list 'mylist' returns a list consisting of the same elements in reverse order. E.g.reverse_list([1,2,3]) should return [3,2,1] (as a list)."""returnlist(reversed(mylist))defadd(a,b):#!f""" Given two numbers `a` and `b` this function should simply return their sum:> add(a,b) = a+b """returna+b*2if__name__=="__main__":# Example usage:print(f"Your result of 2 + 2 ={add(2,2)}")print(f"Reversing a small list",reverse_list([2,3,5,7]))The test:The test consists of individual problems and a report-class. The tests themselves are just regular Unittest (we will see a slightly smarter idea in a moment). For instance:# example_simplest/instructor/cs101/report1.pyfromcs101.homework1importreverse_list,addclassWeek1(unittest.TestCase):deftest_add(self):self.assertEqual(add(2,2),4)self.assertEqual(add(-100,5),-95)deftest_reverse(self):self.assertEqual(reverse_list([1,2,3]),[3,2,1])A number of tests can be collected into aReport, which will allow us to assign points to the tests and use the more advanced features of the framework later. A complete, minimal example:# example_simplest/instructor/cs101/report1.pyimportunittestfromunitgradeimportReport,evaluate_report_studentimportcs101fromcs101.homework1importreverse_list,addclassWeek1(unittest.TestCase):deftest_add(self):self.assertEqual(add(2,2),4)self.assertEqual(add(-100,5),-95)deftest_reverse(self):self.assertEqual(reverse_list([1,2,3]),[3,2,1])classReport1(Report):title="CS 101 Report 1"questions=[(Week1,10)]# Include a single question for a total of 10 credits.pack_imports=[cs101]# Include all .py files in this folderif__name__=="__main__":# from HtmlTestRunner import HTMLTestRunnerimportHtmlTestRunnerunittest.main(testRunner=HtmlTestRunner.HTMLTestRunner(output='example_dir'))# evaluate_report_student(Report1())DeploymentThe above is all you need if you simply want to use the framework as a self-check: Students can run the code and see how well they did.
In order to begin using the framework for evaluation we need to create a bit more structure. We do that by deploying the report class as follows:# example_simplest/instructor/cs101/deploy.pyfromcs101.report1importReport1fromunitgrade_private.hidden_create_filesimportsetup_grade_file_reportfromsnipperimportsnip_dirif__name__=="__main__":setup_grade_file_report(Report1)# Make the report1_grade.py report file# Deploy the files using snipper: https://gitlab.compute.dtu.dk/tuhe/snippersnip_dir("./","../../students/cs101",exclude=['__pycache__','*.token','deploy.py'])The first line creates thereport1_grade.pyscript and any additional data files needed by the tests (none in this case)The second line set up the students directory (remember, we have included the solutions!) and remove the students solutions. You can check the results in the students folder.If you are curious, the grade script looks like this:'''WARNING: Modifying, decompiling or otherwise tampering with this script, it's data or the resulting .token file will be investigated as a cheating attempt.'''importbz2,base64exec(bz2.decompress(base64.b64decode('QlpoOTFBWSZTWY/Cr/0ANxB/gHb2RFR//////+//vv////5gQb3d9962+(etc. etc.)')))Using the framework as a studentAfter you run the deploy-script, the student directory will contain three filesstudents/cs101/homework1.py      # Homework files without solutions (see for yourself)
students/cs101/report1.py        # Identical to the instructor-report
students/cs101/report1_grade.py  # Grade-script which runs the tests in report1.py and generates the .token-file.You can now upload thestudentdirectory to the students. The students can run their tests either by runningcs101/report1.pyin their IDE or by typing:python -m cs101.report1in the command line. This produces a detailed output of the test and the program is 100% compatible with a debugger. When the students are happy with their output they can run (using command line or IDE):python -m cs101.report1_gradeThis runs an identical set of tests and produces the fileReport1_handin_10_of_10.tokenthe students can upload to get credit.Thereport1_grade.pyincludes all tests and the main parts of the framework and is obfuscated by default. You can apply a much strong level of protection by using e.g.pyarmor.The.tokenfile includes the outcome of the tests, the time taken, and all python source code in the package. In other words, the file can be used for manual grading, for plagirism detection and for detecting tampering.You can easily use the framework to include output of functions.See below for how to validate the students resultsViewing the results using the dashboardI recommend to monitor and run the tests from the IDE, as this allows you to use the debugger in conjunction with your tests.
However, unitgrade comes with a dashboard that allows students to see the outcome of individual tests
and what is currently recorded in thetoken-file. To start the dashboard, they should simply run the commandunitgradefrom a directory that contains a test (the directory will be searched recursively for test files).
The command will start a small background service and open a webpage:Features supported in the current version:Shows which files need to be edited to solve the problemCollect hints given in the homework files and display them for the relevant testsfully responsive -- the UI, including the terminal, will update while the test is running regardless of where you launch the testAllows students to re-run tests from the UIShows current test status and results captured in.token-fileTested on Windows/LinuxFrontend is pure javascript and the backend only depends on python packages.The frontend is automatically enabled the moment your classes inherits from theUTestCase-class; no configuration files required, and there are no known bugs.Note the frontend is currently not provided in the pypiunitgradepackage, but only through the gitlab repository (install usinggit cloneand thenpip install -e ./) -- it seems ready, but I want to test it on mac and a few more systems before publishing it.How safe is Unitgrade?There are three principal ways of cheating:Break the framework and submit a.tokenfile that 'lies' about the true number of points'Overfit' the tests by checking for specific inputs and hard-code the outputPlagiarismThe degree to which the above problems needs to be mitigated depends on the course, but there are easy ways to mitigate them, but to address the three ways of cheating I recommend the following:Automatically re-run the students tests on your computer using Docker (see below) to automatically detect difference in their (claimed) outcome and the (actual) outcomeInclude a few hidden tests. If the students tests pass, but hidden tests with minor input-argument variation fail, something is probably upUse the build-in Moss plagiarism input to get a detailed plagiarism report (see below)I think the most important things to keep in mind are the following:The_grade.py-script is self-contained (i.e. contains an independent copy of all tests)The_grade.py-script and.tokenfile is not in an easily editable format.The.tokenfile will contain a copy of all the students source code, as well as any intermediary outputs returned by testsThis means that if a student begins to tamper with the framework, all the evidence of the tampering will be readily available, and any inconsistencies will be very difficult to explain away.
Therefore, unlike for a report, you cannot submit code as a.pdffile, and you cannot afterwards claim you mistook theDownloadfolder for theDesktopand accidentially uploaded your friends version of some of the code.If this is not enough, you can consider usingpyarmoron the_grade.pyscript to create averydifficult challenge for a prospective hacker.Example 2: The frameworkOne of the main advantages ofunitgradeover web-based autograders it that tests are really easy to develop and maintain. To take advantage of this, we simply change the class the questions inherit from toUTestCase(this is still aunittest.TestCase) and we can make use of the chache system. As an example:# example_framework/instructor/cs102/report2.pyfromunitgradeimportUTestCase,cacheclassWeek1(UTestCase):@classmethoddefsetUpClass(cls)->None:a=234deftest_add(self):self.assertEqualC(add(2,2))self.assertEqualC(add(-100,5))# def test_reverse(self):#     self.assertEqualC(reverse_list([1, 2, 3]))Note we have changed the test-function toself.assertEqualC(theCis for cache) and dropped the expected result. Whatunitgradewill do
is to evaluate the teston the working version of the code, compute the results of the test,
and allow them to be available to the user. All this happens in thedeploy.pyscript from before.Nicer titlesTitles can be set either using python docstrings or programmatically. An example:# example_framework/instructor/cs102/report2.py# class Week1Titles(UTestCase):#     """ The same problem as before with nicer titles """#     def test_add(self):#         """ Test the addition method add(a,b) """#         self.assertEqualC(add(2,2))#         print("output generated by test")#         self.assertEqualC(add(-100, 5))#         # self.assertEqual(2,3, msg="This test automatically fails.")##     def test_reverse(self):#         ls = [1, 2, 3]#         reverse = reverse_list(ls)#         self.assertEqualC(reverse)#         # Although the title is set after the test potentially fails, it will *always* show correctly for the student.#         self.title = f"Checking if reverse_list({ls}) = {reverse}"  # Programmatically set the titleWhen this is run, the titles are shown as follows:_   _       _ _   _____               _      
| | | |     (_) | |  __ \             | |     
| | | |_ __  _| |_| |  \/_ __ __ _  __| | ___ 
| | | | '_ \| | __| | __| '__/ _` |/ _` |/ _ \
| |_| | | | | | |_| |_\ \ | | (_| | (_| |  __/
 \___/|_| |_|_|\__|\____/_|  \__,_|\__,_|\___| v0.1.27, started: 16/09/2022 14:30:15

CS 102 Report 2 
Question 1: Week1                                                                                                       
 * q1.1) test_add.................................................................................................FAILED
 * q1.2) test_reverse.............................................................................................FAILED
 * q1.3) test_output_capture........................................................................................PASS
======================================================================
FAIL: test_add (__main__.Week1)
test_add
----------------------------------------------------------------------
Traceback (most recent call last):
  File "<string>", line 882, in _callTestMethod
  File "<string>", line 1699, in test_add
  File "<string>", line 987, in assertEqualC
  File "<string>", line 975, in wrap_assert
AssertionError: 4 != 'Key 0 not found in cache; framework files missing. Please run deploy()'

======================================================================
FAIL: test_reverse (__main__.Week1)
test_reverse
----------------------------------------------------------------------
Traceback (most recent call last):
  File "<string>", line 882, in _callTestMethod
  File "<string>", line 1703, in test_reverse
  File "<string>", line 987, in assertEqualC
  File "<string>", line 975, in wrap_assert
AssertionError: [3, 2, 1] != 'Key 0 not found in cache; framework files missing. Please run deploy()'

 * q1)   Total..................................................................................................... 3/10
 
Question 2: The same problem as before with nicer titles                                                                
 * q2.1) Test the addition method add(a,b)........................................................................FAILED
 * q2.2) test_reverse.............................................................................................FAILED
======================================================================
FAIL: test_add (__main__.Week1Titles)
Test the addition method add(a,b)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "<string>", line 882, in _callTestMethod
  File "<string>", line 1715, in test_add
  File "<string>", line 987, in assertEqualC
  File "<string>", line 975, in wrap_assert
AssertionError: 4 != 'Key 0 not found in cache; framework files missing. Please run deploy()'

======================================================================
FAIL: test_reverse (__main__.Week1Titles)
test_reverse
----------------------------------------------------------------------
Traceback (most recent call last):
  File "<string>", line 882, in _callTestMethod
  File "<string>", line 1723, in test_reverse
  File "<string>", line 987, in assertEqualC
  File "<string>", line 975, in wrap_assert
AssertionError: [3, 2, 1] != 'Key 0 not found in cache; framework files missing. Please run deploy()'

 * q2)   Total...................................................................................................... 0/6
 
Total points at 14:30:15 (0 minutes, 0 seconds).....................................................................3/16

Including files in upload...
path.: _NamespacePath(['/home/tuhe/Documents/unitgrade_private/examples/example_framework/instructor/cs102', '/home/tuhe/Documents/unitgrade_private/examples/example_framework/instructor/cs102'])
 * cs102
> Testing token file integrity...
Done!
 
To get credit for your results, please upload the single unmodified file: 
> /home/tuhe/Documents/unitgrade_private/examples/example_framework/instructor/cs102/Report2_handin_3_of_16.tokenWhat happens behind the scenes when we setself.titleis that the result is pre-computed on the instructors machine and cached. This means the last test will display the correct result regardless of howreverse_listhas been implemented by the student. The titles are also shown correctly when the method is run as a unittest.Caching computationsThe@cache-decorator offers a direct ways to compute the correct result on an instructors computer and submit it to the student. For instance:# example_framework/instructor/cs102/report2.py# class Question2(UTestCase):#     @cache#     def my_reversal(self, ls):#         # The '@cache' decorator ensures the function is not run on the *students* computer#         # Instead the code is run on the teachers computer and the result is passed on with the#         # other pre-computed results -- i.e. this function will run regardless of how the student happens to have#         # implemented reverse_list.#         return reverse_list(ls)##     def test_reverse_tricky(self):#         ls = (2,4,8)#         ls2 = self.my_reversal(tuple(ls))                   # This will always produce the right result, [8, 4, 2]#         print("The correct answer is supposed to be", ls2)  # Show students the correct answer#         self.assertEqualC(reverse_list(ls))                 # This will actually test the students code.#         return "Buy world!"                                 # This value will be stored in the .token fileThe@cachedecorator will make sure the output of the function is pre-computed when the test is set up, and that the function will
simply return the correct result regardless of the function body. This is very helpful in a few situations:if you have exercises that depend on each other, and you want students to have access to the expected result of older methods which they may not have implemented correctly.If you want to use functions the students write to set up appropriate tests without giving away the solution.To simply print out the correct result so it is apparent to the studentFinally, notice how one of the tests has a return value. This will be automatically saved in the.tokenfile (this is useful for open-ended questions or for security).Example 3: Hidden and secure testsTo useunitgradeas part of automatic grading, it is recommended you check the students output locally and use hidden tests. Fortunately, this is very easy.Let's start with the hidden tests. As usual we write a complete report script (report3_complete.py), but this time we use the@hide-decorator to mark tests as hidden:# example_docker/instructor/cs103/report3_complete.pyfromunitgradeimportUTestCase,Reportfromunitgrade.utilsimporthidefromunitgradeimportevaluate_report_studentimportcs103classAutomaticPass(UTestCase):deftest_automatic_pass(self):self.assertEqual(2,2)# For simplicity, this test will always pass@hide# The @hide-decorator tells unitgrade_v1 to hide the test for students.deftest_hidden_fail(self):self.assertEqual(2,3)# For simplicity, this test will always fail.classReport3(Report):title="CS 101 Report 3"questions=[(AutomaticPass,10)]# Include a single question for 10 credits.pack_imports=[cs103]For simplicity, non-hidden test will always pass, and the hidden test will always fail: This makes it easy to interpret the results in the following.Next we need to create students report and grade scripts. This can done as follows:# example_docker/instructor/cs103/deploy.pyif__name__=="__main__":# Step 1: Deploy the students files and return the directory they were written tosetup_grade_file_report(Report3)# Create report3_complete_grade.py which tests everythingfout,Report=remove_hidden_methods(Report3,outfile="report3.py")# Create report3.py without @hide-methodssetup_grade_file_report(Report)# Create report3_grade.py for the studentsstudent_directory="../../students/cs103"snip_dir("./",student_directory,exclude=['*.token','deploy.py','report3_complete*.py','.*'])Just to check, let's have a quick look at the students report scriptreport3.py:# example_docker/instructor/cs103/report3.pyfromunitgradeimportUTestCase,Reportfromunitgrade.utilsimporthidefromunitgradeimportevaluate_report_studentimportcs103classAutomaticPass(UTestCase):deftest_automatic_pass(self):self.assertEqual(2,2)# For simplicity, this test will always passclassReport3(Report):title="CS 101 Report 3"questions=[(AutomaticPass,10)]# Include a single question for 10 credits.pack_imports=[cs103]The grade script works as normal, and just to make the example self-contained, let's generate the students.token-file as follows:# example_docker/instructor/cs103/deploy.pyos.system("cd ../../students && python -m cs103.report3_grade")student_token_file=glob.glob(student_directory+"/*.token").pop()Setting up and using DockerWe are going to run the students tests in a Docker virtual machine so that we avoid any underhanded stuff, and also because it makes sure we get the same result every time (i.e., we can pass the task on to TAs).
To do that, you first have to install Docker (easy), and then build a Docker image. We are going to use one of the pre-baked images fromhttps://gitlab.compute.dtu.dk/tuhe/unitgrade_private/-/tree/master/docker_images, which simply consists of a lean Linux distribution with python 3.8 and whatever packages are found inrequirements.txt. If you need more it is very easy to add.
To download and build the Docker image simply run:# example_docker/instructor/cs103/deploy.py# Step 3: Compile the Docker image (obviously you should only do this once).download_docker_images(destination="../docker")# Download an up-to-date docker image from gitlab.Dockerfile="../docker/unitgrade-docker/Dockerfile"# Location of just downloaded docker filecompile_docker_image(Dockerfile,tag="unitgrade-docker")This takes about 2 minutes but only needs to be done once. If you are keeping track we have the following:A grade script with all tests,report3_complete_grade.py, which we build when the file was deployedA (student).tokenfile we simulated, but in general would have downloaded from DTU LearnA Docker image with the right packagesNext we feed this into unitgrade:# example_docker/instructor/cs103/deploy.py# Step 4: Test the students code in the .token file and get the results-token-file:token=docker_run_token_file(Dockerfile_location=Dockerfile,host_tmp_dir=os.path.dirname(Dockerfile)+"/home",student_token_file=student_token_file,instructor_grade_script="report3_complete_grade.py",tag="unitgrade-docker")Behind the scenes, this code does the following:Load the docker imageCreate a tmp dir where the files in the student.tokenfiles will be placedPut thereport3_complete_grade.pyscript at the right location in the source tree (unitgrade can guess this)Runreport3_complete_grade.pyand collect the resulting token fileJust to show it works we will load both.token-files and print the results:# example_docker/instructor/cs103/deploy.py# Load the two token files and compare their scoreschecked_token,_=load_token(token)results,_=load_token(student_token_file)print("Student's score was:",results['total'])print("My independent evaluation of the students score was",checked_token['total'])The results (shown in a(points_obtained, possible_points)format) will be printed as:Student's score was: (10, 10)
My independent evaluation of the students score was (5, 10)As expected, the (failed) hidden tests reduces the total points obtained. It will be easy to check, for instance by calling the hidden testsdef test_something_hiddenwhen the regular test,test_something, passes and the hidden test fails.Moss plagiarism detectionYou can easily apply Moss to the students token files.  First get moss fromhttps://theory.stanford.edu/~aiken/moss/and create two directories:whitelist/   # Whitelisted files. Code from these files are part of the handouts to students
submissions/ # Where you dump student submissions.The whitelist directory is optional, and the submissions directory contains student submissions (one folder per student):/submissions/<student-id-1>/Report1_74_of_144.token
/submissions/<student-id-2>/Report1_130_of_144.token
...The files in the whitelist/student directory can be either.tokenfiles (which are unpacked) or python files, and they may contain subdirectories: Everything will be unpacked and flattened. The simplest way to set it up is simply to download all files from DTU learn as a zip-file and unzip it somewhere.
When done just call moss as follows:# example_moss/moss_example.pyfromunitgrade_private.plagiarism.mossitimportmoss_it,get_idif__name__=="__main__":# Extract the moss id ("12415...") from the perl script and test:id=get_id("../../../02465private/admin/moss.pl")moss_it(whitelist_dir="whitelist",submissions_dir="student_submissions",moss_id=id)This will generate a report. You can see the example including the report here:https://lab.compute.dtu.dk/tuhe/unitgrade_private/-/tree/master/examples/example_mossSmart hintingTo help students get started, unitgrade will collect hints to solve failed tests from across the codebase and display them. Consider the following homework where two problems depends on each other and the
instructor has given a couple of hints: (example taken fromexample_hints):# example_hints/instructor/cs106/homework1.pydeffind_primes(n):#!f"""Return a list of all primes up to (and including) nHints:* Remember to return a *list* (and not a tuple or numpy ndarray)* Remember to include n if n is a prime* The first few primes are 2, 3, 5, ..."""primes=[pforpinrange(2,n+1)ifis_prime(n)]returnprimesdefis_prime(n):#!f"""Return true iff n is a primeHints:* A number is a prime if it has no divisors* You can check if k divides n using the modulo-operator. I.e. n % k == True if k divides n."""forkinrange(2,n):ifk%n==0:returnFalsereturnTrueThe report_file also contains a single hint:# example_hints/instructor/cs106/report1hints.pyclassWeek1(UTestCase):deftest_find_all_primes(self):"""Hints:* Insert a breakpoint and check what your function find_primes(4) actually outputs"""self.assertEqual(find_primes(4),[2,3],msg="The list should only contain primes <= 4")classReport1Hints(Report):title="CS 106 Report 1"questions=[(Week1,10)]pack_imports=[homework_hints]When students run this homework it will fail and display the hints from the two methods:What happens behind the scenes is that a code-coverage tool is run on the instructors computer
to determine which methods are actually used in solving a problem, and then the hint-texts of those methods are
collected and displayed. This feature requires no external configuration; simply writeHints:in the source code.CMU Autolab support (Experimental)CMU Autolab is a mature, free and opensource web-based autograder developed at Carnegie Mellon University and used across the world. You can find more information here:https://autolabproject.com/. It offers all features you expect from an online autograderWeb-based submission of homeworkClass-managementBuild in TA feedback mechanismClass monitoring/statisticsAutomatic integration with enrollment data (Autolab supports LDAP and Shibboleth) means Autolab can beplugged into existing IT infrastructure (including DTUs)CLI ToolsAn important design choice behind CMU Autolab is the grading is entirely based on Makefiles and Docker VMs. I.e., if you can make your autograding scheme work as Makefile that runs code on a Docker image you specify it will work on Autolab. This makes it very easy to let third-party platforms work with anunmodifiedversion of Autolab. The following contains all steps needed to compile a Unitgrade test to AutolabStep 1: Set up AutolabSimply follow the guide here:https://docs.autolabproject.com/installation/overview/to set up Autolab. I used the 'manual' installation, but it should also work with the Docker-compose installation.Step 2: Compile a unitgrade test to Autolab lab-assignment formatAutolab calls handins forlab assignments, and allow you to import them as.tar-files (see the Autolab documentation for more information).  We can build these automatically in a few lines as this example demonstrates.
The code for the example can be found inexamples/autolab_example. It consists of two steps. The first is that you need to build the Docker image for Autolab/Tango used for grading. This is exactly like our earlier example using Docker for Unitgrade, except the image contains a few additional autolab-specific things. You can find the image here:https://gitlab.compute.dtu.dk/tuhe/unitgrade_private/-/tree/master/docker_images/docker_tango_pythonConcretely, the following code will download and build the image (note this code must be run on the same machine that you have installed Autolab on)# autolab_example_py_upload/instructor/cs102_autolab/deploy_autolab.py# Step 1: Download and compile docker grading image. You only need to do this once.download_docker_images("../docker")# Download docker images from gitlab (only do this once).dockerfile=f"../docker/docker_tango_python/Dockerfile"autograde_image='tango_python_tue2'# Tag given to the image in case you have multiple images.compile_docker_image(Dockerfile=dockerfile,tag=autograde_image,no_cache=False)# Compile docker image.Next, simply call the framework to compile any_grade.py-file into an Autolab-compatible.tarfile that can be imported from the web interface. The script requires you to specify
both the instructor-directory and the directory with the files the student have been handed out (i.e., the same file-system format we have seen earlier).# autolab_example_py_upload/instructor/cs102_autolab/deploy_autolab.py# Step 2: Create the cs102.tar file from the grade scripts.instructor_base=f"."student_base=f"../../students/cs102_autolab"fromreport2_testimportReport2# INSTRUCTOR_GRADE_FILE =output_tar=new_deploy_assignment("cs105h",# Autolab name of assignment (and name of .tar file)INSTRUCTOR_BASE=instructor_base,INSTRUCTOR_GRADE_FILE=f"{instructor_base}/report2_test_grade.py",STUDENT_BASE=student_base,STUDENT_GRADE_FILE=f"{instructor_base}/report2_test.py",autograde_image_tag=autograde_image,homework_file="homework1.py")This will produce a filecs102.tar. Whereas you needed to build the Docker image on the machine where you are running Autolab, you can build the lab assignments on any computer.Step 3: Upload the.tarlab-assignment fileTo install thecs102.tar-file, simply open your course in Autolab and click theINSTALL ASSESSMENTbutton. ClickBrowseand upload thecs102.tarfile:You will immediately see the page for the assignment where you can begin to upload solutions!
The solutions are (of course!).tokenfiles, and they will be automatically unpacked and run on Autolab.To test it, press the big upload square and select the.tokenfile for the second assignment found inexamples/example_framework/instructor/cs102/Report2_handin_18_of_18.token.
The file will now be automatically evaluated and the score registered as any other Autolab assignment:The students can choose to view both the console output or a nicer formatted overview of the individual problems:and TAs can choose to annotate the students code directly in Autolab -- we are here making use of the fact the code is automatically included in the top of the.token-file.Citing@online{unitgrade_devel,title={Unitgrade-devel (0.1.42): \texttt{pip install unitgrade-devel}},url={https://lab.compute.dtu.dk/tuhe/unitgrade_private},urldate={2022-09-16},month={9},publisher={Technical University of Denmark (DTU)},author={Tue Herlau},year={2022},}