Package: gatorgrader
GatorGraderThe only tool you'll need to ensure your student's code and writing is up to
speed!Table of ContentsQuickstart GuideKey FeaturesWhat Do People Think about GatorGrader?Installing GatorGraderTesting GatorGraderAutomated TestingTest CoverageTesting with Multiple Python VersionsCode LintingRunning GatorGraderUsing DockerComparison to Other ToolsPresentationsContributingContributorsQuickstart GuideStarter Repositories
An easy way to get started with GatorGrader is to check out our sample starter repositories.
The following starter repositories provide examples of how GatorGrader files should be created
to check programs and documentation for different languages:Java,Python,LaTexandHTML with CSS. These
examples also show how to integrate GatorGrader withGitHub
ClassroomandTravis
CI. When you follow these examples, the Gradle plugin
for GatorGrader will install it automatically when you rungradle gradein a
terminal window. Please be aware that these repositories are meant to have a majority
of red checks. This is only meant to be a “starter” and give an insight into what must be accomplished within the lab.Solution Repositories
The next step to get involved with GatorGrader is to checkout our sample solution
repositories. The following solution repositories provide examples of how GatorGrader files should be
created to check programs and documentation for different languages:Java,Python,LaTex, andHTML with CSS. These examples
also show how to integrate GatorGrader withGitHub
ClassroomandTravis
CI.  When you follow these examples, the Gradle plugin
for GatorGrader will install it automatically when you rungradle gradein a
terminal window. Please be aware that these repositories are meant to have a majority
of green checks. This is meant to show what must be accomplished within a lab/practical
and what it looks like when those tasks are completed.Key FeaturesGatorGrader automatically checks the work of technical writers and programmers.
It can:EnableGatorGradeto check
projects implemented and documented in a wide variety of languages (e.g.,
Java, Python, LaTeX, Markdown, HTML, and CSS).Integrate withGitHub Classroomto check
solution and starter repositories created for professors and students,
respectively.Run in a cloud-based environment likeTravis CIor
on the command-line of a developer's workstation.Operate as a "batteries included" grading tool, supporting automated checks
like the following:Does a file exist in the correct directory with the requested name?Does technical writing contain the desired number of words and paragraphs?Does source code contain the designated number of language-specific comments?Does source code or technical writing contain a required fragment or match
a specified regular expression?Does a command execute correctly and produce the expected number of output
lines?Does a command execute and produce output containing a fragment or matching
a regular expression?Does a GitHub repository contain the anticipated number of commits?Aligning with key recommendations in a recentNational Academies
report,
GatorGrader helps instructors automatically check student submissions in both
introductory and application-oriented classes using languages like Markdown,
Java, Python, JavaScript, CSS, and HTML. GatorGrader does not aim to solve
problems related to building and linting a project or managing an assignment's
submission, instead integrating with existing tools and systems likeGradle,GitHub, andGitHub
Classroomto effectively handle those tasks.Installing GatorGraderInstalling GatorGrader is not necessary if you intend to use it throughGatorGrade.
If you want to participate in the development of GatorGrader, the project
maintainers suggest the use ofPyenvto
install Python 3.7 or above. In addition to installingGitto access the project's GitHub repository, you should also installPoetryfor its support of package and virtual
environment management. After completing the installation of these tools, you
can type the following command in your terminal window to clone GatorGrader's
GitHub repository:gitclonehttps://github.com/GatorEducator/gatorgrader.gitIf you plan to develop new features for GatorGrader or if you want to run the
tool's test suite inPytest, then you
will need to install a virtual environment for development by typingpoetry installin the directory that contains GatorGrader. For help with this process, refer toPoetry's documentation.Testing GatorGraderAutomated TestingThe developers usePytestfor the testing
of GatorGrader. Depending on your goals, there are several different
configurations in which you can run the provided test suite. If you want to run
the test suite to see if the test cases are passing, then running this command
in a terminal window will perform testing with the version of Python with which
Poetry's virtual environment was initialized.poetry run task testTest CoverageAlong with running the test suite, the developers of GatorGrader use statement
and branch coverage to enhance their testing activities. To see the coverage of
the tests while also highlighting the lines that are not currently covered by
the tests, you can run this command in a terminal window. As with the previous
command, this will run the tests in the version of Python with which Poetry's
virtual environment was initialized.poetry run task coverTesting with Multiple Python VersionsThe previous two commands are restricted to running the test suite in the
version of Python with which Poetry was initialized. If you have installed multiple
versions of Python with Pyenv and you want to iteratively initialized Poetry with each
version and then run the test suite, then you should first run the following
commands to installPipxand use Pipx to
installInvoke. The first of these three
commands will installpipx, a program that supports the execution of Python
packages in isolated environments. The second command makes the directory~/.local/bin/a part of the search path for executable Python programs and the
third one installs theinvokecommand so that it is available on your
workstation outside of a virtual environment managed by Poetry, thereby ensuring
that it is always available to run tasks.pip install pipx --user
python -m userpath append ~/.local/bin/
pipx install invokeNow you can run the test suite in the specified versions of Python with the
following command. This example command will run the test suite in Python 3.6.8
and Python 3.7.3.invoke -c scripts/tasks test --pyenv 3.6.8 --pyenv 3.7.3If you want to track test coverage while running the tests in both Python 3.6.8
and 3.7.3, then you can run the following command.invoke -c scripts/tasks cover --pyenv 3.6.8 --pyenv 3.7.3You can switch the version with which Poetry is initialized by running the following
command that adopts, for instance, Python 3.7.3.invoke -c scripts/tasks switch --pyenv 3.7.3Code LintingThe developers of GatorGrader use linting and code formatting tools, such asPylint,Pydocstyle, andBlack. After installing GatorGrader's
development dependencies with Poetry, you can run all of the linters by typing
this command in a terminal window.poetryruntasklint--checkAutomated ChecksWant to learn about our linting checks? Check us out on our website,GatorGrader! We have detailed
descriptions of our linting checks and more! To get an idea of the linting checks we
offer, here is a quick list:ConfirmFileExistsCountCommandOutputCountCommitsCountFileLinesCountFileParagraphsWant to learn about our automated checks? Check them out on our website,gatorgrader.org! We have detailed
descriptions of our automated checks and more!Something you should know when working with our checks is that all of
them come with someoptional arguments. Optional arguments that you are likely
to encounter:-h--help--exact--advancedIf--helpis tagged along with a check then a help message will be displayed and
then exited. If further assistance is needed, please contact us on GitHub.Another feature with our automated checks is theplug-in based approach. This allows
users to implement their own check if our initial checks do not fulfill a check that
you find necessary.Running GatorGraderStudents and instructors normally use GatorGrader throughGatorGrade,
specifying the requested checks in agatorgrade.ymlfile. When run through GatorGrade, GatorGrader
reports each check that it performed, additionally sharing a diagnostic message
for each check that did not pass. Individuals who want to run GatorGrader as a
stand-alone Python application can install GatorGrader itself throughPipx,
and then rungatorgrader --helpto get more details on the command-line interface.Instructors often run GatorGrader in conjunction with other tools that check
source code and technical writing. For instance, in a Java-based introductory
course, instructors could verify student submissions withCheckstyle, thereby ensuring that
the Java source code adheres to the requirements in theGoogle Java Style
Guide. In this course, an
instructor could require that Markdown files with technical writing meet the
standards described in theMarkdown Syntax
Guide, meaning that all
Markdown files must pass the checks performed by theMarkdown linting
tool. These assignments could
also require that all submitted technical writing must adhere to the standards
set by theProselint tool. Since GatorGrader can run an
arbitrary command and check its error code, it is also possible to integrate it
with a wide variety of other linters, code formatters, and testing tools.Instructors may at times need to see a full list of checks to have a better understanding
and therefore, we feel that it is important to know that there is an easy way for that to happen.
This action will be completed through command line and therefore, you can typegatorgrader ListChecksinto your terminal, if you've installed GatorGrader as detailed above.
This allows for all of the checks to be printed out as output. This output will have the necessary
name labeled with the required and optional arguments. If this output does not give enough content,
we warmly invite you to navigate to our website, where we go into more detail about our Automated Checks.Using DockerNote: Docker supports the older GatorGrader execution tool,GatorGradle. It is not needed for GatorGrade!A vital part of our process for GatorGrader is to implement and use new techniques
to further our tool to grow. This is why we chose to use Docker! Docker is a container
platform and therefore, allows students using GatorGrader to just open a container
and have easy access to run all commands that would allow them to build, run, and
grade their labs and practicals. Docker is an industry standard and therefore,
gives us an advantage. To open a container that will allow for the use of GatorGrader,
run the following command in your terminal window:docker run -it --rm --name dockagator \
  -v "$(pwd)":/project \
  -v "$HOME/.dockagator":/root/.local/share \
  gatoreducator/dockagator /bin/bashFrom here, you are set! Test it out by building, running, or grading your lab/practical!
If you would like to learn more about Docker, please follow thislink.Comparison to Other ToolsOther automated grading tools include:autograde-github-classroom: "scripts to download and grade submissions to Github Classroom"check50: "a tool for checking student code"Classroom Assistant: "desktop application to help you get student repositories for grading"nbgrader: "a system for assigning and grading notebooks"nerfherder: "scripts for automating grading with GitHub Classroom and Moodle"Submitty: "homework submission, automated grading, and TA grading system"WebCat: "all-in-one plugin for full processing and feedback generation"Designed for instructors who want an alternative to simple scripts or
stand-alone platforms that do not integrate with industry-standard
infrastructure like GitHub and Travis CI, GatorGrader is a tool that
automatically checks the work of technical writers and programmers. Unlike other
systems, GatorGrader provides a "batteries included" approach to automated
grading that makes it easy for instructors to combine well-tested checks for
projects that students implement in a wide variety of programming languages.
GatorGrader's developers take its engineering seriously, maintaining
standards-compliant source code, a test suite with 100% statement and branch
coverage, and top-notch source code and user documentation.PresentationsGatorGrader's creators give presentations about the development, use, and
assessment of the tool. Please contact one of the developers if you would like
to feature a presentation about GatorGrader at your technical conference. The
following list includes some of our team's recent presentations:A Hands-on Guide to Teaching Programming with GitHub, Travis CI, and Pythonat PyOhio 2018Using GitHub, Travis CI, and Python to Introduce Collaborative Software Developmentat PyCon Education Summit 2018Using Python, Travis CI, and GitHub to Effectively Teach Programmingat PyGotham 2018ContributingAre you interested in contributing toGatorGrader,GatorGradle, or any of the
sample assignments (e.g.,Java,LaTeX, orHTML
with CSS)? Great,
because we appreciate the involvement of new contributors! Before you raise an
issue or start to make a contribution to GatorGrader's repository, we ask that
you review the project'scode of conductand thecontribution guidelines.ContributorsThanks goes to these wonderful people (emoji key):Gregory M. Kapfhammer📢💻📖🎨🚇⚠️Saejin Mahlau-Heinert💻📖🚇👀Christian Walker🖋Andrew Everitt💻📖Christian Lussier💻📖Simon Burrows💻Austin Bristol💻Matt💻📖Christopher Miller💻🎨📖Spencer Huang💻📖Mohammad Khan💻Zachary Shaffer💻📖Alexander Yarkosky🐛📖Dillon📖Zachary Leonardo💻Jonathan W. Mendez💻📖Tyler Lyle💻finneyj2💻📖schultzh🐛💻📖👀alexheinle🐛💻📖👀Zachary Andrews💻🎨📖Nicholas Tocci📖Devin Ho📖Matthew Baldeosingh📖Jordan Durci💻Karol Vargas📖Jerfenson Cerda Mejia💻📖Tara Douglass💻📖Alexander Butler💻corlettim🐛💻📖👀Carson Quigley💻Joshua Yee💻📖Madelyn Kapfhammer📖🚇⚠️This project follows theall-contributorsspecification. Contributions of any kind are welcome!