Package: gofer-service
Gofer_nb serviceThis repo contains a tornado flask app that accepts .ipynb files and grades them in a dockerized environment. Assuming you are running a Jupyterhub, you can ask Jupyterhub to run this gofer_service as a service; you also have the option to run it in a stand alone manner. Grades are saved to a sqlite database on the gofer_service mounted volume.A seperate Jupyterhub extension,gofer_submit, presents a "Submit" button to the user in a notebook rendered in Jupyterhub. The button is configured to serialize and send the notebook to this gofer_service as well as notify the the user of the successful submission.Database setupThis service, upon startup, creates and configures a sqlite database in the filegradebook.db. Thegradebook.dbfile is created at theVOLUME_PATHenvironment variable configured in thedeployment-config-encrypted.yamlfile found in the directorydeployment/cloud/.  If you are running a local installation the you will need to configure this environment variable in your environment along with a series of other environment variables explained later in this documentation.There is a file,dump_grades.py, that exports all the grades in the database to acsvfile.ConfigurationNotebook(ipynb metadata)The ipynb notebooks need to include the metadata for which assignment they are. In the case of Data 8x, there are three pieces of information that are relevant: the course id, section and lab. These are set in the metadata section of every notebook:metadata:{
    "course": "8x",
    "section": "1",
    "lab": "lab01"
    ...
}Course config pathPlease see the reposiotrydata-8/materials-x22-private. The repository houses a config file, 8x_course_config_edge.json and 8x_course_config.json. The system relies on this file specified in the environment variable,COURSE_CONFIG_PATH, to know which lab is associated with which assignment id in EdX. If you are not posting grades to an LTI server, than you do not need to worry about this.Test filesThis gets tricky. The notebooks and the corresponding test files used by this service are of course connected. The filesDockerfileandDockerfile-dev(used for local testing) download the current set of test files from the repository,materials-x22-private, for thematerials-x22notebooks. If you bring in different notebooks, you would need to change the two dockerfiles to bring in the corresponding tests.We assume a specific path to the test files. If you mirror the path found in thematerials-x22-privaterepository, all will work well. If you change the path, then you must change thesolutions-pathvariable ingrade_assignment.py.Docker ImageThis just FYI. The Dockerfile pulls an image :docker pull ucbdsinfra/otter-graderThis image is used by otter-grader to run the containerized grading.EdX/LTI integrationThe system posts the grade back to the EdX via LTI. You need to have theLTI_CONSUMER_KEYandLTI_CONSUMER_SECRETdefined and encoded viasopsfor this to work correctly. The secrets are ingofer_service/secretes/gke_key.yamlExternal installation with a re-direct from JupyterhubThis is the current deployment configuration. We deploy the gofer_service to gcloud and there is a re-direct from the Jupyterhubconfiguration filesin thedatahub repositorythat passes authentication information to gofer_service.Once the GKE cluster is created in gcloud, executing thedeployment/cloud/deploy.shfile  deploys the service to the cloud.Depoloyment Details:Rollback:If we deploy and find problems the quickest way to rollback the deployment is to look at the revision history and undo the deployment by deploying to a previous revision number:kubectl rollout history deployment gofer-pod -n grader-k8-namespacekubectl rollout history deployment gofer-pod -n grader-k8-namespace --revision=# <-- to see details like the version of the image usedkubectl rollout undo deployment/gofer-pod -n grader-k8-namespace --to-revision=#CI/CD:If you push a tag in the standard form of a version number(XX.XX.XX), github action creates a release from this tag, pushes the release to pypi.org, builds the docker image, pushes it google's image repository and deploys the new image into the GKE cluster.Local installation for testing/developingWith docker installed, you can use theDockerfile-devfile to deploy a local instance of gofer_service. Thedeployment/local/build.shfile gives some guidance to building and installing local changes to gofer_service for testing. The usual process is to make changes, executebuild.sh, which relies on adocker-compose.ymlfile. A sample is below but before we look, I would also study the filetests/integration.py. If you execute this file, you can test the service via a web connection.Sample docker-compose.yml:version: "3.9"
services:
  app:
    image: gofer
    build:
      context: .
      dockerfile: Dockerfile-dev
      args:
        GIT_ACCESS_TOKEN: your_access_token generated by your github account -- see below
        GOFER_SERVICE_VERSION: whatever_version you specify in gofer_service/__init__.py
    env_file:
      - ../.local-env
    ports:
      - 10101:10101
    volumes:
      - /tmp/gofer:/mnt/data
    entrypoint: ''

networks:
  default:
    driver: bridgeNotes:GIT_ACCESS_TOKEN is generated in your github account. This is used to download the materials-x22-private archive to the gofer_service docker image -- if you have test files somewhere else and they are not in a private repo this is unnecessary and you would need to change the relevant lines in the Dockerfile and Dockerfile-dev files..local-env These are environment variables that must be set. They mirror the variables indeployment/cloud/deployment-config-encrypted.yaml. You do not need to encrypt your local-env file with sops.Service installation in JupyterHubInstructions can be found here for running it as a service within yourjupyterhub