Using GitHub Actions to Test Student Code Submissions
Tags: javascript, testing, github, tutorial
Published: 2022-07-11 02:37:06

## Background
*This blog is based on Bit Project's [CounselorBot](https://github.com/apps/counselorbot), which you can test out yourself!*

**We currently have two available courses:** 
* [Intro to Serverless](https://github.com/bitprj/Intro-To-Serverless)
* [Intro to JavaScript](https://github.com/bitprj/js-serverless-prep-course)

Get started with this [video tutorial](https://www.loom.com/share/7edcdb4f60a443ebbad80d2e7b962deb) or this [walkthrough page](https://github.com/bitprj/Intro-To-Serverless/blob/main/GETTING_STARTED.md).

## Overview
[GitHub Actions](https://github.com/features/actions) are often used for automated code operations, usually automatic deployments. However, they also have another handy-dandy use case: **checking students' code submissions** so you (being lazy) don't have to run them yourself!

> In a future tutorial, we'll also talk about how output from GitHub Actions can be automatically commented on students' pull requests to provide feedback on what went wrong.

## Outlining Our Plan
All of the workflow files used as examples in this tutorial can be found [here](https://github.com/bitprj/Intro-To-Serverless).
![CounselorBot](https://user-images.githubusercontent.com/69332964/176958285-eb4830b6-cfbd-42fc-a053-e78873dd6b73.png)
*Each "step" in our curriculum requires a different test. When a student (the user) commits to the repository, we need to...*
1. Figure out which workflow file to run
2. Figure out which step the student is on

*...in order to run the correct test.*

Alongside actually running the tests, we will also be using GitHub Actions to determine the answers to these two questions.

## Writing the YAML File
```yaml=
name: Getting Started
on:
  push:
    branches:
      - hello
```
The first section of the [YAML file](https://github.com/bitprj/Intro-To-Serverless/blob/main/.bit/workflows/hello.yml) gives the GitHub Action a display name and configures *when* the action should run. In this case, it'll run when a commit is pushed to the `hello` branch.

```yaml=
jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout Code
        uses: actions/checkout@v2

      - name: Setup Node Environment
        uses: actions/setup-node@v2
        with:
          node-version: '14'

      - name: Install Dependencies
        run: |
          npm install minimist
          npm install node-fetch
```
The next section provides other steps to "set up" our environment. We will mainly be using node to test code; **npm dependencies** can also be installed if they will be used in the tests.

```yaml=
      - name: Get Count
        id: vars
        run: echo ::set-output name=count::$(node ./.bit/scripts/getProgress.js --repo=${{github.event.repository.name}} --owner=${{github.repository_owner}} )
```
Our first "unique" step in this workflow involves using [this script](https://github.com/bitprj/Intro-To-Serverless/blob/main/.bit/scripts/getProgress.js) to retrieve the step that the student is currently on. This "count" value is an integer that correlates with steps in our curriculum.

> **Note:** the `echo ::set-output name=count::` part sets the output of the file that runs to a local variable named `count`, which will be accessed later with `steps.vars.outputs.count`.

## Testing Different Types Of Submissions
**Now that the environment for our workflow is set up, we can run different *test files*, which provide feedback on student code.** With GitHub Actions, you are essentially just running commands on a terminal. This opens up options from running JavaScript files to testing webpages to calling Azure Functions.
> All test examples used in this tutorial can be found in [this directory](https://github.com/bitprj/Intro-To-Serverless/tree/main/.bit/tests) and [this directory](https://github.com/bitprj/Intro-To-Serverless/tree/main/.bit/tests/cypress/integration).

### Testing Program Files (JS, Python)
![test](https://user-images.githubusercontent.com/69332964/177048987-a1f20cc9-1fda-4351-91ea-f4df9cf3aa97.png)
```yaml=
      - name: Step 1
        if: ${{steps.vars.outputs.count == 1 && github.event.head_commit.message != 'Update progress'}}
        run: |
          node .bit/tests/test.1.2.js --repo=${{github.event.repository.name}} --user=${{github.repository_owner}}
```
This step can be added onto the `steps` section of the YAML file for workflow configuration. A **conditional** statement is utilized to *only* run this test if the "count" value is correct. The command `node .bit/tests/test.1.2.js` would then execute [this test](https://github.com/bitprj/Intro-To-Serverless/blob/main/.bit/tests/test.1.2.js).

```javascript=
try { hello = require('./../../week1/helloworld.js') }
```
The test first attempts to import the student's code in the repository.
```javascript=
let helloworld = hello()
let test_output = "Hello World"
```
If successful, it will attempt to execute the imported functions.
```javascript=
await functions.throwError(`Got: '${helloworld}', was expecting: '${test_output}'.`, user, repo)
console.log(`Got: "${helloworld}", was expecting: "${test_output}".`)
process.exit(1)
```
> **Note:** Source code for the `functions.throwError()` method can be found [here](https://github.com/bitprj/Intro-To-Serverless/blob/main/.bit/tests/functions.js). It includes functions used to provide feedback to students and reduce repetition in tests.

Depending on its success, the test will provide feedback by throwing errors in the workflow.

### Testing Webpages with Cypress
[Cypress](https://www.cypress.io/) is a powerful testing tool that allows a simulation of a user's actions on a website.

![cypress](https://docs.cypress.io/_nuxt/img/command-log-of-dynamic-url-test.9da5db5.png)

```javascript=
describe('Testing Bunnimage', () => {
    it('Testing Week 4 Step 1', () => {
        cy.visit('bunnimage/index.html')
        cy.get('input[type="text"]').type('console.log("hi yall")')
        cy.get('input[type="button"]').click()
        cy.get('#output').contains('console.log("hi yall")❤️')
    })
})
```
A simple [example of a test](https://github.com/bitprj/Intro-To-Serverless/blob/main/.bit/tests/cypress/integration/4.1.spec.js) as shown here simulates typing `console.log("hi yall")`, clicking the specified button on the page, and checking to see if the output equals `console.log("hi yall")❤️`. In the workflow's output, feedback from Cypress is provided.

### Testing APIs
Student coded endpoints can also be tested with HTTP requests run in the workflow. The key to this method is asking students to add "repository secrets" **that can then be accessed during the workflow** as environment variables using the below syntax.
```yaml=
      - name: Step 12
        if: ${{steps.vars.outputs.count == 12 && github.event.head_commit.message != 'Update progress'}}
        env:
          MORSE_ENDPOINT: ${{ secrets.MORSE_ENDPOINT }}
        run: |
          npm install node-fetch
          node .bit/tests/test.1.8.js --repo=${{github.event.repository.name}} --user=${{github.repository_owner}}
```
This example accesses the student's `MORSE_ENDPOINT` secret (this can be later accessed in the code with `uri = process.env.MORSE_ENDPOINT`).

```javascript=
try {
    const resp = await fetch(uri + "&plaintext=ilovebitproject", {
        method: 'GET'
    });
    var data = await resp.text()
    let test = JSON.stringify(data)
    functions.validateResponseStatus(resp, uri)
} catch (e) {
    console.error("We're having trouble making a request to your endpoint. Try again?")
    await functions.throwError("We're having trouble making a request to your endpoint. Try again?", user, repo)
    process.exit(1)
}
```
First, utilizing the student's endpoint, an HTTP GET request is made to see if the endpoint is alive.
```javascript=
if (data.length < 3) {
    console.error("No response... Try again!")
    await functions.throwError("No response... Try again!", user, repo)
    process.exit(1)
} else if (data == answer) {
    console.info("Yay!🎉 Success - thanks for helping us on this top secret mission. Welcome to the team.")
    console.info(`We got "${answer}" with the input of ilovebitproject`)
} else {
    console.error(`YIKES! We got "${data}" instead of "${answer}". Try again!`)
    await functions.throwError(`YIKES! We got '${data}' instead of '${answer}'. Try again!`, user, repo)
    process.exit(1)
}
```
If a response is received, the test attempts to determine if the output of the endpoint matches the correct "answer."

As demonstrated in [this test](https://github.com/bitprj/Intro-To-Serverless/blob/main/.bit/tests/test.1.8.js) shown above, multiple test cases can be at once with feedback returned to the student as output of the workflow.

## Conclusion
GitHub Actions are **extremely versatile** as they allow developers to create their own environment, automating tasks and running scripts. Running on repositories, GitHub Actions can easily be incorporated with other services on GitHub: from bots, to branches, and to pull requests.

This use case demonstrates the power of GitHub Actions beyond automating deployment; *extending into the area of learning and education*.