A serverless full-stack application using only git, google drive, and public ci/cd runners
Tags: serverless, showdev, devops, tutorial
Published: 2021-04-16 13:40:49

**TL;DR; How I built the Vilicus Service, a serverless full-stack application with backend workers and database only using git and ci/cd runners.**

---

#### What is Vilicus?
[Vilicus](https://dev.to/edersonbrilhante/vilicus-a-overseer-for-security-scanning-of-container-images-eji) is an open-source tool that orchestrates security scans of container images(Docker/OCI) and centralizes all results into a database for further analysis and metrics.

Vilicus provides many alternatives to use it:
- [Own Installation](https://github.com/edersonbrilhante/vilicus);
- [GitHub Action](https://github.com/edersonbrilhante/vilicus-github-action) in your GitHub workflows;
- [Template CI](https://github.com/edersonbrilhante/vilicus-gitlab) in your GitLab CI/CD pipelines;
- [Free Online Service](http://vilicus.edersonbrilhante.com.br/);

This article explains how it was possible to build the Free Online Service without using a traditional deployment.

---

## Architecture

![Architecture](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/i9eolikd4iicgb3r37js.png)

The Frontend is hosted in GitHub Pages. This frontend is a landing page with a free service to scan or display the vulnerabilities in container images. 

The results of container image scans are stored in a GitLab Repository.

When the user asks to show the results from an image, the frontend consumes the GitLab API to retrieve the file with vulns from this image. In case this image is not scanned yet, the user has the option to schedule a scan using a google form.

When this form is filled, the data is sent to a Google Spreadsheet.

A GitHub Workflow runs every 5 minutes to check if there are new answers in this Spreadsheet. For each new image in the Spreadsheet, this workflow triggers another Workflow to scan the image and save the result in the GitLab Repository.

#### Why store in GitLab? 
GitLab provides bigger limits. 

**Here's a summary of differences in offering on public cloud and free tier:**

| | Free users | Max repo size (GB) | Max file size (MB) | Max API calls per hour (per client) |
|:-----|:-----|:-----|:-----|:-----|
| GitHub | 3 | 2 | 100 | 5000 |
BitBucket | 5 | 1 | Unlimited (up to repo size) | 5000 |
| GitLab | Unlimited | 10 | Unlimited (up to repo size)	| 36000| 

---

### Google Drive
This choice was a "quick win". In a usual deployment, the backend could call an API passing secrets without the clients knowing the secrets. 

But because I am using GitHub Pages I cannot use that(Well, I could do it in the javascript, but anyone using the Browser Inspect would see the secrets. So let's don't do it 😉)

This makes the Google Spreadsheet perform as a Queue.

*Google Form:*
![Form](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/nyxtv3xkmk90k2i6ljhg.png)
*Google Spreadsheet with answers:*
![Answers](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ezihbao6fboqhhjkrg4l.png)

---

### GitHub Workflows
The `Schedule Workflow` runs at most every 5 minutes. This workflow executes the python script that checks if there are new rows in the `Google Spreadsheet`, and for each row is made an HTTP request to trigger the event `repository_dispatch`.

This makes the workflows perform as backend workers.

*Schedule in workflow:*
```yml
name: Schedule
on:
  schedule:
    - cron:  '*/5 * * * *'
...
```
*Event `repository_dispatch` in WorkFlow:*
```yml
name: Report
on: [repository_dispatch]
...
```
#### Screenshots:

*Schedule History:*
![Schedules](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vx0omaxnghu518cjqfqf.png)
*Schedule WorkFlow:*
![Schedule](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0zn81m0ai0je3puhr56v.png)
*Scans History:*
![Scans](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/pzhsml4624pfb2edj4u2.png)
*Report Workflow:*
![Report](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/c6p60gvam145vyajlfmu.png)
*Scan Report stored in GitLab:*
![Report File Example](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5qg7eshm699k7f22t5p1.png)

#### Source Code:
* [Schedule Workflow](https://github.com/edersonbrilhante/vilicus-report/blob/main/.github/workflows/schedule-jobs.yml)
* [Report Workflow](https://github.com/edersonbrilhante/vilicus-report/blob/main/.github/workflows/report.yml)
* [Script to upload the report file to Gitlab](https://github.com/edersonbrilhante/vilicus-report/blob/main/commit.py)
* [Script to iterate the answers and trigger new scans](https://github.com/edersonbrilhante/vilicus-report/blob/main/schedule-jobs.py)
* [GitLab Repo with report files](https://gitlab.com/edersonbrilhante/vilicus-reports-db)

#### Do you want to know more about GitHub Actions?

* [Learn GitHub Actions](https://docs.github.com/en/actions/learn-github-actions)
* [Workflow syntax](https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions)

---

### Github Pages

The Frontend is running in GitHub Pages. 

By default an application running in GH Pages is hosted as `http://<github-user>.github.io/<repository>`. 

But GitHub allows you to customize the domain, because that it's possible to access Vilicus using `https://vilicus.edersonbrilhante.com.br` instead of `http://edersonbrilhante.github.io/vilicus`.

#### GitHub Workflow to build the application and deploy it in GH Pages

*Building the source code:*
```yml
- name: Build
  run: |
    cd website
    npm install
    npm run-script build
  env:
    REACT_APP_GA_CODE: ${{ secrets.REACT_APP_GA_CODE }}
    REACT_APP_FORM_SCAN: ${{ secrets.REACT_APP_FORM_SCAN }}
```

*Deploying the build:*
```yml
- name: Deploy
  uses: JamesIves/github-pages-deploy-action@releases/v3
  with:
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
    BRANCH: gh-pages
    FOLDER: website/build
```

**Source Code:**
* [Workflow to deploy code in GitHub Pages](https://github.com/edersonbrilhante/vilicus/blob/main/.github/workflows/build-gh-pages.yml)
* [Application Source Code](https://github.com/edersonbrilhante/vilicus/tree/main/website)
* [Deployed code in GH Pages](https://github.com/edersonbrilhante/vilicus/tree/gh-pages)

#### Do you want to know more about GitHub Pages?

* [Configuring a publishing source](https://docs.github.com/en/pages/getting-started-with-github-pages/configuring-a-publishing-source-for-your-github-pages-site)
* [Configuring a custom domain](https://docs.github.com/en/pages/configuring-a-custom-domain-for-your-github-pages-site)

---

## That’s it!
In case you have any questions, please leave a comment here or ping me on [LinkedIn](https://www.linkedin.com/in/edersonbrilhante).