Discovered new tools.
Tags: opensource
Published: 2022-04-07 00:29:03

## [Oauth2-proxy](https://github.com/oauth2-proxy/oauth2-proxy)
It's a reverse proxy and static files server that provides authentication in the middle of a user request. 
In telescope, I had to use `oauth2-proxy` and `nginx` to secure the route for Supabase studio.
[Add studio under our subdomain #3098](https://github.com/Seneca-CDOT/telescope/pull/3098)

### Process

#### Configure our docker-compose for staging/production

```yml
# production.yml
  nginx:
    depends_on:
      - oauth2-proxy

  oauth2-proxy:
    image: bitnami/oauth2-proxy:7.2.1
    container_name: 'oauth2-proxy'
    command:
      [
        '--provider=github',
        '--cookie-secure=true',
        '--cookie-secret=${OAUTH2_SUPABASE_COOKIE_SECRET}',
        '--upstream=http://studio:3000',
        '--http-address=0.0.0.0:8080',
        '--reverse-proxy=true',
        '--email-domain=*',
        '--github-org=Seneca-CDOT',
        '--github-team=telescope-admins',
        '--client-id=${OAUTH2_SUPABASE_CLIENT_ID}',
        '--client-secret=${OAUTH2_SUPABASE_CLIENT_SECRET}',
      ]
    depends_on:
      - studio
```

- Set `nginx` services to depend on `oauth2-proxy`. So we let `oauth2-proxy` start first before starting `nginx`.

- Declare our `oauth2-proxy` services to use image `bitnami/oauth2-proxy:7.2.1`, we give the following command.
  - `--provider=github`, this tell to use GitHub Oauth services [see](https://oauth2-proxy.github.io/oauth2-proxy/docs/configuration/oauth_provider#github-auth-provider).
  - `--cookie-secure=true`, Set our cookie to be secure.
  - `--cookie-secret=${OAUTH2_SUPABASE_COOKIE_SECRET}`, set a long seed string for secure cookies.
  - `--upstream=http://studio:3000`, set the upstream to be redirected to after authorized, in our case it would be Supabase studio.
  - `--http-address=0.0.0.0:8080`, listen on for HTTP clients.
  - `--reverse-proxy=true`, set reverse-proxy to be true since we are putting our `oauth2-proxy` under `nginx`. It helps to automatically set a redirect route.
  - `--email-domain=*`, this is for restricting certain email domains, but for us, we allow any domain.
  - `--github-org=Seneca-CDOT`, this is for restricting by GitHub organization, for use we allow only member that is in Seneca-CDOT.
  - `--github-team=telescope-admins`, this is for restricting within an organization to specific teams,  for use we allow only member that is in Seneca-CDOT organization and telescope-admins teams.
  - `--client-id=${OAUTH2_SUPABASE_CLIENT_ID}`, this is our GitHub Oauth client id [see](https://docs.github.com/en/developers/apps/building-oauth-apps/authorizing-oauth-apps).
  - `--client-secret=${OAUTH2_SUPABASE_CLIENT_SECRET}`, this is our GitHub Oauth client secret [see](https://docs.github.com/en/developers/apps/building-oauth-apps/authorizing-oauth-apps).


#### Configure our nginx config for staging/production

```
# nginx.conf.template
  server {
    listen 443 ssl http2;
    server_name ${SUPABASE_HOST};

    location / {
      proxy_set_header Connection "";
      proxy_set_header Host $host;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Scheme $scheme;
      proxy_pass http://oauth2-proxy:8080/;
    }
  }
```
In this PR [#2964](https://github.com/Seneca-CDOT/telescope/issues/2964), we added the DNS and SSL.
- In our nginx config, we declare a new server with a server name of our `${SUPABASE_HOST}` and listen to port 443.

- Set the location to be `/`, set the proxy to be our `oauth2-proxy` services. 

- Set the the proxy header `proxy_set_header`, `proxy_set_header`, `proxy_set_header`, this will give our `oauth2-proxy` the right information to set up the redirect route.


## [Hurl](https://hurl.dev/)

[Hurl](https://github.com/Orange-OpenSource/hurl) is an open-source command-line tool that runs HTTP requests defined in a simple plain text format.
The sources code is written in Rust which makes this tool very fast to run.
It's made by a French Telecom company called [Orange](https://www.orange.com/en), 


### Linux installation

```bash
curl -LO https://github.com/Orange-OpenSource/hurl/releases/download/1.6.1/hurl_1.6.1_amd64.deb
sudo dpkg -i hurl_1.6.1_amd64.deb
```

### To check if it's installed 

```
hurl -h
```

### Scenario to test

Note: any POST, PUT, DELETE, and GET request needs user to be authenticated. For that let's assume our API uses basic auth and the username is `user1@email.com` and the password is `password1`. For the authentication, we use a base64 which can be generated with the following code below.
```js
console.log(btoa('user1@email.com:password1'))
# Output: dXNlcjFAZW1haWwuY29tOnBhc3N3b3JkMQ==
```

1. POST sending a file, it should return a status code of 201, header Locations of the URL to get the data, and JSON body containing an id based on nanoid, create date in string, update date in string
2. GET at URL of the location previously returned in POST, and compare if the data returned matches the content posted.
3. PUT at URL of the location previously returned in POST, and update the content, it should return a status code of 200, and JSON body containing an id based on nanoid, create date in string, newest update date in string (should be greater than create date).
4. GET at URL of the location previously returned in POST, and compare if the data returned matches the content updated.
5. DELETE at URL of the location previously returned in POST, it should return a status code of 200

#### Folder structure 

```plain
.
├── .github
│   ├── workflows
│   │   └── ci.yml
├── tests
│   ├── fixtures
│   │   └── data.txt
│   ├── integration
│   │   ├── myTest.hurl
├── docker
│   ├── docker-compose.yml
├── Dockerfile
```

#### Test case file

~~~plain
# myTest.hurl

# 1. POST request 
# Make a POST and send our txt file located in the fixtures folder
# Set the content type to be a text/plain
POST http://localhost:8080/cat
Content-Type: text/plain
# https://hurl.dev/docs/request.html#file-body
file,./fixtures/data.txt;
# Set the Authorization to be a Basic auth and our base64 credential
Authorization: Basic dXNlcjFAZW1haWwuY29tOnBhc3N3b3JkMQ==

# Check the response to be 201
HTTP/1.1 201

# https://hurl.dev/docs/asserting-response.html
[Asserts]
# Assert header Location to contain the URL for getting our data
header "Location" matches "^http:\/\/localhost:8080\/cat\/[A-Za-z0-9_-]+$"

# Assert id to matches id generated with nanoid https://www.npmjs.com/package/nanoid
jsonpath "$.id" matches "^[A-Za-z0-9_-]+$"

# Assert created to return a string
jsonpath "$.created" isString

# Assert created to return a string
jsonpath "$.updated" isString

# https://hurl.dev/docs/capturing-response.html#captures
[Captures]
# Capture the header location value and store to url
# This value can be call later in the code.
url: header "Location"
# Capture the id from the returned json and store cat_id
cat_id: jsonpath "$.id"
# Capture the created date from the returned json and store cat_created
cat_created: jsonpath "$.created"
# Capture the updated date from the returned json and store cat_updated
cat_updated: jsonpath "$.updated"

# 2. GET request 
# GET at the url previously captured
GET {{url}}
# Set the Authorization to be a Basic auth and our base64 credential
Authorization: Basic dXNlcjFAZW1haWwuY29tOnBhc3N3b3JkMQ==

# Check the response to be 200
HTTP/1.1 200
# Check the response content type to be text/plain
Content-Type: text/plain
# Check the response body to match the content of data.txt.
file,./fixtures/data.txt;

# 3. PUT request
# PUT at the url previously captured
PUT {{url}}
# Set the Authorization to be a Basic auth and our base64 credential
Authorization: Basic dXNlcjFAZW1haWwuY29tOnBhc3N3b3JkMQ==
Content-Type: text/plain
# Set the request body with our new content 
# https://hurl.dev/docs/request.html#raw-string-body
```New content```

# Check the response to be 200
HTTP/1.1 200

[Asserts]
# Assert id to be the same as cat_id (id captured in the POST return)
jsonpath "$.id" == {{cat_id}}

# Assert created to be the same as cat_created (created captured in the POST return)
jsonpath "$.created" == {{cat_created}}

# Assert updated to be different as cat_updated (updated captured in the POST return)
jsonpath "$.updated" != {{cat_updated}}

# 4. GET request 
# GET at the url previously captured
GET {{url}}
# Set the Authorization to be a Basic auth and our base64 credential
Authorization: Basic dXNlcjFAZW1haWwuY29tOnBhc3N3b3JkMQ==

# Check the response to be 200
HTTP/1.1 200
# Check the response content type to be text/plain
Content-Type: text/plain
# Check the response body to match the new content updated from the PUT request
```New content```

# 5. DELETE request 
# DELETE at the url previously captured
DELETE {{url}}
# Set the Authorization to be a Basic auth and our base64 credential
Authorization: Basic dXNlcjFAZW1haWwuY29tOnBhc3N3b3JkMQ==

# Check the response to be 200
HTTP/1.1 200
~~~

#### To run the test case. See [how to run](https://hurl.dev/docs/running-tests.html) and [command option](https://hurl.dev/docs/man-page.html#options)

Let's assume we are in the root of our folder.
Let's assume we have a `docker-compose.yml` that is already running our microservices and exposes at port 8080. 

```bash
hurl --test --file-root "./tests" "./tests/integration/myTest.hurl"
```
- `--test`, activate the test mode which enables `--no-output`, `--progress`, and `--summary`
- `--file-root` set our file root, we need this because we want our fixture to be accessible. 

#### Samples of response 

```bash
❯ hurl --test --file-root "./tests" "./tests/integration/myTest.hurl"
./tests/integration/myTest.hurl: RUNNING [1/1]
./tests/integration/myTest.hurl: SUCCESS
--------------------------------------------------------------------------------
Executed:  1
Succeeded: 1 (100.0%)
Failed:    0 (0.0%)
Duration:  10ms
```

#### Run on CI pipeline

Let's assume we have a `docker-compose.yml` that spins up our microservices and exposes at port 8080. 

```yml
# .github/workflows/ci.yml
name: ci
on:
  pull_request:
    branches:
      - main
  push:
    branches:
      - main

jobs:
  hurl-test:
  runs-on: ubuntu-latest
  steps:
    - name: 🏗 Setup repo
      uses: actions/checkout@v2
    - name: 📦 Install hurl
      run: |
        curl -LO https://github.com/Orange-OpenSource/hurl/releases/download/1.6.1/hurl_1.6.1_amd64.deb
        sudo dpkg -i hurl_1.6.1_amd64.deb
    - name: 🏗 Build Containers
      run: docker-compose -f docker/docker-compose.yml up -d
    - name: ✨ Run Hurl Tests
      run: hurl --test --file-root "./tests" "tests/integration/myTest.hurl"
```

#### Opinion about this new testing tool.

In overall, this is a pretty unique testing tool, it's super fast and powerful since the code base is written in rust. The learning curves are pretty simple (I managed to learn the basics in a couple of hours), less setup todo since it's just a plain file, the syntax is English friendly, besides the [jsonPath](https://goessner.net/articles/JsonPath/) that could take some times to adapt.

There is some limitation, like for the assert, sometimes it's not that easy to compare an object to an object or read from an object since when you read a capture of an object, it tries to construct into a string so you are not able to use jsonPath to get some property.

Thanks to [@humphd](https://github.com/humphd) who showed us this tool.  And I'm happy because it's a French company that made this wonderful open-source tool.