Test Generation for Distributed Apps Made Easy with Skyramp
Tags: kubernetes, testing, openapi, microservices
Published: 2023-10-27 20:05:25

In the ever-evolving world of software development, the importance of testing to ensure quality cannot be overstated. This rings especially true when it comes to distributed applications with loosely coupled microservices. These kinds of distributed applications bring a host of complexities that require a thoughtful approach to testing. With the advent of testing tools provided by [Skyramp](https://www.skyramp.dev/), combined with technology building blocks like [Kubernetes](https://kubernetes.io/) and [OpenAPI](https://www.openapis.org/), test generation for distributed apps is easier and more efficient than ever before. This blog will dive into the process of generating tests for your distributed applications by showing a step-by-step example using Skyramp and the aforementioned standards.

## Talking About Test Generation
Before we delve into the step-by-step walkthrough of generating tests, let's first understand why test generation is beneficial for distributed applications. Distributed applications, most often composed of various microservices, can be particularly intricate in terms of communication, dependencies, and data flow. Manually writing tests for each component can be time-consuming and prone to human error. Automated test generation can streamline this process, ensuring comprehensive coverage and robust testing.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/h0oqxp0h9urgb8x8m2e2.png)

<!-- ![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/y94ncyyleh363p4j6p9w.png) -->

<!-- ![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/s7k0p01d36ltmoyjpuzg.png) -->

OpenAPI, formerly known as Swagger, is a set of specifications that defines a standard, language-agnostic interface to RESTful APIs. It's often used to document and describe the capabilities of a microservice, including the available endpoints, input/output parameters, and response formats. This can act as the foundation for automated test generation by providing a clear and structured definition of a service.

Leveraging OpenAPI, Skyramp can analyze your microservices and automatically create test cases based on your API specifications and service dependencies. This allows you to verify that the various components of your distributed application work seamlessly together, saving you time. It should be noted that Skyramp supports other protocols for test generation like gRPC and Protocol Buffers, but we'll save that for another day.

## Generating Tests: A Step-By-Step Walkthrough
Now, let's get into the nitty-gritty of how to generate tests for your distributed applications using Skyramp. We will use our tried and true [sample-microservices repo on GitHub](https://github.com/letsramp/sample-microservices), which is an 11-tier distributed app with REST support already configured for deployment to a Kubernetes cluster. The app itself is a complete web-based e-commerce store created by Google, known as Online Boutique.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/82vmbiofu9n6wm49ojny.png)

Below is the step-by-step walkthrough to guide you along the journey. These steps were tested on Mac, but should work on Linux or Windows with a bash shell.

### Step 1: Install the Skyramp CLI
The first step is to install the Skyramp Command Line Interface (CLI). See the [Skyramp Docs](https://www.skyramp.dev/docs/) for specifics, or you can run the following command in your terminal:

```bash
bash -c "$(curl -fsSL https://skyramp.dev/installer.sh)"
```
NOTE: You will also need [Docker](https://www.docker.com/) installed to continue.

### Step 2: Clone the Repository
Next, clone the sample repository discussed above. This contains the code and configuration files necessary for your distributed application.

```bash
git clone https://github.com/letsramp/sample-microservices.git
```
### Step 3: Create a Cluster
Now, create a cluster using the Skyramp CLI. This cluster will serve as the environment in which your microservices will be deployed and tested.

```bash
skyramp cluster create -l
```
### Step 4: Change to the Demo Directory
Navigate to the directory containing the prepared REST demo project under "rest-demo".

```bash
cd sample-microservices/skyramp/rest-demo
```
### Step 5: Deploy the Cluster with Skyramp Deployer
Use the Skyramp Deployer to bring up the microservices within the cluster.

```bash
skyramp deployer up rest-app
```
The output will look like this:

```bash
ad-service-7cb787976d-l49n5                    ready
currency-service-5b966cbfb8-q7dvf              ready
email-service-745ddcd9c4-7wdvq                 ready
cart-service-56cd75c447-xzdlk                  ready
checkout-service-c5546db56-lsdtm               ready
product-catalog-service-5467566c8f-sl6wb       ready
skyramp-worker-6c8bfdf587-jr2tq                ready
payment-service-5f9b47cfd4-cw4j2               ready
recommendation-service-6f8cd558b8-dmvh8        ready
shipping-service-599947db74-8dtkn              ready
frontend-6f8fc574f7-nv76g                      ready
redis-6fc54f68c4-v6mqb                         ready
All pods are ready.
```

### Step 6: OpenAPI Configuration
In this project, there is a pre-defined OpenAPI specification file (demo.yaml). This file describes the API endpoints and their expected behavior. Take a peek at the contents of this file.

```bash
cat openapi/demo.yaml
```
Notice that the definition centers around charging a credit card. We can use this definition to test the payment service running in our cluster.

### Step 7: Generate a Test
Now comes the fun part - generating a test using the Skyramp CLI. You can specify the protocol, API schema, alias, and port for your test. Run the command below to see how this works.

```bash
skyramp tester generate \
--protocol openapi \
--api-schema openapi/demo.yaml \
--alias test-rest-demo \
--port 60000
```
This command will automatically generate test cases for your distributed application based on the provided OpenAPI schema.

### Step 8: Tune Your Test
Upon running the test generation command from the step above, Skyramp will create three files that contain your test cases and configuration. You can review and customize these files to fit your specific testing needs. The output will look something like this:

```bash
Creating test files     [##############################################################] 100 %
Successfully created/updated the test description configuration files:
	endpoints/test-rest-demo-tDPT.yaml
	scenarios/scenario-test-rest-demo-tDPT.yaml
	tests/test-test-rest-demo-tDPT.yaml
```

Let's make a few edits to the files that were generated.

1) Change the `serviceName` value in the generated "endpoint" file to use the payment service running in the cluster: 
```yaml
serviceName: payment-service
```
2) Change the `requestName` parameter in the generated "test" file to `scenarioName` and assign the value "scenario_123".
```yaml
 - scenarioName: scenario_123
```
3) Modify the generated "scenario" file to include a `scenarios` section at the end of the file that looks like this:
```yaml
scenarios:
  - name: scenario_123
    steps:
      - requestName: chargeRequest
      - asserts: requests.chargeRequest.res.transaction_id != null
```
4) Also modify generated parameters and values in the "scenario" file under the `requests` section as shown below. Many of the parameters will be auto-generated based on the OpenAPI definition, but we can improve the values.
```yaml
    - name: chargeRequest
      headers:
        Content-Type: "application/json"
      blob: |-
        {
          "amount": {
            "currency_code": "USD",
            "units": 100
          },
          "credit_card": {
            "credit_card_cvv": 100,
            "credit_card_expiration_month": 10,
            "credit_card_expiration_year": 2030,
            "credit_card_number": "4432-8015-6152-0454"
          }
        }
```

### Step 9: Run and Refine

After completing the edits from above, we are ready to run our test against the payment service. Simply run Skyramp Tester to execute the test scenario we composed.
```bash
skyramp tester start test-test-rest-demo-2VHL -n test-rest-demo
```
The output should indicate a passed test such as this:
```bash
Starting tests
Tester finished
Test test-rest-demo------
 [Status: finished] [Started at: 2023-10-26 09:32:56 PDT] [End: 2023-10-26 09:32:57 PDT] [Duration: 1s]
  - pattern0.scenario_123
    [Status: finished] [Started at: 2023-10-26 09:32:57 PDT] [Duration: 0s]
  - pattern0.scenario_123.0.chargeRequest
    [Status: finished] [Started at: 2023-10-26 09:32:57 PDT] [Duration: 0s]
    Executed: {"transaction_id":"e740f1b8-3618-4253-94ae-9be0de082f68"}
  - pattern0.scenario_123.1.assert
    [Status: finished] [Started at: N/A]
    Assert: requests.chargeRequest.res.transaction_id != null
    Passed: true
```
You can continue to fine-tune your tests by modifying the generated files. Working with these files, you can see the modular approach Skyramp takes to testing. Adjust the test cases, add custom scenarios, and configure test environments as needed to ensure comprehensive testing for your distributed applications.

## This Was Only a Test

In conclusion, automated test generation with Skyramp simplifies the testing process for distributed applications. It enhances the efficiency and reliability of testing scenarios, ensuring that your microservices work harmoniously to deliver a robust and error-free application.

By following the step-by-step walkthrough outlined in this blog, you can see how to effectively generate and customize tests for your distributed apps, saving time and ensuring the quality of your software. Happy testing with Skyramp!