Specification-Based Testing: Devise test cases
Tags: java, testing, programming, tutorial
Published: 2024-05-21 20:51:09

This is the last part of the specification-based testing series. Now we are going to put all of the knowledge that we gained from the other steps and start creating our tests efficiently.

If you followed every step, you already have questioned and prepared the scenarios that will help catch as many faults in your code and make sure everything is working as planned.

## Let’s recap:

This is the code that we are looking forward to testing (ADD with carry flag, 8XY4 opcode from the CHIP-8 emulator):

```java
// 8XY4
// ADD
int registerxIndex = ((instruction & 0x0F00) >> 8);
int registeryIndex = ((instruction & 0x00F0) >> 4);

int firstToSum = registers[registerxIndex];
int secondRegisterToSum = registers[registeryIndex];

int registersSum = (firstToSum + secondRegisterToSum);
registers[registerxIndex] = registersSum & 0xFF;
if (registersSum > 255) registers[registers.length - 1] = 1;
else registers[registers.length - 1] = 0;
pc += 2;
```

To make it easier to devise the tests, I’ll isolate this part of the code and put it in a separate class and method to make it easy to exemplify. In the real project, this is part of a CPU class that has the registers array, the program counter, etc, and a decode function to decode the instructions coming from RAM (I’ll be showing the process of creating an emulator in the future in this blog). 

Here’s the simplified version of the code we are going to test:

```java
public class AddWithCarry {

    private int[] registers = new int[16];
    private int pc = 0;

    public void addWithCarry(int instruction) {

        if( ((instruction & 0xF000)>>12) == 8 && (instruction & 0x000F) == 4) {
            int registerxIndex = ((instruction & 0x0F00) >> 8);
            int registeryIndex = ((instruction & 0x00F0) >> 4);

            int firstToSum = registers[registerxIndex];
            int secondRegisterToSum = registers[registeryIndex];

            int registersSum = (firstToSum + secondRegisterToSum);
            registers[registerxIndex] = registersSum & 0xFF;
            if (registersSum > 255) registers[registers.length - 1] = 1;
            else registers[registers.length - 1] = 0;
            pc += 2;
        }
    }

    public void setRegister(int position, int value) {
        this.registers[position] = value;
    }

    public int getPc() {
        return pc;
    }
}
```

### Step 1: Understanding the requirements

As we discussed at this step you’ll use the specification to come up with our first scenarios. By doing that we already figure out some scenarios that we can create to make sure that our code follows the behavior specified. This is mostly the successful cases.

- **Scenario 1 - Sum without carry:**
    - Store a value in a register X (120 for example).
    - Store a value in a register Y (10 for example).
    - The two values sum must not be higher than 255.
    - Set the program counter to 0.
    - Check if the sum result value in the register X is correct (in this scenario the sum will be 130).
    - Check if the carry flag is 0.
    - Check if the register Y was not changed.
    - Check if the program counter was incremented by two.
- **Scenario 2 - Sum with carry:**
    - Store a value in a register X (235, for example).
    - Store a value in a register Y (30, for example).
    - Set the program counter to 0.
    - The two values sum must be higher than 255.
    - Check if the sum result value in the register X is correct (9 in this example). Since we had an overflow of the 255 max value, we store only the 8-bit part, since 265 is a 9-bit number (100001001), we store only the 8-bit part in X which will be 00001001 = 9.
    - The 9th bit that overflows will be set in the carry flag.
    - Check if the carry flag is 1.
    - Check if the register Y was not changed.
    - Check if the program counter was incremented by two.
- **Scenario 3 - The other registers cant affect X and Y:**
    - Store 0 in X and Y.
    - Store values in registers that are not X and Y.
    - Set the program counter to 0.
    - Check if the X remains 0.
    - Check if Y remains 0.
    - Check if the carry is 0.
    - Check if the program counter was updated by two.

### Step 2: Explore the program

In the second step, we read the code. So we explored the program we found some answers about our code. The second step is for you to understand the requirements and specifications by reverse engineering your code, so you can come up with some scenarios as described above.

### Step 3: Explore partitions

In the third step, we identify some scenarios that could break our code. Some possible unexpected behaviors. The key here is to focus on what’s possible to happen so we avoid over-testing scenarios that could never happen.

In this step, we found some possible unexpected inputs like:

- Scenario 1 - The instruction can be 0:
    - Set instruction to 0.
    - Check if register X has not changed.
    - Check if register Y has not changed.
    - Check if the program counter has not changed.
- Scenario 2 - Invalid instruction:
    - Set instruction to an invalid value.
    - Check if register X has not changed.
    - Check if register Y has not changed.
    - Check if the program counter did not change

In this step, we could pick some more scenarios, but to keep it short, I’ll keep these examples based only on the instruction partition. But make sure to create tests for all the other partitions of your code that can assume the expected value.

### Step 4: Analyze boundaries

Last but not least, we get the scenarios we found checking the boundaries, we look at parts where one simple mistake can cause a bug with some specific values. And we found these scenarios:

- Scenario 1 -Test the exact boundary value:
    - in this scenario, since we are looking at a sum, we can make that the sum of X and Y will be exactly 255.
    - Check if the carry is 0.
- Scenario 2 - The boundary minus one:
    - in this scenario, since we are looking at a sum, we can make that the sum of X and Y will be exactly 254.
    - Check if the carry is 0.
- Scenario 3 - The boundary plus one:
    - in this scenario, since we are looking at a sum, we can make that the sum of X and Y will be exactly 256.
    - Check if the carry is 1.

### Step 5: Devise test cases

Now we implement the scenarios that we came up with by doing the other steps of the specification-based techniques:

```java
package org.example;

import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.*;

public class AddWithCarryTest {

    @Test
    public void testSumWithoutCarry() {
        // Scenario 1: Sum without carry
        AddWithCarry addWithCarry = new AddWithCarry();
        addWithCarry.setRegister(1, 120); // Register X
        addWithCarry.setRegister(2, 10);  // Register Y
        int instruction = 0x8124;         // Add register Y to register X with carry

        addWithCarry.addWithCarry(instruction);

        assertEquals(130, addWithCarry.getRegisterValue(1));
        assertEquals(0, addWithCarry.getRegisterValue(15)); // Carry flag should be 0
        assertEquals(10, addWithCarry.getRegisterValue(2)); // Register Y should not change
        assertEquals(2, addWithCarry.getPc());
    }

    @Test
    public void testSumWithCarry() {
        // Scenario 2: Sum with carry
        AddWithCarry addWithCarry = new AddWithCarry();
        addWithCarry.setRegister(1, 235); // Register X
        addWithCarry.setRegister(2, 30);  // Register Y
        int instruction = 0x8124;         // Add register Y to register X with carry

        addWithCarry.addWithCarry(instruction);

        assertEquals(9, addWithCarry.getRegisterValue(1));  // 235 + 30 = 265, only 8-bit part stored
        assertEquals(1, addWithCarry.getRegisterValue(15)); // Carry flag should be 1
        assertEquals(30, addWithCarry.getRegisterValue(2)); // Register Y should not change
        assertEquals(2, addWithCarry.getPc());
    }

    @Test
    public void testOtherRegistersUnaffected() {
        // Scenario 3: Other registers don't affect X and Y
        AddWithCarry addWithCarry = new AddWithCarry();
        addWithCarry.setRegister(1, 0);   // Register X
        addWithCarry.setRegister(2, 0);   // Register Y
        addWithCarry.setRegister(3, 50);  // Another register
        int instruction = 0x8124;         // Add register Y to register X with carry

        addWithCarry.addWithCarry(instruction);

        assertEquals(0, addWithCarry.getRegisterValue(1));
        assertEquals(0, addWithCarry.getRegisterValue(2));
        assertEquals(50, addWithCarry.getRegisterValue(3)); // Other registers should not change
        assertEquals(0, addWithCarry.getRegisterValue(15)); // Carry flag should be 0
        assertEquals(2, addWithCarry.getPc());
    }

    @Test
    public void testZeroInstruction() {
        // Invalid scenario: instruction is 0
        AddWithCarry addWithCarry = new AddWithCarry();
        addWithCarry.setRegister(1, 50);
        addWithCarry.setRegister(2, 20);
        int instruction = 0x0000; // Invalid instruction

        addWithCarry.addWithCarry(instruction);

        assertEquals(50, addWithCarry.getRegisterValue(1));
        assertEquals(20, addWithCarry.getRegisterValue(2));
        assertEquals(0, addWithCarry.getPc()); // PC should not change
    }

    @Test
    public void testInvalidInstruction() {
        // Invalid scenario: instruction is not added with carry
        AddWithCarry addWithCarry = new AddWithCarry();
        addWithCarry.setRegister(1, 50);
        addWithCarry.setRegister(2, 20);
        int instruction = 0x9123; // Invalid instruction for add with carry

        addWithCarry.addWithCarry(instruction);

        assertEquals(50, addWithCarry.getRegisterValue(1));
        assertEquals(20, addWithCarry.getRegisterValue(2));
        assertEquals(0, addWithCarry.getPc()); // PC should not change
    }

    @Test
    public void testExactBoundary() {
        // Boundary scenario: exact 255 sum
        AddWithCarry addWithCarry = new AddWithCarry();
        addWithCarry.setRegister(1, 200);
        addWithCarry.setRegister(2, 55);
        int instruction = 0x8124; // Add register Y to register X with carry

        addWithCarry.addWithCarry(instruction);

        assertEquals(255, addWithCarry.getRegisterValue(1));
        assertEquals(0, addWithCarry.getRegisterValue(15)); // Carry flag should be 0
        assertEquals(2, addWithCarry.getPc());
    }

    @Test
    public void testBoundaryMinusOne() {
        // Boundary scenario: sum 254
        AddWithCarry addWithCarry = new AddWithCarry();
        addWithCarry.setRegister(1, 200);
        addWithCarry.setRegister(2, 54);
        int instruction = 0x8124; // Add register Y to register X with carry

        addWithCarry.addWithCarry(instruction);

        assertEquals(254, addWithCarry.getRegisterValue(1));
        assertEquals(0, addWithCarry.getRegisterValue(15)); // Carry flag should be 0
        assertEquals(2, addWithCarry.getPc());
    }

    @Test
    public void testBoundaryPlusOne() {
        // Boundary scenario: sum 256
        AddWithCarry addWithCarry = new AddWithCarry();
        addWithCarry.setRegister(1, 200);
        addWithCarry.setRegister(2, 56);
        int instruction = 0x8124; // Add register Y to register X with carry

        addWithCarry.addWithCarry(instruction);

        assertEquals(0, addWithCarry.getRegisterValue(1));  // 256 wraps around to 0
        assertEquals(1, addWithCarry.getRegisterValue(15)); // Carry flag should be 1
        assertEquals(2, addWithCarry.getPc());
    }
}

```

And that’s it! Now you have a good amount of scenarios that will help you guarantee the proper behavior of your code according to what was specified.

### **Want to know more about this topic?**

In the following days, I’ll be talking about structural-based tests. Learn how to look at your code structure to come up with some test scenarios!

Stay tuned to learn more! Don't miss out!

[Willian Moya (@WillianFMoya) / X (twitter.com)](https://twitter.com/WillianFMoya)

[Willian Ferreira Moya | LinkedIn](https://www.linkedin.com/in/willianmoya/)

Follow me here on dev.to!