Simple blackbox testing with Raku and Sparrow6
Tags: raku, rakulang, sparrow6, testing
Published: 2019-12-24 21:29:03

---
title: Simple blackbox testing with Raku and Sparrow6
published: true
description: How to write blackbox test using Sparrow6 Automation Framework
tags: Raku, Rakulang, Sparrow6, Testing
---


[Sparrow6](https://github.com/melezhik/Sparrow6/) is a Raku automation framework with embedded tool that allows to create blackbox testing scenarios.

The feature is called [Task Checks DSL](https://github.com/melezhik/Sparrow6/blob/master/documentation/taskchecks.md).

[Blackbox](https://en.wikipedia.org/wiki/Black-box_testing) testing is based on the idea of agnosticism of internal structure of an object being tested. Sparrow6 takes a very simple approach allows one to create regexp based rules to analyze external scripts output. 

Following are some examples.

---

# Plain string search

Just create a _task check_ file that includes a searched string 


`task.bash`

```!bash
!bash
echo "Hello World"
```

`task.check`

```
Hello
```

Test run:

`perl6 -MSparrow6::DSL -e task-run` 

Output:

```
20:01:06 12/23/2019 [/home/melezhik/projects/blackbox-testing/plain-string] Hello World
[task check] stdout match <Hello> True
```

# Regexp search
  
Sparrow6 allows to use Raku regular expressions to search in script output:


`task.check`

```
regexp: ^^ Hello \s+ World
```

Test run:

`perl6 -MSparrow6::DSL -e task-run` 


Output:

```
20:03:20 12/23/2019 [/home/melezhik/projects/blackbox-testing/regexp] Hello World
[task check] stdout match <^^ Hello \s+ World> True
```

# Sequential search 

Sometime one need to test if a _sequence_ of lines appears in a script output. `begin:`, `end:` notation is used to check sequences:

`task.bash`

```!bash
!bash
echo "ON"
echo 1
echo 2
echo 3
echo "OFF"
```

`task.check`

```
begin:
  ON
  1
  2
  3
  OFF
end
```

Test run:

`perl6 -MSparrow6::DSL -e task-run`


Output:

```
20:34:06 12/23/2019 [/home/melezhik/projects/blackbox-testing/sequential] ON
20:34:06 12/23/2019 [/home/melezhik/projects/blackbox-testing/sequential] 1
20:34:06 12/23/2019 [/home/melezhik/projects/blackbox-testing/sequential] 2
20:34:06 12/23/2019 [/home/melezhik/projects/blackbox-testing/sequential] 3
20:34:06 12/23/2019 [/home/melezhik/projects/blackbox-testing/sequential] OFF
[task check] stdout match (s) <ON> True
[task check] stdout match (s) <1> True
[task check] stdout match (s) <2> True
[task check] stdout match (s) <3> True
[task check] stdout match (s) <OFF> True
```

If switch first two digits, the test will fail:

`task.bash`

```!bash
!bash
echo "ON"
echo 2
echo 1
echo 3
echo "OFF"
```


Test run:

`perl6 -MSparrow6::DSL -e task-run`


Output:

```
20:37:22 12/23/2019 [/home/melezhik/projects/blackbox-testing/sequential] ON
20:37:22 12/23/2019 [/home/melezhik/projects/blackbox-testing/sequential] 2
20:37:22 12/23/2019 [/home/melezhik/projects/blackbox-testing/sequential] 1
20:37:22 12/23/2019 [/home/melezhik/projects/blackbox-testing/sequential] 3
20:37:23 12/23/2019 [/home/melezhik/projects/blackbox-testing/sequential] OFF
[task check] stdout match (s) <ON> True
[task check] stdout match (s) <1> False
[task check] stdout match (s) <2> False
[task check] stdout match (s) <3> False
[task check] stdout match (s) <OFF> False
=================
TASK CHECK FAIL
```

# Range search

If one doesn't care about lines order and need to check
that lines _are included between_ certain lines, it's possible to reshape the previous example using _range_ notation: `between {'RexExp1'} {'RegExp2'}`, where _RexExp1_ sets a regexp pattern for the very first string in the range, and the second one does the same for the very last string in the range, all other strings should appear _within_ the range in no particular order.


`task.check`

```
between: {'ON'} {'OFF'}
  1
  2
  3
end:
```

Test run:

`perl6 -MSparrow6::DSL -e task-run`


Output:

```
20:49:56 12/23/2019 [/home/melezhik/projects/blackbox-testing/ranges] ON
20:49:56 12/23/2019 [/home/melezhik/projects/blackbox-testing/ranges] 2
20:49:56 12/23/2019 [/home/melezhik/projects/blackbox-testing/ranges] 1
20:49:56 12/23/2019 [/home/melezhik/projects/blackbox-testing/ranges] 3
20:49:56 12/23/2019 [/home/melezhik/projects/blackbox-testing/ranges] OFF
[task check] stdout match (r) <1> True
[task check] stdout match (r) <2> True
[task check] stdout match (r) <3> True
```


# Dynamic search

Sparrow6 equips one with an ability to generate search criteria in run time, using a variety of programming languages ( Bash,Perl,Python,Ruby,Raku,Powershell)


Let's reshape the last example using `generator:` notation:


`task.check`

```
generator: <<CODE
!perl6

  say "between: \{'ON'\} \{'OFF'\}\n",(1 ... 3).join("\n"), "\nend:";

CODE
```

Changing shebang we choose different language to implement dynamic search.

Let's do implementation on Bash.


`task.check`

```
generator: <<CODE
!bash

  echo "between: {'ON'} {'OFF'}"
  for i in {1..3}; do echo $i; done
  echo 'end:'

CODE
``` 

# Captures

Captures allow to extract certain chunks from searched strings and process them.


Let's count a total sum for all numbers included between "ON" and "OFF" strings:


`task.check`

```
between: {'ON'} {'OFF'}
 regexp: (\d+)
end:

code: <<CODE
!perl6

my $total;
for captures()<> -> $c {
 $total+=$c[0]
}

update_state(%( cnt => $total ))

CODE
```

Test run:

`perl6 -MSparrow6::DSL -e 'my %state = task-run; say %state.perl'`


Output:

```
23:13:23 12/23/2019 [/home/melezhik/projects/blackbox-testing/captures] ON
23:13:23 12/23/2019 [/home/melezhik/projects/blackbox-testing/captures] 2
23:13:23 12/23/2019 [/home/melezhik/projects/blackbox-testing/captures] 1
23:13:23 12/23/2019 [/home/melezhik/projects/blackbox-testing/captures] 3
23:13:23 12/23/2019 [/home/melezhik/projects/blackbox-testing/captures]
OFF
[task check] stdout match (r) <(\d+)> True
{:cnt(6)}
```

# Asserts

Asserts allow to set predicates - statements that return `true` of `false` and create related checks.

In previous example let's check if total sum of numbers is equal `6`:
 

`task.check`

```
between: {'ON'} {'OFF'}
 regexp: (\d+)
end:

generator: <<CODE
!perl6

my $total;
for captures()<> -> $c {
 $total+=$c[0]
}

say "assert: { $total == 6  }", " total sum is 6";

CODE
```

Test run:

`perl6 -MSparrow6::DSL -e task-run`


Output:

```
19:28:12 12/24/2019 [/home/melezhik/projects/blackbox-testing/asserts] ON
19:28:12 12/24/2019 [/home/melezhik/projects/blackbox-testing/asserts] 1
19:28:12 12/24/2019 [/home/melezhik/projects/blackbox-testing/asserts] 2
19:28:12 12/24/2019 [/home/melezhik/projects/blackbox-testing/asserts] 3
19:28:12 12/24/2019 [/home/melezhik/projects/blackbox-testing/asserts] OFF
[task check] stdout match (r) <(\d+)> True
[task check] <total sum is 6> True
```


# Passing parameters

One can pass parameters handled in dynamic search constructions.


`task.check`

```
between: {'ON'} {'OFF'}
 regexp: (\d+)
end:

generator: <<CODE
!perl6

my $amount = config()<amount>;
 
my $total;
for captures()<> -> $c {
 $total+=$c[0]
}

say "assert: { $total == $amount  }", " total sum is amount";
CODE
```

Output:

```
perl6 -MSparrow6::DSL -e 'task-run("{$*CWD}",%( amount => 10 ))'
19:34:46 12/24/2019 [/home/melezhik/projects/blackbox-testing/parameters] ON
19:34:46 12/24/2019 [/home/melezhik/projects/blackbox-testing/parameters] 1
19:34:46 12/24/2019 [/home/melezhik/projects/blackbox-testing/parameters] 2
19:34:46 12/24/2019 [/home/melezhik/projects/blackbox-testing/parameters] 3
19:34:46 12/24/2019 [/home/melezhik/projects/blackbox-testing/parameters] OFF
[task check] stdout match (r) <(\d+)> True
[task check] <total sum is 10> False
=================
TASK CHECK FAIL
```

# Streams

The last interesting feature to be mentioned here is called _streams_.
Steams allow one to iterate over _blocks_ of data found during sequential or range search.

In the last example consider the case when we have more then one block of numbers between _'ON'_ and _'OFF'_ delimiters.

`task.bash`

```bash

echo "ON"
echo 1
echo 2
echo 3
echo "OFF"

echo "ON"
echo 3
echo 3
echo 3
echo "OFF"

echo "ON"
echo 2
echo 2
echo 2
echo "OFF"
```

Let's write an iterator for all the blocks found: 


```
between: {'ON'} {'OFF'}
 regexp: (\d+)
end:

code: <<CODE
!perl6

  my $stream-num = 1;
  for streams_array()<> -> $s {
    for $s<> -> $i {
      say "stream#{$stream-num}: {$i[0]}";
    }
    $stream-num++;
  }

CODE
```

Test run:

`perl6 -MSparrow6::DSL -e 'task-run`

Output:

```
19:50:43 12/24/2019 [/home/melezhik/projects/blackbox-testing/streams] ON
19:50:43 12/24/2019 [/home/melezhik/projects/blackbox-testing/streams] 1
19:50:43 12/24/2019 [/home/melezhik/projects/blackbox-testing/streams] 2
19:50:43 12/24/2019 [/home/melezhik/projects/blackbox-testing/streams] 3
19:50:43 12/24/2019 [/home/melezhik/projects/blackbox-testing/streams] OFF
19:50:43 12/24/2019 [/home/melezhik/projects/blackbox-testing/streams] ON
19:50:43 12/24/2019 [/home/melezhik/projects/blackbox-testing/streams] 3
19:50:43 12/24/2019 [/home/melezhik/projects/blackbox-testing/streams] 3
19:50:43 12/24/2019 [/home/melezhik/projects/blackbox-testing/streams] 3
19:50:43 12/24/2019 [/home/melezhik/projects/blackbox-testing/streams] OFF
19:50:43 12/24/2019 [/home/melezhik/projects/blackbox-testing/streams] ON
19:50:43 12/24/2019 [/home/melezhik/projects/blackbox-testing/streams] 2
19:50:43 12/24/2019 [/home/melezhik/projects/blackbox-testing/streams] 2
19:50:43 12/24/2019 [/home/melezhik/projects/blackbox-testing/streams] 2
19:50:43 12/24/2019 [/home/melezhik/projects/blackbox-testing/streams] OFF
[task check] stdout match (r) <(\d+)> True
[task check] stream#1: 1
[task check] stream#1: 2
[task check] stream#1: 3
[task check] stream#2: 3
[task check] stream#2: 3
[task check] stream#2: 3
[task check] stream#3: 2
[task check] stream#3: 2
[task check] stream#3: 2
```

# Conclusion

[Task Checks DSL](https://github.com/melezhik/Sparrow6/blob/master/documentation/taskchecks.md) is a simple, yet powerful tool allow one to write blackbox tests. The feature is a part of Sparrow6 core and provided out of the box.

---

Thank you for reading and Merry Christmas!