RASA - Testing unhappy paths
Tags: rasa, chatbot, nlp, ai
Published: 2021-04-01 08:07:21

In the [previous chapter](https://dev.to/petr7555/rasa-unhappy-paths-d49), we have written two unhappy paths for our **newsletter_form**. Now let's test that those paths work as we want them to.

```yml
# tests/test_stories.yml
# ... previous contents ...
  - story: Chitchat during email
    steps:
      - user: |
          I want to subscribe
        intent: subscribe
      - action: newsletter_form
      - active_loop: newsletter_form
      - user: |
          How's the weather?
        intent: chitchat
      - action: utter_ask_continue
      - user: |
          Yes, I do
        intent: affirm
      - action: newsletter_form
      - active_loop: null
      - action: utter_subscribed

  - story: Chitchat during frequency
    steps:
      - user: |
          I want to subscribe
        intent: subscribe
      - action: newsletter_form
      - active_loop: newsletter_form
      - user: |
          my email is [example@email.com](email)
        intent: inform_email
      - action: newsletter_form
      - user: |
          How's the weather?
        intent: chitchat
      - action: utter_ask_continue
      - user: |
          Yes, I do
        intent: affirm
      - action: newsletter_form
      - active_loop: null
      - action: utter_subscribed
```

If you run the tests, they will all pass.

But if we add another test, testing both interruption during email and interruption during frequency, we will see it fail.

![Failed test](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/einf5jc3qctrv7fbj8wy.png)

We can see why it failed in `results/failed_test_stories.yml`.

```yml
# results/failed_test_stories.yml
version: "2.0"
stories:
- story: Newsletter form two interruptions (/var/folders/ng/z4dw_m9d1bz6m08vw7nlwwwh0000gq/T/tmpqlha244q/7bc3772cad9348a589e105ce91f29c74_test_stories.yml)
  steps:
  - intent: subscribe
  - action: newsletter_form
  - active_loop: newsletter_form
  - intent: chitchat
  - action: utter_ask_continue
  - intent: affirm
  - action: newsletter_form
  - slot_was_set:
    - email: example@email.com
  - intent: chitchat
  - action: utter_ask_continue
  - intent: affirm
  - action: newsletter_form  # predicted: action_two_stage_fallback
  - active_loop: null
  - action: utter_subscribed
```

To fix this, start an interactive chat – `rasa interactive`.
<a href="https://www.loom.com/share/bee1676949b74966adda2227a33374b9"><img style="max-width:300px;" src="https://cdn.loom.com/sessions/thumbnails/bee1676949b74966adda2227a33374b9-with-play.gif"></a>

Throughout the chat, I said what the correct actions shuld be. At the end of the chat, I chose to save this chat to `data/stories.yml`.
Let's inspect the story that has been created for us.

```yml
# data/stories.yml
# ... previous content ...
  - story: interactive_story_1
    steps:
      - intent: subscribe
      - action: newsletter_form
      - active_loop: newsletter_form
      - slot_was_set:
          - requested_slot: email
      - intent: chitchat
      - action: utter_ask_continue
      - intent: affirm
      - action: newsletter_form
      - slot_was_set:
          - requested_slot: email
      - slot_was_set:
          - email: example@email.com
      - slot_was_set:
          - email: example@email.com
      - slot_was_set:
          - requested_slot: frequency
      - intent: chitchat
      - action: utter_ask_continue
      - intent: affirm
      - action: newsletter_form
      - slot_was_set:
          - requested_slot: frequency
      - slot_was_set:
          - frequency: twice a week
      - slot_was_set:
          - requested_slot: null
      - active_loop: null
      - action: utter_subscribed
```

Now let's look at new kind of steps that we have not been working with before.

This step is saying, that **email** is currently being requested:
```yml
      - slot_was_set:
          - requested_slot: email
```

This step is saying, that **email** has been filled with value **example@email.com**. We do not need this step for our test to pass.
```yml
      - slot_was_set:
          - email: example@email.com
```

This step is saying, that **frequency** has been filled with value **twice a week**. We do not need this step for our test to pass either.
```yml
      - slot_was_set:
          - frequency: twice a week
```

This step means, that no more slots are being requested.
```yml
      - slot_was_set:
          - requested_slot: null
```

You can learn more about requested_slot in my [next article](https://dev.to/petr7555/rasa-requestedslot-4ai7) or in the [documentation](https://rasa.com/docs/rasa/forms#the-requested_slot-slot).

After droping the unnecessary steps, we get a bit simpler story.

```yml
# data/stories.yml
# ... previous content ...
  - story: interactive_story_1
    steps:
      - intent: subscribe
      - action: newsletter_form
      - active_loop: newsletter_form
      - slot_was_set:
          - requested_slot: email
      - intent: chitchat
      - action: utter_ask_continue
      - intent: affirm
      - action: newsletter_form
      - slot_was_set:
          - requested_slot: email
      - slot_was_set:
          - requested_slot: frequency
      - intent: chitchat
      - action: utter_ask_continue
      - intent: affirm
      - action: newsletter_form
      - slot_was_set:
          - requested_slot: frequency
      - active_loop: null
      - action: utter_subscribed
```

I also reverted changes that have been made to `domain.yml` and `data/nlu.yml` and kept only the generated story.

If we retrain the model and run tests again, they will all pass now.

![Passing tests](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ndno9ci0wpm29gfij6hz.png)

You can learn more about testing unhappy paths in the [documentation](https://rasa.com/docs/rasa/testing-your-assistant).

In the next chapter, we will look at how to communicate with our chatbot through REST API.

Repository for this tutorial:
{% github petr7555/rasa-dev-tutorial no-readme %}

You can checkout the state of the repository at the end of this tutorial by running:
```bash
git clone --branch 06-testing-unhappy-paths git@github.com:petr7555/rasa-dev-tutorial.git
```