rasa-model-report 1.5.0
Simple open-source Rasa command-line add-on that generates training model health reports for your projects.
License: Apache 2.0
Requires Python: >=3.8

<div align="center">
<br />
<br />
<img
    height="180"
    alt="logo"
    src="https://raw.githubusercontent.com/brunohjs/rasa-model-report/main/docs/image/logo.png"
/>
<h4>Simple open-source Rasa command-line add-on that generates training model health reports for your projects.</h4>
</div>
<hr />


<!-- Badges -->
![Version](https://img.shields.io/pypi/v/rasa-model-report)
![Python version](https://img.shields.io/static/v1?label=python&message=v3.8&color=3776AB)
![Apache 2.0 License](https://img.shields.io/static/v1?label=license&message=Apache%202.0&color=yellowgreen)
![Code coverage](https://img.shields.io/static/v1?label=coverage&message=100%&color=brightgreen)
![Stars](https://img.shields.io/github/stars/brunohjs/rasa-model-report?style=social)
![Followers](https://img.shields.io/github/followers/brunohjs?style=social)
<!-- ![Contributors](https://img.shields.io/github/contributors/brunohjs/rasa-model-report) -->
<!-- ![Code Climate maintainability](https://img.shields.io/codeclimate/maintainability/brunohjs/rasa-model-report) -->
<!-- ![Code Climate issues](https://img.shields.io/codeclimate/issues/brunohjs/rasa-model-report) -->
<!-- ![Code Climate technical debt](https://img.shields.io/codeclimate/tech-debt/brunohjs/rasa-model-report) -->
<!--  -->

## 🔍 About
[Rasa](https://rasa.com/) is the most popular open-source framework for building chat and voice-based AI assistants. The **rasa-model-report** is a unofficial Rasa add-on to facilitate the work of developers and curators of Rasa chatbots. Rasa provides a lot of valuable data that can be "faceted" and extract different information about the training model. This information makes it possible to discover problems in the training model. The **rasa-model-report** does just that, it extracts this information to be displayed more clearly in a report. You can see [this](https://github.com/brunohjs/rasa-model-report/blob/main/docs/markdown/another_sample_model_report.md) example.

<img
    height="22"
    alt="logo"
    src="https://raw.githubusercontent.com/brunohjs/rasa-model-report/main/docs/image/open_source_logo.png"
/>
***rasa-model-report** is an open-source project.*

## ☕ Donate
You can collaborate with this project donating any amount. Thanks! ☕

[!["Buy Me A Coffee"](https://www.buymeacoffee.com/assets/img/custom_images/orange_img.png)](https://www.buymeacoffee.com/brunohjs)

## 📜 Changelog
Changelog can be found [here](https://github.com/brunohjs/rasa-model-report/blob/main/CHANGELOG.md#changelog). You can also follow the [releases](https://github.com/brunohjs/rasa-model-report/releases) on Github and find planned enhancements for project in the [Project Board](https://github.com/users/brunohjs/projects/2).


## 📦 Installation

This module is distributed via [Pypi](https://pypi.org/) and is required to use **Python v3.8** or higher. To install the package, use the command:
```
pip install rasa-model-report
```


## 🚀 Execution
Before anything, is necessary to have the reports generated by the `rasa test` command. To run the program, use the command:
```
rasa-model-report
```
This command must be used in the root of your Rasa project. Otherwise, you can use `--path` parameter to pass the project path.

### Step-by-step
This is the step-by-step guide for using **rasa-model-report** in your project.
1. Go to the root folder of your Rasa project.
2. Train model on your Rasa project using `rasa train` command.
3. Run Rasa end-to-end tests using `rasa test` command.
   - This command will generate some data in json, markdown and image files in `result/` directory.
   - This data **is needed** for **rasa-model-report** to generate the report.
4. (Optional) If you want to know model NLU rating for each sentence in your project, run your project's Rasa API through the command `rasa run --enable-api`.
   - When you run **rasa-model-report**, automatically it will request NLU rating for each sentence. The result will be in the *NLU* section of the report.
   - If you don't want to use this option, just pass the parameter `--disable-nlu` or don't run Rasa API (if you don't run Rasa API, **rasa-model-report** will try to connect, after two tries it will skip this step).
5. Run **rasa-model-report** in root project.
   - If you haven't installed it, see [how to install](https://github.com/brunohjs/rasa-model-report#-installation).
6. The result will be in the `model_report.md` file generated in the project root folder.

Below, I created this video to show how to use the **rasa-model-report v1.0.0**. I used the Rasa sample project (from `rasa-init` command). In this [link](https://github.com/brunohjs/rasa-model-report/blob/main/docs/markdown/sample_model_report.md) is the generated report.

https://github.com/brunohjs/rasa-model-report/assets/26513013/953e5792-e2d8-426a-8d43-2db8941a93e4


Furthermore, I used the tool in [**Sara - the Rasa Demo Bot**](https://github.com/RasaHQ/rasa-demo). The result you can check [here](https://github.com/brunohjs/rasa-model-report/blob/main/docs/markdown/another_sample_model_report.md).

## 🦾 Rasa Version Support
Not every version of Rasa is supported. Check the table below:
|Rasa version|Supported|
|:-:|:-:|
|3.X|✅|
|2.X|✅|
|1.X|❌|
|0.X|❌|


## ⚙️ Options
There are parameters that can be used. Available options are below:

```
--actions-path TEXT     Actions path. (default: actions/ inside Rasa project
                        path)
--disable-nlu           Disable processing NLU sentences. NLU section will
                        not be generated in the report. Required Rasa API.
-e, --exclude LIST      List of utter and actions that will be exclude in
                        the E2E test coverage. Use commas to separate items.
                        Example: utter_greet,utter_goodbye,action_listen
-h, --help              Show this help message.
--model-link TEXT       Model download link. It's only displayed in the
                        report to model download.
--no-images             Generate model report without images.
--output-path TEXT      Report output path. (default: ./)
-p, --path TEXT         Rasa project path. (default: ./)
--precision INTEGER     Score precision. Used to change precision of the
                        model report overview scores. Can vary between 0 and
                        5 (default: 2)
--project-name TEXT     Rasa project name. It's only displayed in the
                        report. (default: My project)
--project-version TEXT  Project version. It's only displayed in the report
                        for project documentation.
--rasa-api TEXT         Rasa API URL. Is needed to create NLU section of
                        report. (default: http://localhost:5005)
--rasa-version TEXT     Rasa version. It's only displayed in the report for
                        project documentation.
-v, --version           Show installed rasa-model-report version.
```


### Usage examples
Some usage examples with parameters:
- Without parameters is usually used at the root of the Rasa project.
    ```
    rasa-model-report
    ```
- When you aren't at the root of the project, use `--path` parameter.
    ```
    rasa-model-report --path path/to/rasa/project
    ```
- Aren't at the root of the project and without NLU report.
    ```
    rasa-model-report --path path/to/rasa/project --disable-nlu
    ```
- Aren't at the root of the project and change the report output directory.
    ```
    rasa-model-report --path path/to/rasa/project --output-path path/to/place/report
    ```
- Aren't at the root of the project, the actions path isn't at the root project and change the report output directory.
    ```
    rasa-model-report --path path/to/rasa/project --output-path path/to/place/report --actions-path path/to/actions/path
    ```
- If you want exclude some utters and actions from the E2E test coverage.
    ```
    rasa-model-report --exclude utter_greet,action_help
    ```

## 💻 Development
The instructions for development and contributing are in the [CONTRIBUTING.md](CONTRIBUTING.md) file.


## 🐞 Bugs
Please file an issue for bugs, missing documentation, or unexpected behavior.

[See open issues](https://github.com/brunohjs/rasa-model-report/issues?q=is%3Aopen+is%3Aissue+label%3Abug)


## 💬 Discussions
Please file an issue to suggest new features. Vote on feature requests. This helps maintainers prioritize what to work on.

[See new ideas discussion](https://github.com/brunohjs/rasa-model-report/discussions/categories/ideas)


## ❓ Questions
For questions related to using the add-on, please ask the community on Q&A.

[See Q&A](https://github.com/brunohjs/rasa-model-report/discussions/categories/q-a)