Repository: asreview/asreview-insights (language: python)

Issue:
Update the README and add figures



Discussion / hints:
Great work!!
Thanks!

Fix patch:
diff --git a/.github/workflows/ci-workflow.yml b/.github/workflows/ci-workflow.yml
index 042fe6e..94df53b 100644
--- a/.github/workflows/ci-workflow.yml
+++ b/.github/workflows/ci-workflow.yml
@@ -16,35 +16,12 @@ jobs:
     - name: Lint python with flake8
       run: |
         flake8 . --max-complexity=10 --statistics
-  test-master:
-    name: test-asreview-latest
-    runs-on: ubuntu-latest
-    steps:
-    - uses: actions/checkout@v2
-      with:
-        path: asr-plot
-
-    - uses: actions/checkout@v2
-      with:
-        repository: asreview/asreview
-        path: asr-core
-    - uses: actions/setup-python@v1
-      with:
-        python-version: '3.8'
-        architecture: 'x64'
-    - name: Install packages and run tests
-      run: |
-        pip install pytest
-        pip install --upgrade setuptools>=41.0.0
-        pip install ./asr-core[all]
-        pip install ./asr-plot
-        pytest asr-plot/tests
-  test-older:
-    name: test-asreview-0-7-2
+  test-asreview:
+    name: Test Insights for ASReview LAB versions
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        asr_versions: ['0.7.2']
+        asr_versions: ['1.0rc2']
     steps:
     - uses: actions/checkout@v2
     - uses: actions/setup-python@v1
@@ -54,8 +31,6 @@ jobs:
     - name: Install packages and run tests
       run: |
         pip install pytest
-        pip install --upgrade setuptools>=41.0.0
-        pip install "numpy<1.19"
         pip install asreview[all]==${{ matrix.asr_versions }}
         pip install .
         pytest tests
diff --git a/.github/workflows/pythonpublish.yml b/.github/workflows/pythonpublish.yml
index 1017af4..f09f948 100644
--- a/.github/workflows/pythonpublish.yml
+++ b/.github/workflows/pythonpublish.yml
@@ -43,8 +43,8 @@ jobs:
         GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
       with:
         upload_url: ${{ steps.create_release.outputs.upload_url }}
-        asset_path: ./dist/asreview_visualization-${{ steps.get_version.outputs.VERSION }}-py3-none-any.whl
-        asset_name: asreview_visualization-${{ steps.get_version.outputs.VERSION }}-py3-none-any.whl
+        asset_path: ./dist/asreview_insights-${{ steps.get_version.outputs.VERSION }}-py3-none-any.whl
+        asset_name: asreview_insights-${{ steps.get_version.outputs.VERSION }}-py3-none-any.whl
         asset_content_type: application/x-wheel+zip
     - name: Upload Release Asset (Sdist)
       id: upload-release-asset-sdist
@@ -53,8 +53,8 @@ jobs:
         GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
       with:
         upload_url: ${{ steps.create_release.outputs.upload_url }}
-        asset_path: ./dist/asreview-visualization-${{ steps.get_version.outputs.VERSION }}.tar.gz
-        asset_name: asreview-visualization-${{ steps.get_version.outputs.VERSION }}.tar.gz
+        asset_path: ./dist/asreview-insights-${{ steps.get_version.outputs.VERSION }}.tar.gz
+        asset_name: asreview-insights-${{ steps.get_version.outputs.VERSION }}.tar.gz
         asset_content_type: application/zip
     - name: Publish package
       uses: pypa/gh-action-pypi-publish@master
diff --git a/README.md b/README.md
index b86b201..cc13e2e 100644
--- a/README.md
+++ b/README.md
@@ -1,154 +1,457 @@
-# ASReview-visualization
+# asreview-insights
 
-[![PyPI version](https://badge.fury.io/py/asreview-visualization.svg)](https://badge.fury.io/py/asreview-visualization) [![Downloads](https://pepy.tech/badge/asreview-visualization)](https://pepy.tech/project/asreview-visualization) ![PyPI - License](https://img.shields.io/pypi/l/asreview-visualization) ![Deploy and release](https://github.com/asreview/asreview-visualization/workflows/Deploy%20and%20release/badge.svg) ![Build status](https://github.com/asreview/asreview-visualization/workflows/test-suite/badge.svg)
+[![PyPI version](https://badge.fury.io/py/asreview-insights.svg)](https://badge.fury.io/py/asreview-insights) [![Downloads](https://pepy.tech/badge/asreview-insights)](https://pepy.tech/project/asreview-insights) ![PyPI - License](https://img.shields.io/pypi/l/asreview-insights) ![Deploy and release](https://github.com/asreview/asreview-insights/workflows/Deploy%20and%20release/badge.svg) ![Build status](https://github.com/asreview/asreview-insights/workflows/test-suite/badge.svg)
+
+❣️ ASReview-insights is the successor to [ASReview-visualization](https://pypi.org/project/asreview-visualization/). ASReview insights is available for version 1 or later. Use ASReview visualization for versions 0.x. 
+
+This official extension to [ASReview
+LAB](https://github.com/asreview/asreview) extends asreview with tools for
+plotting and extraction of metrics and statistics. The extension is especially
+useful in combination with the simulation feature of ASReview LAB.
 
-This is a plotting/visualization supplemental package for the
-[ASReview](https://github.com/asreview/asreview) software. It is a fast way to
-create a visual impression of the ASReview with different datasets, models and
-model parameters.
 
 ## Installation
 
-The easiest way to install the visualization package is to install from PyPI:
+ASReview-insights can be installed from PyPI:
 
 ``` bash
-pip install asreview-visualization
+pip install asreview-insights
 ```
 
-After installation of the visualization package, `asreview` should automatically
-detect it. Test this by:
+After installation, check if the ` asreview-insights` package is listed as an
+extension. Use the following command:
 
 ```bash
 asreview --help
 ```
 
-It should list the 'plot' modus.
+It should list the 'plot' subcommand and the 'stats' subcommand.
 
-## Basic usage
+## Active learning performance
 
-State files that were created with the same ASReview settings can be put
-together/averaged by putting them in the same directory. State files with
-different settings/datasets should be put in different directories to compare
-them.
+The `ASReview-insights` extension is useful to measure the performance of
+active learning models on collections of binary labeled text. The performance
+can be measured with widely used metrics like Recall and Work Saved over
+Sampling (WSS). The `ASReview-insights` extension can plot or compute metrics
+from ASReview project files, ideally as the result of a simulation. See
+[LINK]() for more information on simulating with ASReview LAB.
 
-As an example consider the following directory structure, where we have two
-datasets, called `ace` and `ptsd`, each of which have 8 runs:
+The following plot (with generated, fake data) explains the different metrics
+Recall, WSS, and ERF. The dataset contains 1000 records. The stepped line on
+the diagonal is the naive labeling approach (screening randomly sorted
+records).
 
-```
-├── ace
-│   ├── results_0.h5
-│   ├── results_1.h5
-│   ├── results_2.h5
-│   ├── results_3.h5
-│   ├── results_4.h5
-│   ├── results_5.h5
-│   ├── results_6.h5
-│   └── results_7.h5
-└── ptsd
-    ├── results_0.h5
-    ├── results_1.h5
-    ├── results_2.h5
-    ├── results_3.h5
-    ├── results_4.h5
-    ├── results_5.h5
-    ├── results_6.h5
-    └── results_7.h5
+![ASReview metrics explained](docs/stats_explainer.png)
+
+
+## Plots
+
+The plots in this section are derived from an ASReview (v1.0) file generated
+from
+
+```python
+asreview simulate benchmark:van_de_schoot_2017 -s sim_van_de_schoot_2017.asreview --init_seed 535
 ```
 
-Then we can plot the results by:
+### Plot types
+
+#### Recall
+
+The recall is an important metric to study the performance of active learning
+algorithms in the context of information retrieval. The ASReview Insights
+offers a straightforward command line interface to plot a "recall curve". The
+recall curve is the recall at any moment in the active learning process.
+
+To plot the recall curve, you need a ASReview file (extension `.asreview`).
+The file can be [exported from the ASReview LAB user interface](), or is the
+[result of a simulation](). To plot the recall, use this syntax (Replace
+`YOUR_ASREVIEW_FILE.asreview` by your ASReview file name.):
 
 ```bash
-asreview plot ace ptsd
+asreview plot recall YOUR_ASREVIEW_FILE.asreview
 ```
 
-By default, the values shown are expressed as percentages of the total number of papers. Use the
-`-a` or `--show-absolute-values` flags to have them expressed in absolute numbers:
+The following plot is the result of simulating the [`van_de_schoot_2017`]() in
+the benchmark platform (command `asreview simulate
+benchmark:van_de_schoot_2017 -s sim_van_de_schoot_2017.asreview`).
+
+![Recall plot of Van de Schoot 2017](figures/tests_recall_sim_van_de_schoot_2017_1.png)
+
+On the vertical axis, you find the recall after every labeling decision. The
+number of labeling actions on the horizontal axis is equal to the number of
+label actions and can't exceed the number of records in the dataset.
+
+#### WSS
+
+The Work Saved over Sampling (WSS) metric is an useful metric to study the
+performance of active learning alorithms compared with a naive (random order)
+approach. The ASReview Insights offers a straightforward command line
+interface to plot the wss at any recall.
+
+To plot the WSS curve, you need a ASReview file (extension `.asreview`).
+The file can be [exported from the ASReview LAB user interface](), or is the
+[result of a simulation](). To plot the WSS, use this syntax (Replace
+`YOUR_ASREVIEW_FILE.asreview` by your ASReview file name.):
 
 ```bash
-asreview plot ace ptsd --show-absolute-values
+asreview plot wss YOUR_ASREVIEW_FILE.asreview
 ```
 
+The following plot is the result of simulating the [`van_de_schoot_2017`]() in
+the benchmark platform (command `asreview simulate
+benchmark:van_de_schoot_2017 -s sim_van_de_schoot_2017.asreview`).
+
+![Recall plot of Van de Schoot 2017](figures/tests_wss_default_sim_van_de_schoot_2017_1.png)
 
-## Plot types
+On the vertical axis, you find the WSS after every labeling decision. The
+recall is displayed on the horizontal axis.
 
-There are currently four plot types implemented: _inclusion_, _discovery_,
-_limit_, _progression_. They can be individually selected with the `-t` or
-`--type` switch. Multiple plots can be made by using `,` as a separator:
+
+#### ERF
 
 ```bash
-asreview plot ace ptsd --type 'inclusion,discovery'
+asreview plot erf YOUR_ASREVIEW_FILE.asreview
+```
+
+The following plot is the result of simulating the [`van_de_schoot_2017`]() in
+the benchmark platform (command `asreview simulate
+benchmark:van_de_schoot_2017 -s sim_van_de_schoot_2017.asreview`).
+
+![Recall plot of Van de Schoot 2017](figures/tests_erf_default_sim_van_de_schoot_2017_1.png)
+
+On the vertical axis, you find the ERF after every labeling decision. The
+number of labeling actions on the horizontal axis is equal to the number of
+label actions and can't exceed the number of records in the dataset.
+
+### Very sparse datasets
+
+Very sparse or small datasets can provide good explanation on interesting
+details of the plotting subcommands in this extension. Important details are
+for example the handling of prior knowledge and the computation of the recall
+prediction in case of random screening.
+
+The following plot shows the result of a collection of 4 records with 3
+relevant items (inclusions). The relevant items are found in the following
+order:
+
+```
+[1, 1, 0, 1, 0]
+```
+
+![Recall of small dataset example](figures/tests_small_dataset_recall.png)
+
+The black line is an estimate of the recall after every screened record in a
+naive manner (also refered to as 'random').
+
+```
+Recall (est) when screening 1 = (3 relevant records / 4 records left)  / 3 = 0.25
+Recall (est) when screening 2 = (1/4) * (3 relevant records / 3 records left)  / 3 +
+                                (3/4) * (2 relevant records / 3 records left)  / 3 = (1/4 + 3/4*2/3) / 3 = 0.25
+
 ```
 
-### Inclusion
+The Work Saved over Sampling (WSS) is the difference between the recall of the
+simulation and the theoretical recall of random screening.
 
-This figure shows the number/percentage of included papers found as a function
-of the number/percentage of papers reviewed. Initial included/excluded papers
-are subtracted so that the line always starts at (0,0).
+![WSS for small dataset example](figures/tests_small_dataset_wss.png)
 
-The quicker the line goes to a 100%, the better the performance.
+The following graph shows the recall versus the WSS. This comparison is
+important because it is the fundamental of the `WSS@95%` metric used in the
+literature about Active Learning for systematic reviewing.
 
-Use `--show-metric-labels` to display the labels and values of WSS and RFF
-lines.
+![ERF for small dataset example](figures/tests_small_dataset_erf.png)
 
-![alt text](https://github.com/msdslab/asreview-visualization/blob/master/docs/inclusions.png?raw=true "Inclusions")
 
-### Discovery
+### Plotting API
 
-This figure shows the distribution of the number of papers that have to be
-read before discovering each inclusion. Not every paper is equally hard to
-find.
+To make use of the more advanced features, you can make use of the Python API.
+The advantage is that you can tweak every single element of the plot in the
+way you like. The following examples show how the Python API can be used. They
+make use of matplotlib extensively. See the [Introduction to
+Matplotlib](https://matplotlib.org/stable/tutorials/introductory/usage.html)
+for examples on using the API.
 
-The closer to the left, the better.
+The following example show how to plot the recall with the API and save the
+result. The plot is saved using the matplotlib API.
 
-![alt text](https://github.com/msdslab/asreview-visualization/blob/master/docs/discovery.png?raw=true "Discovery")
+```python
+import matplotlib.pyplot as plt
+
+from asreview import open_state
+from asreviewcontrib.insights.plot import plot_recall
 
+with open_state("example.asreview") as s:
 
-### Limit
+    fig, ax = plt.subplots()
 
-This figure shows how many papers need to be read with a given criterion. A
-criterion is expressed as "after reading _y_ % of the papers, at most an
-average of _z_ included papers have been not been seen by the reviewer, if he
-is using max sampling.". Here, _y_ is shown on the y-axis, while three values
-of _z_ are plotted as three different lines with the same color. The three
-values for _z_ are 0.1, 0.5 and 2.0.
+    plot_recall(ax, s)
 
-The quicker the lines touch the black (`y=x`) line, the better.
+    fig.savefig("example.png")
+```
 
-![alt text](https://github.com/msdslab/asreview-visualization/blob/master/docs/limits.png?raw=true "Limits")
+Other options are `plot_wss` and `plot_erf`.
 
-### Progression
+#### Example: Customize plot
 
-This figure shows the average inclusion rate as a function of time, number of
-papers read. The more concentrated on the left, the better. The thick line is
-the average of individual runs (thin lines). The visualization package will
-automatically detect which are directories and which are files. The curve is
-smoothed out by using a Gaussian smoothing algorithm.
+It's straightforward to customize the plots if you are familiar with
+`matplotlib`. The following example shows how to update the title of the plot.
 
-![alt text](https://github.com/msdslab/asreview-visualization/blob/master/docs/progression.png?raw=true "Progression")
+```python
+import matplotlib.pyplot as plt
 
+from asreview import open_state
+from asreviewcontrib.insights.plot import plot_wss
 
-## API
+with open_state("example.asreview") as s:
 
-To make use of the more advanced features, you can also use the visualization
-package as a library. The advantage is that you can make more reproducible
-plots where text, etc. is in the place *you* want it. Examples can be found in
-module `asreviewcontrib.visualization.quick`. Those are the scripts that are
-used for the command line interface.
+    fig, ax = plt.subplots()
+    plot_wss(ax, s)
+
+    plt.title("WSS with custom title")
+
+    fig.savefig("example_custom_title.png")
+```
+
+![WSS with custom title](docs/example_custom_title.png)
+
+#### Example: Prior knowledge
+
+It's possible to include prior knowledge to your plot. By default, prior
+knowledge is excluded from the plot.
 
 ```python
-from asreviewcontrib.visualization.plot import Plot
+import matplotlib.pyplot as plt
+
+from asreview import open_state
+from asreviewcontrib.insights.plot import plot_wss
 
-with Plot.from_paths(["PATH_1", "PATH_2"]) as plot:
-	inc_plot = plot.new("inclusion")
-	inc_plot.set_grid()
-	inc_plot.set_xlim(0, 30)
-	inc_plot.set_ylim(0, 101)
-	inc_plot.set_legend()
-	inc_plot.show()
-	inc_plot.save("SOME_FILE.png")
+with open_state("example.asreview") as s:
+
+    fig, ax = plt.subplots()
+    plot_wss(ax, s, priors=True)
+
+```
+
+#### Example: Relative versus absolute axes
+
+By default, all axes in ASReview-insights are relative. The API can be used to
+change this behavior. The arguments are identical for each plot function.
+
+```python
+import matplotlib.pyplot as plt
+
+from asreview import open_state
+from asreviewcontrib.insights.plot import plot_wss
+
+with open_state("example.asreview") as s:
+
+    fig, ax = plt.subplots()
+    plot_wss(ax, s, x_relative=False, y_relative=False)
+
+    fig.savefig("example_absolute_axis.png")
 ```
 
-Of course fill in `PATH_1` and `PATH_2` as the files you would like to plot.
+![Recall with absolute axes](docs/example_absolute_axes.png)
+
+
+#### Example: Multiple curves in one plot
+
+It is possible to have multiple curves in one plot by using the API,
+and add a legend.
+
+```python
+import matplotlib.pyplot as plt
+
+from asreview import open_state
+from asreviewcontrib.insights.plot import plot_recall
+
+
+fig, ax = plt.subplots()
+
+with open_state("tests/asreview_files/sim_van_de_schoot_2017_1.asreview") as s1:
+    plot_recall(ax, s1)
+
+with open_state("tests/asreview_files/"
+                "sim_van_de_schoot_2017_logistic.asreview") as s2:
+    plot_recall(ax, s2)
+
+ax.lines[0].set_label("Naive Bayes")
+ax.lines[2].set_label("Logistic")
+ax.legend()
+
+fig.savefig("docs/example_multiple_lines.png")
+```
+
+![Recall with multiple lines](docs/example_multiple_lines.png)
+
+## Metrics
+
+The metrics in ASReview-insights can be used to extract metrics at given
+values. The easiest way to get metrics on a ASReview project file is with the
+following command don the command line:
+
+```
+asreview stats sim_van_de_schoot_2017.asreview
+```
+
+which results in
+
+```
+{
+    "metrics": {
+        "recall": [
+            {
+                "x": 0.1,
+                "y": 1.0
+            },
+            {
+                "x": 0.25,
+                "y": 1.0
+            },
+            {
+                "x": 0.5,
+                "y": 1.0
+            },
+            {
+                "x": 0.75,
+                "y": 1.0
+            },
+            {
+                "x": 0.9,
+                "y": 1.0
+            }
+        ],
+        "wss_recall": [
+            {
+                "x": 0.95,
+                "y": 0.9107806691449815
+            }
+        ],
+        "erf": [
+            {
+                "x": 0.95,
+                "y": 0.047619047619047616
+            }
+        ]
+    }
+}
+```
+
+Each available metric has `x` and `y` values. The `x` value is the value at
+which the metric is computed. In the plots above, this is the x-axis. The `y`
+value is the output of the metric. Some metrics are computed for multiple
+values.
+
+| Metric | X description | Y description | Default |
+|---|---|---|---|
+| `recall` | Labels | Recall | 0.1, 0.25, 0.5, 0.75, 0.9 |
+| `wss_recall` | Recall | Work Saved over Sampling at recall | 0.95 |
+| `erf` | Labels | ERF | 0.95 |
+
+
+### Override default values
+
+It is possible to override the default values of `asreview stats`. See
+`asreview stats -h` for more information or see the example below.
+
+```
+asreview stats sim_van_de_schoot_2017.asreview --wss_recall 0.9 0.95
+```
+
+```
+{
+    "metrics": {
+        "recall": [
+            {
+                "x": 0.1,
+                "y": 1.0
+            },
+            {
+                "x": 0.25,
+                "y": 1.0
+            },
+            {
+                "x": 0.5,
+                "y": 1.0
+            },
+            {
+                "x": 0.75,
+                "y": 1.0
+            },
+            {
+                "x": 0.9,
+                "y": 1.0
+            }
+        ],
+        "wss_recall": [
+            {
+                "x": 0.9,
+                "y": 0.8692419589461775
+            },
+            {
+                "x": 0.95,
+                "y": 0.9107806691449815
+            }
+        ],
+        "erf": [
+            {
+                "x": 0.95,
+                "y": 0.047619047619047616
+            }
+        ]
+    }
+}
+```
+
+### Save metrics to file
+
+Metrics can be saved to a file in the JSON format. Use the flag `-o` or
+`--output`.
+
+```
+asreview stats sim_van_de_schoot_2017.asreview -o my_file.json
+```
+
+### Metrics API
+
+Metrics are easily accesible with the `ASReview-insights` API.
+
+Compute the recall after reading half of the dataset.
+
+```python
+
+from asreview import open_state
+from asreviewcontrib.insights.metrics import recall
+
+with open_state("example.asreview") as s:
+
+    print(recall(s, 0.5))
+```
+
+Other metrics are available like `wss` and `erf`.
+
+#### Example: Prior knowledge
+
+It's possible to include prior knowledge to your metric. By default, prior
+knowledge is excluded from the metric.
+
+```python
+
+from asreview import open_state
+from asreviewcontrib.insights.metrics import recall
+
+with open_state("example.asreview") as s:
+
+    print(recall(s, 0.5, priors=True))
+```
+
+## License
+
+This extension is published under the [MIT license](/LICENSE).
+
+## Contact
 
-If the customization is not sufficient, you can also directly manipulate the
-`self.ax` and `self.fig` attributes of the plotting class.
+This extension is part of the ASReview project ([asreview.ai](https://asreview.ai)). It is maintained by the
+maintainers of ASReview LAB. See [ASReview
+LAB](https://github.com/asreview/asreview) for contact information and more
+resources.
diff --git a/asreviewcontrib/insights/__init__.py b/asreviewcontrib/insights/__init__.py
new file mode 100644
index 0000000..8d601cb
--- /dev/null
+++ b/asreviewcontrib/insights/__init__.py
@@ -0,0 +1,10 @@
+from asreviewcontrib.insights.plot import plot_recall
+from asreviewcontrib.insights.plot import plot_wss
+from asreviewcontrib.insights.plot import plot_erf
+
+from ._version import get_versions
+
+__version__ = get_versions()['version']
+del get_versions
+
+__all__ = ['plot_recall', 'plot_wss', 'plot_erf']
diff --git a/asreviewcontrib/visualization/_version.py b/asreviewcontrib/insights/_version.py
similarity index 99%
rename from asreviewcontrib/visualization/_version.py
rename to asreviewcontrib/insights/_version.py
index a29da92..52444f3 100644
--- a/asreviewcontrib/visualization/_version.py
+++ b/asreviewcontrib/insights/_version.py
@@ -40,8 +40,8 @@ def get_config():
     cfg.VCS = "git"
     cfg.style = "pep440"
     cfg.tag_prefix = "v"
-    cfg.parentdir_prefix = "asreview-visualization-"
-    cfg.versionfile_source = "asreviewcontrib/visualization/_version.py"
+    cfg.parentdir_prefix = "asreview-insights-"
+    cfg.versionfile_source = "asreviewcontrib/insights/_version.py"
     cfg.verbose = False
     return cfg
 
@@ -56,6 +56,7 @@ class NotThisMethod(Exception):
 
 def register_vcs_handler(vcs, method):  # decorator
     """Decorator to mark a method as the handler for a particular VCS."""
+
     def decorate(f):
         """Store f in HANDLERS[vcs][method]."""
         if vcs not in HANDLERS:
diff --git a/asreviewcontrib/insights/algorithms.py b/asreviewcontrib/insights/algorithms.py
new file mode 100644
index 0000000..7b97b2e
--- /dev/null
+++ b/asreviewcontrib/insights/algorithms.py
@@ -0,0 +1,66 @@
+import numpy as np
+
+
+def _recall_values(labels, x_relative=True, y_relative=True):
+    n_docs = len(labels)
+    n_pos_docs = sum(labels)
+
+    x = np.arange(1, n_docs + 1)
+    recall = np.cumsum(labels)
+
+    if x_relative:
+        x = x / n_docs
+
+    if y_relative:
+        y = recall / n_pos_docs
+    else:
+        y = recall
+
+    return x, y
+
+
+def _wss_values(labels, x_relative=True, y_relative=True):
+    n_docs = len(labels)
+    n_pos_docs = sum(labels)
+
+    docs_found = np.cumsum(labels)
+    docs_found_random = np.round(np.linspace(0, n_pos_docs, n_docs))
+
+    # Get the first occurrence of 1, 2, 3, ..., n_pos_docs in both arrays.
+    when_found = np.searchsorted(docs_found, np.arange(1, n_pos_docs + 1))
+    when_found_random = np.searchsorted(docs_found_random,
+                                        np.arange(1, n_pos_docs + 1))
+    n_found_earlier = when_found_random - when_found
+
+    x = np.arange(1, n_pos_docs + 1)
+    if x_relative:
+        x = x / n_pos_docs
+
+    if y_relative:
+        y = n_found_earlier / n_docs
+    else:
+        y = n_found_earlier
+
+    return x, y
+
+
+def _erf_values(labels, x_relative=True, y_relative=True):
+
+    n_docs = len(labels)
+    n_pos_docs = sum(labels)
+
+    docs_found = np.cumsum(labels)
+    docs_found_random = np.round(np.linspace(0, n_pos_docs, n_docs))
+
+    extra_records_found = docs_found - docs_found_random
+
+    x = np.arange(1, n_docs + 1)
+    if x_relative:
+        x = x / n_docs
+
+    if y_relative:
+        y = extra_records_found / n_pos_docs
+    else:
+        y = extra_records_found
+
+    return x, y
diff --git a/asreviewcontrib/insights/entrypoint.py b/asreviewcontrib/insights/entrypoint.py
new file mode 100644
index 0000000..f8300f1
--- /dev/null
+++ b/asreviewcontrib/insights/entrypoint.py
@@ -0,0 +1,172 @@
+import argparse
+import json
+
+import matplotlib.pyplot as plt
+
+# from asreview.config import LOGGER_EXTENSIONS
+from asreview.entry_points import BaseEntryPoint
+from asreview import open_state
+
+from asreviewcontrib.insights import plot_recall
+from asreviewcontrib.insights import plot_wss
+from asreviewcontrib.insights import plot_erf
+from asreviewcontrib.insights.stats import get_stats, print_stats
+
+PLOT_TYPES = ['recall']
+TYPE_TO_FUNC = {'recall': plot_recall, 'wss': plot_wss, 'erf': plot_erf}
+
+
+class PlotEntryPoint(BaseEntryPoint):
+    description = "Plotting functionality for ASReview files."
+    extension_name = "asreview-insights"
+
+    @property
+    def version(self):
+        from asreviewcontrib.insights.__init__ import __version__
+        return __version__
+
+    def execute(self, argv):
+        parser = argparse.ArgumentParser(prog='asreview plot')
+        parser.add_argument("plot_type",
+                            metavar='type',
+                            type=str,
+                            default="recall",
+                            help="Plot type. Default 'recall'.")
+        parser.add_argument('asreview_files',
+                            metavar='asreview_files',
+                            type=str,
+                            nargs='+',
+                            help='A (list of) ASReview files.')
+        # parser.add_argument('--show_priors',
+        #                     metavar='priors',
+        #                     type=bool,
+        #                     default=False,
+        #                     help='Show records used as prior knowledge '
+        #                     'in the plot.')
+        # parser.add_argument('--x_relative',
+        #                     type=bool,
+        #                     default=True,
+        #                     help='Make use of relative coordinates on'
+        #                     ' the x-axis. Default: True.')
+        # parser.add_argument('--y_relative',
+        #                     type=bool,
+        #                     default=True,
+        #                     help='Make use of relative coordinates on'
+        #                     ' the y-axis. Default: True.')
+        parser.add_argument(
+            "-V",
+            "--version",
+            action="version",
+            version=f"asreview-insights: {self.version}",
+        )
+        parser.add_argument(
+            "-o",
+            "--output",
+            default=None,
+            help='Save the plot to a file. File formats are detected '
+            ' by the matplotlib library, check there to see available '
+            'formats.')
+        args = parser.parse_args(argv)
+
+        if len(args.asreview_files) > 1:
+            raise ValueError("Plotting multiple project files"
+                             " via the CLI is not supported yet.")
+
+        with open_state(args.asreview_files[0]) as s:
+
+            fig, ax = plt.subplots()
+            plot_func = TYPE_TO_FUNC[args.plot_type]
+            plot_func(ax,
+                      s
+                      # priors=args.priors,
+                      # x_relative=args.x_relative,
+                      # y_relative=args.y_relative
+                      )
+
+            if args.output:
+                fig.savefig(args.output)
+            else:
+                plt.show()
+
+
+class StatsEntryPoint(BaseEntryPoint):
+    description = "Statistics and metrics entry point."
+    extension_name = "asreview-insights"
+
+    @property
+    def version(self):
+        from asreviewcontrib.insights.__init__ import __version__
+        return __version__
+
+    def execute(self, argv):
+        parser = argparse.ArgumentParser(prog='asreview stats')
+        parser.add_argument('asreview_files',
+                            metavar='asreview_files',
+                            type=str,
+                            nargs='+',
+                            help='A combination of data directories or files.')
+        parser.add_argument(
+            "-V",
+            "--version",
+            action="version",
+            version=f"asreview-insights: {self.version}",
+        )
+        parser.add_argument(
+            '--recall',
+            metavar='recall',
+            type=float,
+            default=[0.1, 0.25, 0.5, 0.75, 0.9],
+            nargs='+',
+            help='A (list of) values to compute the recall at.')
+        parser.add_argument('--wss_recall',
+                            metavar='wss',
+                            type=float,
+                            nargs='+',
+                            default=[0.95],
+                            help='A (list of) values to compute the wss at.')
+        parser.add_argument('--erf',
+                            metavar='erf',
+                            type=float,
+                            nargs='+',
+                            default=[0.95],
+                            help='A (list of) values to compute the erf at.')
+        parser.add_argument('--show_priors',
+                            metavar='priors',
+                            type=bool,
+                            default=False,
+                            help='Show records used as prior knowledge '
+                            'in the plot.')
+        parser.add_argument('--x_relative',
+                            type=bool,
+                            default=True,
+                            help='Make use of relative coordinates on'
+                            ' the x-axis. Default: True.')
+        parser.add_argument('--y_relative',
+                            type=bool,
+                            default=True,
+                            help='Make use of relative coordinates on'
+                            ' the y-axis. Default: True.')
+        parser.add_argument(
+            "-o",
+            "--output",
+            default=None,
+            help='Save the statistics and metrics to a JSON file.')
+        args = parser.parse_args(argv)
+
+        if len(args.asreview_files) > 1:
+            raise ValueError("Computing metrics for multiple project files"
+                             " via the CLI is not supported yet.")
+
+        with open_state(args.asreview_files[0]) as s:
+            stats = get_stats(s,
+                              recall=args.recall,
+                              wss=args.wss,
+                              erf=args.erf,
+                              priors=args.priors,
+                              x_relative=args.x_relative,
+                              y_relative=args.y_relative)
+            print_stats(stats)
+
+        if args.output:
+            with open(args.output, "w") as f:
+                json.dump(stats, f, indent=4)
diff --git a/asreviewcontrib/insights/metrics.py b/asreviewcontrib/insights/metrics.py
new file mode 100644
index 0000000..0e4ec88
--- /dev/null
+++ b/asreviewcontrib/insights/metrics.py
@@ -0,0 +1,90 @@
+import numpy as np
+
+from asreviewcontrib.insights.algorithms import _recall_values
+from asreviewcontrib.insights.algorithms import _wss_values
+from asreviewcontrib.insights.algorithms import _erf_values
+from asreviewcontrib.insights.utils import get_labels
+
+
+def _slice_metric(x, y, intercept):
+    """Find the first value after the intercept.
+
+    intercept[i-1] <= v < intercept[i]
+
+    Arguments
+    ---------
+    x: numpy.array or list
+        The values of the x-axis.
+    y: numpy.array or list
+        The values of the y-axis.
+    intercept: float
+        The value of the x-axis to map to the y-axis. If value
+        is not present, the first value greater than the intercept
+        is used.
+
+    Returns
+    -------
+    float
+    """
+
+    i = np.searchsorted(x, intercept, side='right')
+    return y[i - 1]
+
+
+def recall(state_obj,
+           intercept,
+           priors=False,
+           x_relative=True,
+           y_relative=True):
+
+    labels = get_labels(state_obj)
+
+    return _recall(labels,
+                   intercept,
+                   x_relative=x_relative,
+                   y_relative=y_relative)
+
+
+def _recall(labels, intercept, x_relative=True, y_relative=True):
+
+    x, y = _recall_values(labels, x_relative=x_relative, y_relative=y_relative)
+    print(x, y)
+
+    if intercept < x[0]:
+        return 0
+
+    return _slice_metric(x, y, intercept)
+
+
+def wss(state_obj, intercept, priors=False, x_relative=True, y_relative=True):
+
+    labels = get_labels(state_obj)
+
+    return _wss(labels,
+                intercept,
+                x_relative=x_relative,
+                y_relative=y_relative)
+
+
+def _wss(labels, intercept, x_relative=True, y_relative=True):
+
+    x, y = _wss_values(labels, x_relative=x_relative, y_relative=y_relative)
+
+    return _slice_metric(x, y, intercept)
+
+
+def erf(state_obj, intercept, priors=False, x_relative=True, y_relative=True):
+
+    labels = get_labels(state_obj)
+
+    return _erf(labels,
+                intercept,
+                x_relative=x_relative,
+                y_relative=y_relative)
+
+
+def _erf(labels, intercept, x_relative=True, y_relative=True):
+
+    x, y = _erf_values(labels, x_relative=x_relative, y_relative=y_relative)
+
+    return _slice_metric(x, y, intercept)
diff --git a/asreviewcontrib/insights/plot.py b/asreviewcontrib/insights/plot.py
new file mode 100644
index 0000000..b851380
--- /dev/null
+++ b/asreviewcontrib/insights/plot.py
@@ -0,0 +1,280 @@
+import numpy as np
+
+from asreviewcontrib.insights.algorithms import _recall_values
+from asreviewcontrib.insights.algorithms import _wss_values
+from asreviewcontrib.insights.algorithms import _erf_values
+from asreviewcontrib.insights.utils import get_labels
+
+
+def _fix_start_tick(ax):
+
+    # correct x axis if tick is at position 0
+    locs = ax.get_xticks()
+    if locs[1] == 0:
+        locs[1] = 1
+        ax.set_xticks(locs[1:-1])
+
+    return ax
+
+
+def plot_recall(ax, state_obj, priors=False, x_relative=True, y_relative=True):
+    """Plot the recall@T for all thresholds T.
+
+    Arguments
+    ---------
+    state_obj: asreview.state.SQLiteState
+        State object from which to get the labels for the plot.
+    priors: bool
+        Include the prior in plot or not.
+    x_relative: bool
+        If False, the number of records is on the x-axis.
+        If True, the fraction of the whole dataset is on the x-axis.
+    y_relative: bool
+        If False, the number of included records found is on the y-axis.
+        If True, the fraction of all included records found is on the y-axis.
+
+    Returns
+    -------
+    matplotlib.axes.Axes
+
+    Notes
+    -----
+    The recall at T statistic is defined as the number of relevant records
+    found after reviewing T records.
+    """
+
+    labels = get_labels(state_obj)
+
+    return _plot_recall(ax,
+                        labels,
+                        x_relative=x_relative,
+                        y_relative=y_relative)
+
+
+def _plot_recall(ax, labels, x_relative=True, y_relative=True):
+    """Plot the recall of state object(s).
+
+    labels:
+        An ASReview state object.
+    """
+
+    x, y = _recall_values(labels, x_relative=x_relative, y_relative=y_relative)
+
+    if y_relative:
+        y_lim = [-0.05, 1.05]
+        yticks = [0, 0.2, 0.4, 0.6, 0.8, 1.0]
+    else:
+        n_pos_docs = sum(labels)
+        y_lim = [-n_pos_docs * 0.05, n_pos_docs * 1.05]
+        yticks = [int(n_pos_docs * r) for r in [0, 0.2, 0.4, 0.6, 0.8, 1.0]]
+
+    ax.step(x, y, where='post')
+    ax.set_title("Recall")
+    ax.set(xlabel='#', ylabel='Recall')
+    ax.set_ylim(y_lim)
+    ax.set_yticks(yticks)
+
+    if not x_relative:
+        ax.xaxis.get_major_locator().set_params(integer=True)
+
+    # add random line if required
+    _plot_random_recall(ax,
+                        labels,
+                        x_relative=x_relative,
+                        y_relative=y_relative)
+
+    # correct x axis if tick is at position 0
+    _fix_start_tick(ax)
+
+    return ax
+
+
+def _plot_random_recall(ax, labels, x_relative, y_relative):
+    """Plot the recall of state object(s).
+
+    labels:
+        An ASReview state object.
+    """
+    n_docs = len(labels)
+    n_pos_docs = sum(labels)
+
+    # add random line if required
+    x = np.arange(1, n_docs + 1)
+    recall_random = np.round(np.linspace(0, n_pos_docs, n_docs))
+
+    if x_relative:
+        x = x / n_docs
+
+    if y_relative:
+        y = recall_random / n_pos_docs
+    else:
+        y = recall_random
+
+    ax.step(x, y, color="black", where='post')
+
+    return ax
+
+
+def plot_wss(ax, state_obj, priors=False, x_relative=True, y_relative=True):
+    """Plot the WSS@T for all thresholds T.
+
+    Arguments
+    ---------
+    state_obj: asreview.state.SQLiteState
+        State object from which to get the labels for the plot.
+    priors: bool
+        Include the prior in plot or not.
+    x_relative: bool
+        If False, the number of included records is on the x-axis.
+        If True, the fraction of the all included records is on the x-axis.
+    y_relative: bool
+        If False, the number of records reviewed less is on the y-axis.
+        If True, the fraction of all records reviewed less is on the y-axis.
+
+    Returns
+    -------
+    matplotlib.axes.Axes
+
+    Notes
+    -----
+    The Work Saved over Sampling at T (WSS@T) statistic is defined as the number
+    of records that were reviewed less to find T included records, compared to
+    reviewing in random order.
+
+    The number of included records found after reading k papers when reviewing
+    in random order is defined as
+
+        round(n_pos * k / n),
+
+    where n_pos is the number of included records in the dataset and n is the
+    total number of records in the dataset. Hence the number or records that
+    need to be reviewed to find T included papers, when reviewing in random
+    order, is defined as the smallest integer k_T, such that
+
+        round(n_pos * k_T / n) = T
+
+    i.e. it is the smallest integer such that
+
+        n_pos * k_T / n > (T - 0.5).
+
+    If the T'th included record in the dataset is found after reviewing r_T
+    records, then we have
+
+        WSS@T = r_T - k_T.
+
+    Example
+    -------
+    [Can we include the stats_explainer picture in the docs?]
+    """
+
+    labels = get_labels(state_obj)
+
+    return _plot_wss(ax, labels, x_relative=x_relative, y_relative=y_relative)
+
+
+def _plot_wss(ax, labels, x_relative=True, y_relative=True):
+    """Plot for each threshold T in [0,1] the WSS@T."""
+    n_docs = len(labels)
+
+    x, y = _wss_values(labels, x_relative=x_relative, y_relative=y_relative)
+
+    if y_relative:
+        y_lim = [-0.05, 1.05]
+        yticks = [0, 0.2, 0.4, 0.6, 0.8, 1.0]
+    else:
+        y_lim = [-n_docs * 0.05, n_docs * 1.05]
+        yticks = [int(n_docs * r) for r in [0, 0.2, 0.4, 0.6, 0.8, 1.0]]
+
+    ax.step(x, y, where='post')
+    ax.set_title("WSS")
+    ax.set(xlabel='Recall', ylabel='WSS')
+    ax.set_ylim(y_lim)
+    ax.set_yticks(yticks)
+
+    if not x_relative:
+        ax.xaxis.get_major_locator().set_params(integer=True)
+
+        # correct x axis if tick is at position 0
+        _fix_start_tick(ax)
+
+    return ax
+
+
+def plot_erf(ax, state_obj, priors=False, x_relative=True, y_relative=True):
+    """Plot the ERF@T for all thresholds T.
+
+    Arguments
+    ---------
+    state_obj: asreview.state.SQLiteState
+        State object from which to get the labels for the plot.
+    priors: bool
+        Include the prior in plot or not.
+    x_relative: bool
+        If False, the number of records is on the x-axis.
+        If True, the fraction of the whole dataset is on the x-axis.
+    y_relative: bool
+        If False, the number of included records found is on the y-axis.
+        If True, the fraction of all included records found is on the y-axis.
+
+    Returns
+    -------
+    matplotlib.axes.Axes
+
+    Notes
+    -----
+    The Extra Records Found at T (ERF@T) statistic is defined as the number of
+    extra included papers that were found more after reviewing T records,
+    compared to reviewing in random order.
+
+    The number of included records found after reading T papers when reviewing
+    in random order is defined as
+
+        round(n_pos * T / n),
+
+    where n_pos is the number of included records in the dataset and n is the
+    total number of records in the dataset. Hence if n_pos_T is the number of
+    positive records found after reviewing T records, then
+
+        ERF@T = n_pos_T - round(n_pos * T / n).
+
+    If y_relative=True, the ERF will be returned as a fraction of the total
+    number of included records, in which case we have
+
+        ERF@T = (n_pos_T - round(n_pos * T / n)) / n_pos.
+
+    Example
+    -------
+    [Can we include the stats_explainer picture in the docs?]
+    """
+
+    labels = get_labels(state_obj)
+
+    return _plot_erf(ax, labels, x_relative=x_relative, y_relative=y_relative)
+
+
+def _plot_erf(ax, labels, x_relative=True, y_relative=True):
+    """Plot for each threshold T the ERF@T."""
+    n_pos_docs = sum(labels)
+
+    x, y = _erf_values(labels, x_relative=x_relative, y_relative=y_relative)
+
+    if y_relative:
+        y_lim = [-0.05, 1.05]
+        yticks = [0, 0.2, 0.4, 0.6, 0.8, 1.0]
+    else:
+        y_lim = [-n_pos_docs * 0.05, n_pos_docs * 1.05]
+        yticks = [int(n_pos_docs * r) for r in [0, 0.2, 0.4, 0.6, 0.8, 1.0]]
+
+    ax.step(x, y, where='post')
+    ax.set_title("ERF")
+    ax.set(xlabel='#', ylabel='ERF')
+    ax.set_ylim(y_lim)
+    ax.set_yticks(yticks)
+
+    if not x_relative:
+        ax.xaxis.get_major_locator().set_params(integer=True)
+
+        # correct x axis if tick is at position 0
+        _fix_start_tick(ax)
+
+    return ax
diff --git a/asreviewcontrib/insights/stats.py b/asreviewcontrib/insights/stats.py
new file mode 100644
index 0000000..0396400
--- /dev/null
+++ b/asreviewcontrib/insights/stats.py
@@ -0,0 +1,57 @@
+import json
+
+from asreviewcontrib.insights.metrics import _recall
+from asreviewcontrib.insights.metrics import _wss
+from asreviewcontrib.insights.metrics import _erf
+
+
+def get_stats(state_obj,
+              recall=[0.1, 0.25, 0.5, 0.75, 0.9],
+              wss=[0.95],
+              erf=[0.95],
+              priors=False,
+              x_relative=True,
+              y_relative=True):
+
+    recall = [recall] if not isinstance(recall, list) else recall
+    wss = [wss] if not isinstance(wss, list) else wss
+    erf = [erf] if not isinstance(erf, list) else erf
+
+    labels = state_obj.get_labels(priors=priors).to_list()
+
+    recall_values = [
+        _recall(labels, v, x_relative=x_relative, y_relative=y_relative)
+        for v in recall
+    ]
+    wss_values = [
+        _wss(labels, v, x_relative=x_relative, y_relative=y_relative)
+        for v in wss
+    ]
+    erf_values = [
+        _erf(labels, v, x_relative=x_relative, y_relative=y_relative)
+        for v in erf
+    ]
+
+    result = {
+        "metrics": {
+            "recall": [{
+                "x": i,
+                "y": v
+            } for i, v in zip(recall, recall_values)],
+            "wss": [{
+                "x": i,
+                "y": v
+            } for i, v in zip(wss, wss_values)],
+            "erf": [{
+                "x": i,
+                "y": v
+            } for i, v in zip(erf, erf_values)]
+        }
+    }
+
+    return result
+
+
+def print_stats(stats):
+
+    print(json.dumps(stats, indent=4))
diff --git a/asreviewcontrib/insights/utils.py b/asreviewcontrib/insights/utils.py
new file mode 100644
index 0000000..7cabd10
--- /dev/null
+++ b/asreviewcontrib/insights/utils.py
@@ -0,0 +1,17 @@
+import numpy as np
+
+
+def get_labels(state_obj, priors=False):
+
+    # get the number of records
+    n_records = state_obj.n_records
+
+    # get the labels
+    labels = state_obj.get_labels(priors=priors).to_list()
+
+    # if less labels than records, check if all labels available
+    if len(labels) < n_records:
+
+        labels = labels + np.zeros(n_records - len(labels)).tolist()
+
+    return labels
diff --git a/asreviewcontrib/visualization/__init__.py b/asreviewcontrib/visualization/__init__.py
deleted file mode 100644
index 203adf2..0000000
--- a/asreviewcontrib/visualization/__init__.py
+++ /dev/null
@@ -1,20 +0,0 @@
-# Copyright 2020 The ASReview Authors. All Rights Reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-from asreviewcontrib.visualization.plot import Plot  # noqa
-from asreviewcontrib.visualization.entrypoint import PlotEntryPoint  # noqa
-
-from ._version import get_versions
-__version__ = get_versions()['version']
-del get_versions
diff --git a/asreviewcontrib/visualization/entrypoint.py b/asreviewcontrib/visualization/entrypoint.py
deleted file mode 100644
index 2f3b5e3..0000000
--- a/asreviewcontrib/visualization/entrypoint.py
+++ /dev/null
@@ -1,156 +0,0 @@
-# Copyright 2020 The ASReview Authors. All Rights Reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-import argparse
-import logging
-
-from asreview.config import LOGGER_EXTENSIONS
-from asreview.entry_points import BaseEntryPoint
-
-from asreviewcontrib.visualization import Plot
-from asreviewcontrib.visualization.quick import progression_plot
-from asreviewcontrib.visualization.quick import limit_plot
-from asreviewcontrib.visualization.quick import discovery_plot
-from asreviewcontrib.visualization.quick import inclusion_plot
-
-PLOT_TYPES = ['inclusion', 'discovery', 'limit', 'progression']
-
-
-class PlotEntryPoint(BaseEntryPoint):
-    description = "Plotting functionality for logging files produced by "\
-        "ASReview."
-    extension_name = "asreview-visualization"
-
-    def __init__(self):
-        from asreviewcontrib.visualization.__init__ import __version__
-        super(PlotEntryPoint, self).__init__()
-
-        self.version = __version__
-
-    def execute(self, argv):
-        parser = _parse_arguments(
-            version=f"{self.extension_name}: {self.version}")
-        args_dict = vars(parser.parse_args(argv))
-
-        if args_dict['type'] == 'all':
-            types = PLOT_TYPES
-        else:
-            arg_types = args_dict['type'].split(',')
-            types = []
-            for plot_type in arg_types:
-                if plot_type not in PLOT_TYPES:
-                    logging.warning(f'Plotting type "{plot_type} unknown."')
-                else:
-                    types.append(plot_type)
-
-        if args_dict["absolute_format"]:
-            result_format = "number"
-        else:
-            result_format = "percentage"
-
-        output = args_dict["output"]
-
-        prefix = args_dict["prefix"]
-        data_paths = args_dict["data_paths"]
-        keys = args_dict["keys"] + \
-            (len(data_paths) - len(args_dict["keys"])) * [None]
-        paths = dict(zip(data_paths, keys))
-
-        with Plot.from_paths(paths, prefix=prefix) as plot:
-            if len(plot.analyses) == 0:
-                print(f"No log files found in {args_dict['data_paths']}.\n"
-                      f"To be detected log files have to start with '{prefix}'"
-                      f" and end with one of the following: \n"
-                      f"{', '.join(LOGGER_EXTENSIONS)}.")
-                return
-
-            if "inclusion" in types:
-                inclusion_plot(plot,
-                               output=output,
-                               result_format=result_format,
-                               show_metric_labels=args_dict["show_metric_labels"])  # noqa
-            if "discovery" in types:
-                discovery_plot(plot,
-                               output=output,
-                               result_format=result_format)  # noqa
-            if "limit" in types:
-                limit_plot(plot, output=output,
-                           result_format=result_format)  # noqa
-            if "progression" in types:
-                progression_plot(plot,
-                                 output=output,
-                                 result_format=result_format,
-                                 sigma=args_dict["sigma"])
-
-
-def _parse_arguments(version="Unknown"):
-    parser = argparse.ArgumentParser(prog='asreview plot')
-    parser.add_argument('data_paths',
-                        metavar='DATA_PATHS',
-                        type=str,
-                        nargs='+',
-                        help='A combination of data directories or files.')
-    parser.add_argument(
-        "-V",
-        "--version",
-        action="version",
-        version=version,
-    )
-    parser.add_argument(
-        "-t",
-        "--type",
-        type=str,
-        default="all",
-        help="Type of plot to make. Available plot types: inclusion, "
-        "discovery, limit and progression. "
-        "Separate by commas (no spaces) for"
-        " multiple plots. By default plots all types in sequence.")
-    parser.add_argument(
-        "-a",
-        "--show-absolute-values",
-        dest="absolute_format",
-        action='store_true',
-        help='Use absolute values on the axis instead of percentages.')
-    parser.add_argument(
-        "--show-metric-labels",
-        dest="show_metric_labels",
-        action='store_true',
-        help='Show the labels and values for metrics.')
-    parser.add_argument(
-        "--prefix",
-        default="",
-        help='Filter files in the data directory to only contain files'
-        'starting with a prefix.')
-    parser.add_argument(
-        "-o",
-        "--output",
-        default=None,
-        help='Save the plot to a file. If multiple plots are made, only one'
-        ' is saved (non-deterministically). File formats are detected '
-        ' by the matplotlib library, check there to see available '
-        'formats.')
-    parser.add_argument("-s",
-                        "--sigma",
-                        default=25,
-                        type=int,
-                        help="Smoothing width for the progression plot.")
-    parser.add_argument(
-        "-k",
-        "--keys",
-        default=[],
-        nargs="*",
-        type=str,
-        help="Set the key of each curve. The order should be the same as those"
-        " of the data paths.")
-    return parser
diff --git a/asreviewcontrib/visualization/plot.py b/asreviewcontrib/visualization/plot.py
deleted file mode 100644
index 5cfeb7c..0000000
--- a/asreviewcontrib/visualization/plot.py
+++ /dev/null
@@ -1,86 +0,0 @@
-# Copyright 2020 The ASReview Authors. All Rights Reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-import os
-from collections import OrderedDict
-
-from asreview.analysis import Analysis
-
-from asreviewcontrib.visualization.plot_discovery import PlotDiscovery
-from asreviewcontrib.visualization.plot_inclusions import PlotInclusions
-from asreviewcontrib.visualization.plot_limit import PlotLimit
-from asreviewcontrib.visualization.plot_progression import PlotProgression
-
-
-class Plot():
-    def __init__(self, paths, prefix="result"):
-        self.analyses = OrderedDict()
-        self.is_file = OrderedDict()
-
-        if isinstance(paths, dict):
-            for path, key in paths.items():
-                new_analyis = Analysis.from_path(path, prefix=prefix)
-                if new_analyis is None:
-                    continue
-                if key is None:
-                    key = new_analyis.key
-                else:
-                    new_analyis.key = key
-                self.analyses[key] = new_analyis
-                if os.path.isfile(path):
-                    self.is_file[key] = True
-                else:
-                    self.is_file[key] = False
-        else:
-            for path in paths:
-                new_analysis = Analysis.from_path(path, prefix=prefix)
-                if new_analysis is None:
-                    continue
-                data_key = new_analysis.key
-                self.analyses[data_key] = new_analysis
-                if os.path.isfile(path):
-                    self.is_file[data_key] = True
-                else:
-                    self.is_file[data_key] = False
-        all_files = all(self.is_file.values())
-        if all_files:
-            self.thick = {key: True for key in list(self.analyses)}
-        else:
-            self.thick = {key: not f for key, f in self.is_file.items()}
-
-    def __enter__(self):
-        return self
-
-    def __exit__(self, *_, **__):
-        for analysis in self.analyses.values():
-            analysis.close()
-
-    @classmethod
-    def from_paths(cls, paths, prefix="result"):
-        plot_inst = Plot(paths, prefix=prefix)
-        return plot_inst
-
-    def new(self, plot_type="inclusion", **kwargs):
-        thick = kwargs.pop("thick", None)
-        if thick is None:
-            thick = self.thick
-        if plot_type == "inclusion":
-            return PlotInclusions(self.analyses, thick=thick, **kwargs)
-        elif plot_type == "progression":
-            return PlotProgression(self.analyses, thick=thick, **kwargs)
-        elif plot_type == "discovery":
-            return PlotDiscovery(self.analyses, **kwargs)
-        elif plot_type == "limit":
-            return PlotLimit(self.analyses, **kwargs)
-        raise ValueError(f"Error: plot type '{plot_type}' not found.")
diff --git a/asreviewcontrib/visualization/plot_base.py b/asreviewcontrib/visualization/plot_base.py
deleted file mode 100644
index c65d3ca..0000000
--- a/asreviewcontrib/visualization/plot_base.py
+++ /dev/null
@@ -1,50 +0,0 @@
-# Copyright 2020 The ASReview Authors. All Rights Reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-import matplotlib.pyplot as plt
-
-
-class PlotBase():
-    def __init__(self, analyses):
-        """Base class for plots.
-
-        Plot the number of queries that turned out to be included
-        in the final review.
-        """
-        super(PlotBase, self).__init__()
-        self.legend_name = []
-        self.legend_plt = []
-        self.fig, self.ax = plt.subplots()
-        self.analyses = analyses
-
-    def set_legend(self, loc="lower right"):
-        self.ax.legend(self.legend_plt, self.legend_name, loc=loc)
-
-    def set_grid(self):
-        self.ax.grid()
-
-    def set_xlim(self, x_start, x_end):
-        self.ax.set_xlim(x_start, x_end)
-
-    def set_ylim(self, y_start, y_end):
-        self.ax.set_ylim(y_start, y_end)
-
-    def show(self):
-        plt.show()
-
-    def save(self, fp, *args, **kwargs):
-        self.fig.savefig(fp, *args, **kwargs)
-
-    def close(self):
-        plt.close(self.fig)
diff --git a/asreviewcontrib/visualization/plot_discovery.py b/asreviewcontrib/visualization/plot_discovery.py
deleted file mode 100644
index 241944c..0000000
--- a/asreviewcontrib/visualization/plot_discovery.py
+++ /dev/null
@@ -1,34 +0,0 @@
-from asreviewcontrib.visualization.plot_base import PlotBase
-
-
-class PlotDiscovery(PlotBase):
-    def __init__(self, analyses, result_format="percentage"):
-        """Class for the Discovery plot."""
-        super(PlotDiscovery, self).__init__(analyses)
-        self.result_format = result_format
-
-        avg_times = []
-        for analysis in self.analyses.values():
-            results = analysis.avg_time_to_discovery(
-                result_format=result_format)
-            avg_times.append(list(results.values()))
-
-        if result_format == "number":
-            self.ax.hist(avg_times,
-                         30,
-                         histtype='bar',
-                         density=False,
-                         label=self.analyses.keys())
-            self.ax.set_xlabel("# Reviewed")
-            self.ax.set_ylabel("# of papers included")
-        else:
-            self.ax.hist(avg_times,
-                         30,
-                         histtype='bar',
-                         density=True,
-                         label=self.analyses.keys())
-            self.ax.set_xlabel("% Reviewed")
-            self.ax.set_ylabel("Fraction of papers included")
-
-    def set_legend(self, loc="upper right"):
-        self.ax.legend(loc=loc)
diff --git a/asreviewcontrib/visualization/plot_inclusions.py b/asreviewcontrib/visualization/plot_inclusions.py
deleted file mode 100644
index 264dfb5..0000000
--- a/asreviewcontrib/visualization/plot_inclusions.py
+++ /dev/null
@@ -1,200 +0,0 @@
-import warnings
-
-import numpy as np
-from matplotlib.ticker import MaxNLocator
-
-from asreviewcontrib.visualization.plot_base import PlotBase
-
-
-class PlotInclusions(PlotBase):
-    def __init__(self, analyses, result_format="percentage", thick=None):
-        """Class for the Inclusions plot.
-
-        Plot the number of queries that turned out to be included
-        in the final review.
-        """
-        super(PlotInclusions, self).__init__(analyses)
-        self.result_format = result_format
-        self.col = {}
-
-        if thick is None:
-            thick = {key: True for key in list(self.analyses)}
-        self.thick = thick
-
-        if result_format == "percentage":
-            self.box_dist = 0.5
-        else:
-            self.box_dist = 100
-
-        max_len = 0
-        for i, data_key in enumerate(reversed(self.analyses)):
-            analysis = self.analyses[data_key]
-
-            inc_found = analysis.inclusions_found(result_format=result_format)
-            n_initial = analysis.inc_found[False]["n_initial"]
-            n_after_init = len(analysis.labels) - n_initial
-            max_len = max(max_len, n_after_init)
-
-            max_y = analysis.inc_found[False]["inc_after_init"]
-
-            self.col[data_key] = "C" + str((len(self.analyses) - 1 - i) % 10)
-            col = self.col[data_key]
-
-            if self.thick[data_key]:
-                lw = 2
-            else:
-                lw = 0.7
-
-            recall_plot = self.ax.step(
-                inc_found[0],
-                inc_found[1],
-                color=col,
-                lw=lw
-            )
-
-            if self.thick[data_key]:
-                self.legend_name.append(f"{data_key}")
-                self.legend_plt.append(recall_plot)
-
-        # show absolute number next to percentages
-        if result_format == "number":
-
-            # duplicate x axis
-            self.ax2 = self.ax.twiny()
-
-            # top axis
-            self.ax.set_xlim(0, max_len)
-            self.ax.set_xlabel("# Reviewed")
-
-            # bottom axis
-            self.ax2.set_xlim(0, 100)
-            self.ax2.set_xlabel("% Reviewed")
-
-            self.ax3 = self.ax.twinx()
-
-            # left axis
-            spacing_top = 1.05
-            self.ax.set_ylim(0, spacing_top * max_y)
-            self.ax.set_ylabel("# Relevant records found")
-
-            # right axis
-            self.ax3.set_ylim(0, spacing_top * 100)
-            self.ax3.set_ylabel("% Relevant records found")
-
-        # only display percentage
-        elif result_format == "percentage":
-            self.ax.set_xlabel("% Records reviewed")
-            self.ax.set_ylabel("% Relevant records found")
-
-        # no decimals on y-axis
-        self.ax.yaxis.set_major_locator(MaxNLocator(integer=True))
-
-        self.fig.tight_layout()
-
-    def add_WSS(self, *args, **kwargs):  # noqa
-        warnings.warn("add_WSS is deprecated, use add_wss instead",
-                      DeprecationWarning)
-        self.add_wss(*args, **kwargs)
-
-    def add_wss(self,
-                data_key,
-                value=95,
-                text_at=None,
-                add_value=False,
-                alpha=0.8,
-                text_col="white",
-                add_text=True,
-                **kwargs):
-        analysis = self.analyses[data_key]
-        col = self.col[data_key]
-
-        if value is None:
-            return
-
-        text = f"WSS@{value}%"
-        wss_val, wss_x, wss_y = analysis.wss(value,
-                                             x_format=self.result_format,
-                                             **kwargs)
-        if wss_x is None or wss_y is None:
-            return
-
-        if add_value:
-            text += r"$\approx" + f" {round(wss_val, 2)}" + r"\%$"
-
-        if text_at is None:
-            text_at = (wss_x[0] + self.box_dist, (wss_y[0] + wss_y[1]) / 2)
-
-        self.ax.plot(wss_x, wss_y, color=col, ls=(0, (5, 1)))
-        self.ax.plot(wss_x, (0, wss_y[0]), color=col, ls=(0, (1, 5)))
-        bbox = dict(boxstyle='round', facecolor=col, alpha=alpha)
-        if add_text:
-            self.ax.text(*text_at, text, color=text_col, bbox=bbox)
-
-    def add_RRF(self, *args, **kwargs):  # noqa
-        warnings.warn("add_RRF is deprecated, use add_rrf instead",
-                      DeprecationWarning)
-        self.add_rrf(*args, **kwargs)
-
-    def add_rrf(self,
-                data_key,
-                value=10,
-                text_at=None,
-                add_value=False,
-                alpha=0.8,
-                text_col="white",
-                add_text=True,
-                **kwargs):
-        analysis = self.analyses[data_key]
-        col = self.col[data_key]
-        if value is None:
-            return
-
-        rrf_val, rrf_x, rrf_y = analysis.rrf(value,
-                                             x_format=self.result_format,
-                                             **kwargs)
-        if rrf_x is None or rrf_y is None:
-            return
-
-        text = f"RRF@{value}%"
-        if add_value:
-            text += r"$\approx" + f" {round(rrf_val, 2)}" + r"\%$"
-
-        rrf_x = 0, rrf_x[0]
-        rrf_y = rrf_y[1], rrf_y[1]
-        if text_at is None:
-            text_at = (rrf_x[0] + self.box_dist, rrf_y[0] * 0.9)
-
-        self.ax.plot(rrf_x, rrf_y, color=col, ls="--")
-        bbox = dict(boxstyle='round', facecolor=col, alpha=alpha)
-        if add_text:
-            self.ax.text(*text_at, text, color=text_col, bbox=bbox)
-
-    def add_random(self, line_col='grey', add_text=True, text_at=None):
-        xlim = self.ax.get_xlim()
-        ylim = self.ax.get_ylim()
-        if text_at is None:
-            xlim = self.ax.get_xlim()
-            ylim = self.ax.get_ylim()
-            text_at = (
-                np.average(xlim) - 0.07 * (xlim[1] - xlim[0]),
-                np.average(ylim) + 0.07 * (ylim[1] - ylim[0]),
-            )
-
-        bbox = dict(boxstyle='round', facecolor='0.65')
-
-        # add label for random line
-        if add_text:
-            self.ax.text(*text_at, "random", color="black", bbox=bbox)
-
-        xlim = [max(x, 0) for x in xlim]
-        if self.result_format == "percentage":
-            xlim = [min(x, 100) for x in xlim]
-            y_vals = xlim
-        else:
-            analysis = self.analyses[list(self.analyses)[0]]
-            n_labels = len(analysis.labels)
-            n_initial = analysis.inc_found[False]["n_initial"]
-            max_y = analysis.inc_found[False]["inc_after_init"]
-            label_after_init = n_labels - n_initial
-            y_vals = max_y * np.array(xlim) / label_after_init
-        self.ax.plot(xlim, y_vals, color=line_col, ls="--")
diff --git a/asreviewcontrib/visualization/plot_limit.py b/asreviewcontrib/visualization/plot_limit.py
deleted file mode 100644
index 3464cdd..0000000
--- a/asreviewcontrib/visualization/plot_limit.py
+++ /dev/null
@@ -1,42 +0,0 @@
-import numpy as np
-
-from asreviewcontrib.visualization.plot_base import PlotBase
-
-
-class PlotLimit(PlotBase):
-    def __init__(self,
-                 analyses,
-                 prob_allow_miss=[0.1, 0.5, 2.0],
-                 result_format="percentage"):
-        """Class for the Limit plot."""
-        super(PlotLimit, self).__init__(analyses)
-
-        self.legend_plt = []
-        self.legend_name = []
-        linestyles = ['-', '--', '-.', ':']
-        self.result_format = result_format
-
-        for i, data_key in enumerate(self.analyses):
-            res = self.analyses[data_key].limits(
-                prob_allow_miss=prob_allow_miss, result_format=result_format)
-            x_range = res["x_range"]
-            col = "C" + str(i % 10)
-
-            for i_limit, limit in enumerate(res["limits"]):
-                ls = linestyles[i_limit % len(linestyles)]
-                my_plot, = self.ax.plot(x_range,
-                                        np.array(limit) + np.array(x_range),
-                                        color=col,
-                                        ls=ls)
-                if i_limit == 0:
-                    self.legend_plt.append(my_plot)
-                    self.legend_name.append(f"{data_key}")
-
-        self.ax.plot(x_range, x_range, color="black", ls='--')
-        if result_format == "percentage":
-            self.ax.set_xlabel("% of papers read")
-            self.ax.set_ylabel("Estimate of % of papers that need to be read")
-        else:
-            self.ax.set_xlabel("# of papers read")
-            self.ax.set_ylabel("Estimate of # of papers that need to be read")
-        self.ax.set_title("Articles left to read")
diff --git a/asreviewcontrib/visualization/plot_progression.py b/asreviewcontrib/visualization/plot_progression.py
deleted file mode 100644
index 783e19a..0000000
--- a/asreviewcontrib/visualization/plot_progression.py
+++ /dev/null
@@ -1,68 +0,0 @@
-import numpy as np
-
-from asreviewcontrib.visualization.plot_base import PlotBase
-
-
-def _gaussian_window(rel_ids, sigma):
-    factors = np.exp(-rel_ids**2 / sigma**2)
-    return factors / np.sum(factors)
-
-
-class PlotProgression(PlotBase):
-    def __init__(self,
-                 analyses,
-                 result_format="percentage",
-                 thick=None,
-                 sigma=25,
-                 window=40):
-        """Class for the Regression plot."""
-        super(PlotProgression, self).__init__(analyses)
-        self.col = {}
-        self.result_format = result_format
-
-        if thick is None:
-            thick = {key: True for key in list(self.analyses)}
-        self.thick = thick
-
-        for i, data_key in enumerate(reversed(self.analyses)):
-            analysis = self.analyses[data_key]
-            inc_found = analysis.inclusions_found(result_format="number")
-
-            dy_inc = (inc_found[1] - np.append([0], inc_found[1][:-1]))
-
-            self.col[data_key] = "C" + str((len(self.analyses) - 1 - i) % 10)
-            col = self.col[data_key]
-
-            if self.thick[data_key]:
-                lw = 2
-            else:
-                lw = 0.7
-
-            smooth_inc_perc = []
-            for i in range(len(dy_inc)):
-                idx = np.arange(max(0, i - window),
-                                min(len(dy_inc), i + window + 1))
-                factor = _gaussian_window(idx - i, sigma)
-                smooth_inc_perc.append(np.sum(dy_inc[idx] * factor))
-
-            if self.result_format == "percentage":
-                x_values = 100 * inc_found[0] / len(analysis.labels)
-            else:
-                x_values = inc_found[0]
-            myplot, = self.ax.plot(x_values,
-                                   100 * np.array(smooth_inc_perc),
-                                   color=col,
-                                   lw=lw)
-            if self.thick[data_key]:
-                self.legend_plt.append(myplot)
-                self.legend_name.append(data_key)
-
-        if self.result_format == "percentage":
-            self.ax.set_xlabel("% papers reviewed")
-        else:
-            self.ax.set_xlabel("# papers reviewed")
-
-        self.ax.set_ylabel("% of proposed papers accepted")
-
-    def set_legend(self, loc="upper right"):
-        super(PlotProgression, self).set_legend(loc=loc)
diff --git a/asreviewcontrib/visualization/quick.py b/asreviewcontrib/visualization/quick.py
deleted file mode 100644
index ffe3573..0000000
--- a/asreviewcontrib/visualization/quick.py
+++ /dev/null
@@ -1,73 +0,0 @@
-# Copyright 2020 The ASReview Authors. All Rights Reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-
-def inclusion_plot(plot, output=None, show_metric_labels=True, **kwargs):
-    """Make an inclusion plot."""
-    all_files = all(plot.is_file.values())
-
-    inc_plot = plot.new("inclusion", **kwargs)
-    inc_plot.set_grid()
-
-    for key in list(plot.analyses):
-        if all_files or not plot.is_file[key]:
-            inc_plot.add_wss(
-                key, 95, add_text=show_metric_labels, add_value=True)
-            inc_plot.add_rrf(
-                key, 10, add_text=show_metric_labels, add_value=True)
-    inc_plot.add_random(add_text=False)
-
-    # TODO {Make legend in flexible argument}
-    # inc_plot.set_legend()
-
-    if output is None:
-        inc_plot.show()
-    else:
-        inc_plot.save(output)
-        inc_plot.close()
-
-
-def progression_plot(plot, output=None, **kwargs):
-    """Make a progression plot."""
-    prog_plot = plot.new("progression", **kwargs)
-    prog_plot.set_grid()
-    prog_plot.set_legend()
-    if output is None:
-        prog_plot.show()
-    else:
-        prog_plot.save(output)
-        prog_plot.close()
-
-
-def discovery_plot(plot, output=None, **kwargs):
-    """Make a discovery plot."""
-    disc_plot = plot.new("discovery", **kwargs)
-    disc_plot.set_legend()
-    if output is None:
-        disc_plot.show()
-    else:
-        disc_plot.save(output)
-        disc_plot.close()
-
-
-def limit_plot(plot, output=None, **kwargs):
-    """Make a limit plot."""
-    limit_plot = plot.new("limit", **kwargs)
-    limit_plot.set_legend()
-    limit_plot.set_grid()
-    if output is None:
-        limit_plot.show()
-    else:
-        limit_plot.save(output)
-        limit_plot.close()
diff --git a/docs/discovery.png b/docs/discovery.png
deleted file mode 100644
index d71b5bb..0000000
Binary files a/docs/discovery.png and /dev/null differ
diff --git a/docs/example_absolute_axes.png b/docs/example_absolute_axes.png
new file mode 100644
index 0000000..db73d10
Binary files /dev/null and b/docs/example_absolute_axes.png differ
diff --git a/docs/example_absolute_axes.py b/docs/example_absolute_axes.py
new file mode 100644
index 0000000..1d0286f
--- /dev/null
+++ b/docs/example_absolute_axes.py
@@ -0,0 +1,11 @@
+import matplotlib.pyplot as plt
+
+from asreview import open_state
+from asreviewcontrib.insights.plot import plot_recall
+
+with open_state("tests/asreview_files/sim_van_de_schoot_2017_1.asreview") as s:
+
+    fig, ax = plt.subplots()
+    plot_recall(ax, s, x_relative=False, y_relative=False)
+
+    fig.savefig("docs/example_absolute_axes.png")
diff --git a/docs/example_custom_title.png b/docs/example_custom_title.png
new file mode 100644
index 0000000..6a83c2f
Binary files /dev/null and b/docs/example_custom_title.png differ
diff --git a/docs/example_custom_title.py b/docs/example_custom_title.py
new file mode 100644
index 0000000..4ff837b
--- /dev/null
+++ b/docs/example_custom_title.py
@@ -0,0 +1,14 @@
+import matplotlib.pyplot as plt
+
+from asreview import open_state
+from asreviewcontrib.insights.plot import plot_wss
+
+with open_state("tests/asreview_files/sim_van_de_schoot_2017_1.asreview") as s:
+
+    fig, ax = plt.subplots()
+
+    plot_wss(ax, s)
+
+    plt.title("WSS with custom title")
+
+    fig.savefig("docs/example_custom_title.png")
diff --git a/docs/example_multiple_lines.png b/docs/example_multiple_lines.png
new file mode 100644
index 0000000..af7378b
Binary files /dev/null and b/docs/example_multiple_lines.png differ
diff --git a/docs/example_multiple_lines.py b/docs/example_multiple_lines.py
new file mode 100644
index 0000000..e100db0
--- /dev/null
+++ b/docs/example_multiple_lines.py
@@ -0,0 +1,25 @@
+# An example of plotting the recall curves for two ASREVIEW files in one plot,
+# with a legend.
+
+import matplotlib.pyplot as plt
+
+from asreview import open_state
+from asreviewcontrib.insights.plot import plot_recall
+
+
+fig, ax = plt.subplots()
+
+with open_state("tests/asreview_files/sim_van_de_schoot_2017_1.asreview") as s1:
+    plot_recall(ax, s1)
+
+with open_state("tests/asreview_files/"
+                "sim_van_de_schoot_2017_logistic.asreview") as s2:
+    plot_recall(ax, s2)
+
+# Set the labels for the legend. Both plots add the recall line and the random
+# line. Hence the recall lines are the 0th and 2nd line.
+ax.lines[0].set_label("Naive Bayes")
+ax.lines[2].set_label("Logistic")
+ax.legend()
+
+fig.savefig("docs/example_multiple_lines.png")
diff --git a/docs/inclusions.png b/docs/inclusions.png
deleted file mode 100644
index 4f378c9..0000000
Binary files a/docs/inclusions.png and /dev/null differ
diff --git a/docs/limits.png b/docs/limits.png
deleted file mode 100644
index 1b6e492..0000000
Binary files a/docs/limits.png and /dev/null differ
diff --git a/docs/progression.png b/docs/progression.png
deleted file mode 100644
index d174094..0000000
Binary files a/docs/progression.png and /dev/null differ
diff --git a/docs/stats_explainer.png b/docs/stats_explainer.png
new file mode 100644
index 0000000..a82f4db
Binary files /dev/null and b/docs/stats_explainer.png differ
diff --git a/docs/stats_explainer.py b/docs/stats_explainer.py
new file mode 100644
index 0000000..b80c3f4
--- /dev/null
+++ b/docs/stats_explainer.py
@@ -0,0 +1,76 @@
+"""Create example plot with different metrics.
+
+Example
+-------
+
+python docs/stats_explainer.py
+"""
+
+
+import numpy as np
+import matplotlib.pyplot as plt
+
+from asreviewcontrib.insights.plot import _fix_start_tick
+
+# The recall at a given number of documents read is the fraction of the
+# relevant records found at that moment. (recall = n_pos_records / n_records)
+
+# The old RRF@X (relevant records found) is basically the same as the recall.
+
+# The WSS@X (work saved over sampling) is the number of records you need to
+# read less to find the fraction X of relevant records. (wss = recall - recall_random)
+
+# The (my suggestion for a name) ERF@X (extra records found) is the number of
+# extra relevant records found after reading a fraction X of the total number of
+# records.
+
+# Create fictive data.
+n_docs = 1000
+n_pos_docs = 30
+
+percentages = np.array([x**(1 / 3) for x in np.linspace(0, 1, n_docs)])
+n_docs_found = np.round(percentages * n_pos_docs)
+labels = [
+    n_docs_found[i + 1] - n_docs_found[i]
+    for i in range(len(n_docs_found) - 1)
+] + [0]
+labels[0] = 1
+labels[5] = 1
+labels[8] = 1
+
+# Plot the recall curve.
+fig, ax = plt.subplots()
+
+x = list(range(1, n_docs + 1))
+
+# Recall curve.
+recall = np.cumsum(labels) / np.sum(labels)
+ax.step(x, recall, where='post')
+
+# Random
+recall_random = np.round(np.linspace(0, n_pos_docs, n_docs)) / np.sum(labels)
+ax.step(x, recall_random, where='post', color="black")
+
+# Add the ERF@.137 line (recall > 0.5 at 137, recall_random 0.5 at 517).
+ax.plot((137, 137), (137 / 1000, recall[137]), color='red')
+erf_x_offset = -70
+ax.text(137 + erf_x_offset, (137 / 1000 + recall[137]) * 0.9 / 2,
+        'ERF',
+        color='red')
+# Add the WSS@.5 line.
+ax.plot((137, 517), (recall[137], recall[137]), color='blue')
+wss_y_offset = 0.03
+ax.text((137 + recall[137] * 1000) / 2,
+        recall[137] + wss_y_offset,
+        'WSS',
+        color='blue')
+
+ax.set_title("Explaining Recall, WSS and ERF")
+ax.set(xlabel='#', ylabel='Recall')
+ax.set_ylim([-0.05, 1.05])
+ax.set_yticks([0, 0.2, 0.4, 0.6, 0.8, 1.0])
+ax.xaxis.get_major_locator().set_params(integer=True)
+
+_fix_start_tick(ax)
+
+fig.savefig('docs/stats_explainer.png')
diff --git a/output-schema.json b/output-schema.json
new file mode 100644
index 0000000..92ba96b
--- /dev/null
+++ b/output-schema.json
@@ -0,0 +1,62 @@
+{
+  "$schema": "http://json-schema.org/draft-04/schema#",
+  "type": "object",
+  "properties": {
+    "metrics": {
+      "type": "object",
+      "properties": {
+        "wss": {
+          "type": "array",
+          "items": [
+            {
+              "type": "object",
+              "properties": {
+                "type": {
+                  "type": "string"
+                },
+                "valueType": {
+                  "type": "number"
+                },
+                "value": {
+                  "type": "number"
+                }
+              },
+              "required": [
+                "type",
+                "valueType",
+                "value"
+              ]
+            }
+          ]
+        },
+        "rrf": {
+          "type": "array",
+          "items": [
+            {
+              "type": "object",
+              "properties": {
+                "valueType": {
+                  "type": "number"
+                },
+                "value": {
+                  "type": "number"
+                }
+              },
+              "required": [
+                "valueType",
+                "value"
+              ]
+            }
+          ]
+        }
+      },
+      "required": [
+        "wss",
+        "rrf"
+      ]
+    }
+  },
+  "required": [
+    "metrics"
+  ]
+}
diff --git a/setup.cfg b/setup.cfg
index 1be3d79..85b69fd 100644
--- a/setup.cfg
+++ b/setup.cfg
@@ -1,10 +1,10 @@
 [versioneer]
 VCS = git
 style = pep440
-versionfile_source = asreviewcontrib/visualization/_version.py
-versionfile_build = asreviewcontrib/visualization/_version.py
+versionfile_source = asreviewcontrib/insights/_version.py
+versionfile_build = asreviewcontrib/insights/_version.py
 tag_prefix = v
-parentdir_prefix = asreview-visualization-
+parentdir_prefix = asreview-insights-
 
 [flake8]
 max-line-length = 88
@@ -12,4 +12,4 @@ ignore =
     E402,  # module level import not at top of file
 exclude =
     versioneer.py,
-    asreviewcontrib/visualization/_version.py
+    asreviewcontrib/insights/_version.py
diff --git a/setup.py b/setup.py
index eb92974..581aee1 100644
--- a/setup.py
+++ b/setup.py
@@ -15,13 +15,13 @@
     long_description = f.read()
 
 setup(
-    name='asreview-visualization',
+    name='asreview-insights',
     version=versioneer.get_version(),
     cmdclass=versioneer.get_cmdclass(),
-    description='Visualization tools for the ASReview project',
+    description='Insight tools for the ASReview project',
     long_description=long_description,
     long_description_content_type='text/markdown',
-    url='https://github.com/asreview/asreview-visualization',
+    url='https://github.com/asreview/asreview-insights',
     author='Utrecht University',
     author_email='asreview@uu.nl',
     classifiers=[
@@ -34,11 +34,11 @@
         # Pick your license as you wish
         'License :: OSI Approved :: Apache Software License',
 
-        'Programming Language :: Python :: 3.5',
-        'Programming Language :: Python :: 3.6',
         'Programming Language :: Python :: 3.7',
+        'Programming Language :: Python :: 3.8',
+        'Programming Language :: Python :: 3.9',
     ],
-    keywords='asreview plot visualization',
+    keywords='asreview plot insights',
     packages=find_namespace_packages(include=['asreviewcontrib.*']),
     install_requires=[
         "numpy", "matplotlib", "asreview",
@@ -49,14 +49,15 @@
 
     entry_points={
         "asreview.entry_points": [
-            "plot = asreviewcontrib.visualization.entrypoint:PlotEntryPoint",
+            "plot = asreviewcontrib.insights.entrypoint:PlotEntryPoint",
+            "stats = asreviewcontrib.insights.entrypoint:StatsEntryPoint",
         ]
     },
 
     project_urls={
         'Bug Reports':
-            "https://github.com/asreview/asreview-visualization/issues",
+            "https://github.com/asreview/asreview-insights/issues",
         'Source':
-            "https://github.com/asreview/asreview-visualization",
+            "https://github.com/asreview/asreview-insights",
     },
 )


Test patch:
diff --git a/figures/tests_erf_default_sim_van_de_schoot_2017_1.png b/figures/tests_erf_default_sim_van_de_schoot_2017_1.png
new file mode 100644
index 0000000..ecfdbb7
Binary files /dev/null and b/figures/tests_erf_default_sim_van_de_schoot_2017_1.png differ
diff --git a/figures/tests_erf_xabs_sim_van_de_schoot_2017_1.png b/figures/tests_erf_xabs_sim_van_de_schoot_2017_1.png
new file mode 100644
index 0000000..7dc68f0
Binary files /dev/null and b/figures/tests_erf_xabs_sim_van_de_schoot_2017_1.png differ
diff --git a/figures/tests_erf_xyabs_sim_van_de_schoot_2017_1.png b/figures/tests_erf_xyabs_sim_van_de_schoot_2017_1.png
new file mode 100644
index 0000000..7bae89b
Binary files /dev/null and b/figures/tests_erf_xyabs_sim_van_de_schoot_2017_1.png differ
diff --git a/figures/tests_erf_yabs_sim_van_de_schoot_2017_1.png b/figures/tests_erf_yabs_sim_van_de_schoot_2017_1.png
new file mode 100644
index 0000000..fa92be7
Binary files /dev/null and b/figures/tests_erf_yabs_sim_van_de_schoot_2017_1.png differ
diff --git a/figures/tests_recall_sim_van_de_schoot_2017_1.png b/figures/tests_recall_sim_van_de_schoot_2017_1.png
new file mode 100644
index 0000000..00a66ac
Binary files /dev/null and b/figures/tests_recall_sim_van_de_schoot_2017_1.png differ
diff --git a/figures/tests_recall_wss_sim_van_de_schoot_2017_1.png b/figures/tests_recall_wss_sim_van_de_schoot_2017_1.png
new file mode 100644
index 0000000..183f9ab
Binary files /dev/null and b/figures/tests_recall_wss_sim_van_de_schoot_2017_1.png differ
diff --git a/figures/tests_small_dataset_erf.png b/figures/tests_small_dataset_erf.png
new file mode 100644
index 0000000..2b006cb
Binary files /dev/null and b/figures/tests_small_dataset_erf.png differ
diff --git a/figures/tests_small_dataset_recall.png b/figures/tests_small_dataset_recall.png
new file mode 100644
index 0000000..dcb73d9
Binary files /dev/null and b/figures/tests_small_dataset_recall.png differ
diff --git a/figures/tests_small_dataset_wss.png b/figures/tests_small_dataset_wss.png
new file mode 100644
index 0000000..af99d7d
Binary files /dev/null and b/figures/tests_small_dataset_wss.png differ
diff --git a/figures/tests_wss_default_sim_van_de_schoot_2017_1.png b/figures/tests_wss_default_sim_van_de_schoot_2017_1.png
new file mode 100644
index 0000000..f768f9f
Binary files /dev/null and b/figures/tests_wss_default_sim_van_de_schoot_2017_1.png differ
diff --git a/figures/tests_wss_xabs_sim_van_de_schoot_2017_1.png b/figures/tests_wss_xabs_sim_van_de_schoot_2017_1.png
new file mode 100644
index 0000000..54212b7
Binary files /dev/null and b/figures/tests_wss_xabs_sim_van_de_schoot_2017_1.png differ
diff --git a/figures/tests_wss_xyabs_sim_van_de_schoot_2017_1.png b/figures/tests_wss_xyabs_sim_van_de_schoot_2017_1.png
new file mode 100644
index 0000000..777da52
Binary files /dev/null and b/figures/tests_wss_xyabs_sim_van_de_schoot_2017_1.png differ
diff --git a/figures/tests_wss_yabs_sim_van_de_schoot_2017_1.png b/figures/tests_wss_yabs_sim_van_de_schoot_2017_1.png
new file mode 100644
index 0000000..851ed62
Binary files /dev/null and b/figures/tests_wss_yabs_sim_van_de_schoot_2017_1.png differ
diff --git a/tests/README.md b/tests/README.md
new file mode 100644
index 0000000..be200be
--- /dev/null
+++ b/tests/README.md
@@ -0,0 +1,7 @@
+# Testing ASReview-insights
+
+## Create ASReview files
+
+```
+asreview simulate benchmark:van_de_schoot_2017 -s sim_van_de_schoot_2017.asreview --seed_init 535
+```
diff --git a/tests/asreview_files/sim_van_de_schoot_2017_1.asreview b/tests/asreview_files/sim_van_de_schoot_2017_1.asreview
new file mode 100644
index 0000000..533ca3d
Binary files /dev/null and b/tests/asreview_files/sim_van_de_schoot_2017_1.asreview differ
diff --git a/tests/asreview_files/sim_van_de_schoot_2017_logistic.asreview b/tests/asreview_files/sim_van_de_schoot_2017_logistic.asreview
new file mode 100644
index 0000000..38bdf30
Binary files /dev/null and b/tests/asreview_files/sim_van_de_schoot_2017_logistic.asreview differ
diff --git a/tests/data/embase_labelled.csv b/tests/data/embase_labelled.csv
deleted file mode 100644
index 797a1ef..0000000
--- a/tests/data/embase_labelled.csv
+++ /dev/null
@@ -1,7 +0,0 @@
-Title,Original Title,Author Names,Correspondence Address,Editors,Source,Abstract,Original Abstract,Emtree Drug Index Terms (Major Focus),Emtree Drug Index Terms,Emtree Medical Index Terms (Major Focus),Emtree Medical Index Terms,keywords,Drug Tradenames,Drug Manufacturer,Device Tradenames,Device Manufacturer,Clinical Trial Numbers,Open URL Link,Copyright,included
-"Duplication cysts: Diagnosis, management, and the role of endoscopic ultrasound",,"Liu R., Adler D.G.","D.G. Adler, Department of Gastroenterology and Hepatology, University of Utah School of Medicine, Huntsman Cancer Center, Salt Lake City, UT, United States. Email: douglas.adler@hsc.utah.edu",,Endoscopic Ultrasound (2014) 3:3 (152-160). Date of Publication: 2014,"Gastrointestinal tract duplication cysts are rare congenital gastrointestinal malformation in young patients and adults. They consist of foregut duplication cysts, small bowel duplication cysts, and large bowel duplication cysts. Endoscopic ultrasound (EUS) has been widely used as a modality for the evaluation and diagnosis of duplication cysts. EUS is the diagnostic tool of choice to investigate duplication cysts since it can distinguish between solid and cystic lesions. The question of whether or not to perform EUS-fine needle aspiration (EUS-FNA) on a lesion suspected of being a duplication cyst is controversial as these lesions can become infected with significant consequences, although EUS-FNA is often required to obtain a definitive diagnosis and to rule out more ominous lesions. This manuscript will review the literature on duplication cysts throughout the body and will also focus on the role of EUS and FNA with regards to these lesions.",,,,"endoscopic ultrasonography, gastrointestinal malformation (diagnosis), gastrointestinal tract duplication cyst (diagnosis)","article, bronchogenic duplication cyst (diagnosis, surgery), colonic duplication cyst (diagnosis, surgery), colonoscopy, computer assisted tomography, cytology, duodenal duplication cyst (diagnosis, surgery), endoscopic ultrasound guided fine needle biopsy, enucleation, esophageal duplication cyst (diagnosis, surgery), gastric duplication cyst (diagnosis), gastrointestinal mucosa, hemicolectomy, histology, human, ilial duplication cyst (diagnosis), intestine surgery, jejunal duplication cyst (diagnosis, surgery), symptomatology","article, bronchogenic duplication cyst (diagnosis, surgery), colonic duplication cyst (diagnosis, surgery), colonoscopy, computer assisted tomography, cytology, duodenal duplication cyst (diagnosis, surgery), endoscopic ultrasound guided fine needle biopsy, enucleation, esophageal duplication cyst (diagnosis, surgery), gastric duplication cyst (diagnosis), gastrointestinal mucosa, hemicolectomy, histology, human, ilial duplication cyst (diagnosis), intestine surgery, jejunal duplication cyst (diagnosis, surgery), symptomatology",,,,,,http://sfx.library.uu.nl/utrecht?sid=EMBASE&issn=22267190&id=doi:10.4103%2F2303-9027.138783&atitle=Duplication+cysts%3A+Diagnosis%2C+management%2C+and+the+role+of+endoscopic+ultrasound&stitle=Endoscopic+Ultrasound&title=Endoscopic+Ultrasound&volume=3&issue=3&spage=152&epage=160&aulast=Liu&aufirst=Roy&auinit=R.&aufull=Liu+R.&coden=&isbn=&pages=152-160&date=2014&auinit1=R&auinitm=,"Copyright 2014 Elsevier B.V., All rights reserved.",0
-Endoscopic ultrasound-guided fine-needle aspiration in the diagnosis of foregut duplication cysts: The value of demonstrating detached ciliary tufts in cyst fluid,,"Eloubeidi M.A., Cohn M., Cerfolio R.J., Chhieng D.C., Jhala N., Jhala D., Eltoum I.A.","M.A. Eloubeidi, Endoscopic Ultrasound Program, Div. of Gastroenterol. and Hepatol., University of Alabama-Birmingham, 1530 3rd Avenue South-ZRB 633, Birmingham, AL 35294-0007, United States. Email: meloubeidi@uabmc.edu",,Cancer (2004) 102:4 (253-258). Date of Publication: 25 Aug 2004,,,,,"cyst fluid, endoscopic ultrasonography, fine needle aspiration biopsy, foregut duplication cyst (diagnosis), mediastinum cyst (diagnosis)","adult, aged, article, cancer cell, cancer cytodiagnosis, ciliary tuft, computer assisted tomography, conservative treatment, diagnostic accuracy, diagnostic value, electron microscopy, evaluation study, female, histopathology, human, major clinical study, male, mediastinum mass, priority journal, risk assessment, solid malignant neoplasm, symptomatology","adult, aged, article, cancer cell, cancer cytodiagnosis, ciliary tuft, computer assisted tomography, conservative treatment, diagnostic accuracy, diagnostic value, electron microscopy, evaluation study, female, histopathology, human, major clinical study, male, mediastinum mass, priority journal, risk assessment, solid malignant neoplasm, symptomatology",,,,,,http://sfx.library.uu.nl/utrecht?sid=EMBASE&issn=0008543X&id=doi:10.1002%2Fcncr.20369&atitle=Endoscopic+ultrasound-guided+fine-needle+aspiration+in+the+diagnosis+of+foregut+duplication+cysts%3A+The+value+of+demonstrating+detached+ciliary+tufts+in+cyst+fluid&stitle=Cancer&title=Cancer&volume=102&issue=4&spage=253&epage=258&aulast=Eloubeidi&aufirst=Mohamad+A.&auinit=M.A.&aufull=Eloubeidi+M.A.&coden=CANCA&isbn=&pages=253-258&date=2004&auinit1=M&auinitm=A,"Copyright 2009 Elsevier B.V., All rights reserved.",1
-A case of completely isolated advanced enteric duplication cyst cancer performed partial pancreatectomy,,"Nakashima S., Yamada T., Sato G., Sakai T., Chinen Y., Itakura H., Kato R., Ueda M., Tsuda Y., Ohta K., Matsuyama J., Ikenaga M.","T. Yamada, Department of Gastroenterological Surgery, Higashiosaka City Medical Center, Nishiiwata 3-4-5, Higashiosaka, Japan. Email: yamada-t@higashiosaka-hosp.jp",,International Journal of Surgery Case Reports (2019) 54 (83-86). Date of Publication: 1 Jan 2019,"Introduction: Enteric duplication cysts are rare and, in addition, isolated enteric duplication cysts are lower morbidity prevalence rate. These cysts lack a connection to the gastrointestinal tract or the adjacent mesenteric vasculature and have only been reported in 10 case reports. In these reports, only two reports were cases with malignant transformation. Our case was a report for the advanced cancer of the isolated enteric duplication cyst. Case presentation: The patient was a 43 year-old woman with slightly abdominal pain and mass formation. The abdominal contrast-enhanced computed tomography showed 130 × 100 × 90 mm huge cystic mass existed in right upper peritoneal cavity. The cystic mass had thickened wall and many enhanced nodules. As these imaging findings suggested a tumor originated from pancreas and the preoperative diagnose was suspect of mucinous cystic neoplasm. In operative findings, the tumor originated from pancreatic head and did not attach to gastrointestinal tract. Final pathology indicated the cyst was an isolated advanced enteric duplication cyst cancer and not originated from pancreas. Conclusion: We experienced an extremely rare case of completely isolated advanced enteric duplication cyst cancer. Unique to this case, the preoperative diagnosis was suspect of mucinous cystic neoplasm arising from pancreas head and partial pancreatectomy was performed. However, in the pathological findings, this cyst diagnosed advanced enteric duplication cyst cancer.",,,"cytokeratin 20 (endogenous compound), cytokeratin 7 (endogenous compound), gadolinium, gimeracil plus oteracil potassium plus tegafur (drug therapy), transcription factor Cdx2 (endogenous compound)","advanced cancer (diagnosis, drug therapy, surgery), cyst (diagnosis, drug therapy, surgery), intestine duplication (diagnosis, surgery)","abdominal pain, abdominal tenderness, abdominal tumor, adjuvant chemotherapy, adult, article, cancer diagnosis, cancer surgery, case report, clinical article, clinical examination, computer assisted tomography, consensus, contrast enhancement, differential diagnosis, diffusion weighted imaging, enteric duplication cyst (drug therapy), female, human, human tissue, immunohistochemistry, lymph node dissection, lymphatic system, malignant transformation, mucinous cystic neoplasm (diagnosis), nervous system, pancreatectomy, pancreaticoduodenectomy, pathology, peritoneal cavity, physical examination, practice guideline, priority journal, tumor invasion, upper abdominal pain, venous circulation","abdominal pain, abdominal tenderness, abdominal tumor, adjuvant chemotherapy, adult, article, cancer diagnosis, cancer surgery, case report, clinical article, clinical examination, computer assisted tomography, consensus, contrast enhancement, differential diagnosis, diffusion weighted imaging, enteric duplication cyst (drug therapy), female, human, human tissue, immunohistochemistry, lymph node dissection, lymphatic system, malignant transformation, mucinous cystic neoplasm (diagnosis), nervous system, pancreatectomy, pancreaticoduodenectomy, pathology, peritoneal cavity, physical examination, practice guideline, priority journal, tumor invasion, upper abdominal pain, venous circulation",s 1,,,,,http://sfx.library.uu.nl/utrecht?sid=EMBASE&issn=22102612&id=doi:10.1016%2Fj.ijscr.2018.11.060&atitle=A+case+of+completely+isolated+advanced+enteric+duplication+cyst+cancer+performed+partial+pancreatectomy&stitle=Int.+J.+Surg.+Case+Rep.&title=International+Journal+of+Surgery+Case+Reports&volume=54&issue=&spage=83&epage=86&aulast=Nakashima&aufirst=Shinsuke&auinit=S.&aufull=Nakashima+S.&coden=&isbn=&pages=83-86&date=2019&auinit1=S&auinitm=,"Copyright 2018 Elsevier B.V., All rights reserved.",0
-,,"Quintanilla-Dieck L., Penn E.B.","L. Quintanilla-Dieck, Department of Otolaryngology Head and Neck Surgery, Oregon Health & Science University, 3181 Southwest Sam Jackson Park Road, PV-01, Portland, United States. Email: quintani@ohsu.edu",,Clinics in Perinatology (2018) 45:4 (769-785). Date of Publication: 1 Dec 2018,"Congenital neck masses can be a developmental anomaly of cystic, solid, or vascular origin. They can also constitute neoplasms, including malignancies, although this is rare in the pediatric population. The history and examination can help quickly narrow the differential diagnosis. Imaging also plays an essential role in defining the characteristics and likely cause of neck masses. The most common neck masses in young children are thyroglossal duct cysts, branchial cleft anomalies, and dermoid cysts. Also important to consider in the differential diagnosis are solid tumors, such as teratomas, or vascular lesions, such as hemangiomas.",,,"antineoplastic agent (drug therapy), bleomycin (drug therapy), doxycycline (drug therapy), propranolol (drug therapy), tetradecyl sulfate sodium (drug therapy), thyroid hormone (drug therapy)","congenital tumor (congenital disorder), neck tumor (congenital disorder)","ablation therapy, branchial defect (congenital disorder, surgery), cancer chemotherapy, cancer radiotherapy, cancer surgery, capillary hemangioma (congenital disorder, diagnosis, drug therapy, surgery), carbon dioxide laser, congenital blood vessel malformation (congenital disorder, surgery), cyst (congenital disorder, surgery), differential diagnosis, ectopic thyroid gland (congenital disorder, drug therapy, surgery, therapy), epidermoid cyst (congenital disorder, surgery), foregut duplication cyst (congenital disorder, surgery), head and neck tumor (congenital disorder, diagnosis), hormone substitution, human, lung cyst (congenital disorder), lymphatic malformation (congenital disorder, diagnosis, drug therapy, surgery, therapy), prenatal diagnosis, priority journal, respiration control, review, rhabdomyosarcoma (congenital disorder, drug therapy, radiotherapy, surgery), sclerotherapy, Sistrunk procedure, surgical approach, surgical technique, teratoma (congenital disorder, surgery), thymus cyst (congenital disorder, surgery), thyroglossal duct cyst (congenital disorder, surgery), thyroidectomy","ablation therapy, branchial defect (congenital disorder, surgery), cancer chemotherapy, cancer radiotherapy, cancer surgery, capillary hemangioma (congenital disorder, diagnosis, drug therapy, surgery), carbon dioxide laser, congenital blood vessel malformation (congenital disorder, surgery), cyst (congenital disorder, surgery), differential diagnosis, ectopic thyroid gland (congenital disorder, drug therapy, surgery, therapy), epidermoid cyst (congenital disorder, surgery), foregut duplication cyst (congenital disorder, surgery), head and neck tumor (congenital disorder, diagnosis), hormone substitution, human, lung cyst (congenital disorder), lymphatic malformation (congenital disorder, diagnosis, drug therapy, surgery, therapy), prenatal diagnosis, priority journal, respiration control, review, rhabdomyosarcoma (congenital disorder, drug therapy, radiotherapy, surgery), sclerotherapy, Sistrunk procedure, surgical approach, surgical technique, teratoma (congenital disorder, surgery), thymus cyst (congenital disorder, surgery), thyroglossal duct cyst (congenital disorder, surgery), thyroidectomy",,,,,,http://sfx.library.uu.nl/utrecht?sid=EMBASE&issn=15579840&id=doi:10.1016%2Fj.clp.2018.07.012&atitle=Congenital+Neck+Masses&stitle=Clin.+Perinatol.&title=Clinics+in+Perinatology&volume=45&issue=4&spage=769&epage=785&aulast=Quintanilla-Dieck&aufirst=Lourdes&auinit=L.&aufull=Quintanilla-Dieck+L.&coden=CLPED&isbn=&pages=769-785&date=2018&auinit1=L&auinitm=,"Copyright 2018 Elsevier B.V., All rights reserved.",1
-Foregut duplication cysts: A report of two cases with emphasis on embryogenesis,,"Khoury T., Rivera L.","T. Khoury, Department of Pathology, Roswell Park Cancer Institute, Elm and Carlton Streets, Buffalo, NY 14263, United States. Email: thaer.khoury@roswellpark.org",,"World Journal of Gastroenterology (2011) 17:1 (130-134). Date of Publication: January 7, 2011","Duplication cyst of the stomach with a pseudostratified columnar ciliated epithelium is extremely rare. We describe two cases of these cysts, with emphasis on their immunophenotype and embryogenesis. The first patient was a 29-year-old man who presented with cramping abdominal pain in his left lower quadrant. The second patient was a 26-year-old woman who had a history, over several years, of chronic epigastric abdominal pain radiating to her back. Both lesions were surgically removed. They showed the same histomorphology. The cysts were lined by a pseudostratified respiratory epithelium with ciliated cells. The first cyst was connected to the stomach, while the second cyst was not connected. Both cysts expressed thyroid transcription factor-1 (TTF-1) and surfactant. In this report, we explore the possible embryogenesis of these lesions in the light of TTF-1 and surfactant expression. © 2011 Baishideng. All rights reserved.",,,"homeobox protein Nkx 2.1 (endogenous compound), surfactant (endogenous compound)","cyst (diagnosis, surgery), gastric duplication cyst (diagnosis, surgery), gastrointestinal duplication cyst (diagnosis, surgery)","abdominal pain, adult, article, case report, computer assisted tomography, embryo development, endoscopic ultrasonography, epigastric pain, female, human, immunophenotyping, male, partial gastrectomy, protein expression","abdominal pain, adult, article, case report, computer assisted tomography, embryo development, endoscopic ultrasonography, epigastric pain, female, human, immunophenotyping, male, partial gastrectomy, protein expression",,,,,,http://sfx.library.uu.nl/utrecht?sid=EMBASE&issn=10079327&id=doi:10.3748%2Fwjg.v17.i1.130&atitle=Foregut+duplication+cysts%3A+A+report+of+two+cases+with+emphasis+on+embryogenesis&stitle=World+J.+Gastroenterol.&title=World+Journal+of+Gastroenterology&volume=17&issue=1&spage=130&epage=134&aulast=Khoury&aufirst=Thaer&auinit=T.&aufull=Khoury+T.&coden=WJGAF&isbn=&pages=130-134&date=2011&auinit1=T&auinitm=,"Copyright 2011 Elsevier B.V., All rights reserved.",0
-Completely Isolated Retroperitoneal Enteric Duplication Cyst with Adenocarcinoma Transformation Managed with Robotic Radical Nephrectomy,,"Faraj K., Edwards L., Gupta A., Seifman B.","B. Seifman, Michigan Institute of Urology, 1701 E South Boulevard, Rochester Hills, United States. Email: seifmanb@michiganurology.com",,Journal of Endourology Case Reports (2017) 3:1 (31-33). Date of Publication: 1 Mar 2017,"Background: Enteric duplication cysts are congenital malformations that typically affect children in infancy, but can also affect adults. Rarely, these cysts can be complicated by malignancy. We present the first case of retroperitoneal duplication cyst that was complicated by malignancy transformation and managed by robot-assisted excision. Case presentation: A 64-year-old female with a history of a left-sided renal cyst presented with a 4-month history of abdominal pain and fatigue. MRI revealed a bilobed cyst, with components measuring 6.9 × 6.6 and 6.1 × 6.9 cm, which had grown since previous imaging, and hemorrhage in some portions of the cysts, as well as cystic wall enhancement, suggesting a possible malignancy. The patient consented to a robot-assisted partial (possible radical) nephrectomy. During the procedure, the cystic structure appeared to have grown since imaging, was intimately associated with the hilum, and had a complex vasculature, which prompted us to perform a radical nephrectomy. Grossly, the specimen consisted of a 14.8 cm cystic structure at the superior portion of the kidney, but was not contained within the renal parenchyma. Histologically, the internal mucosa of the cyst showed columnar epithelium with high-grade dysplasia and carcinoma in situ with focal individual cell infiltration into the superficial portion of the inferior part of the cyst. The patient saw a medical oncologist and was instructed to follow up with quarterly imaging to assess for disease progression. Conclusion: Enteric duplication cysts are uncommon entities that can occur in various locations in the body, causing a wide spectrum of symptoms, and are rarely complicated by malignancy transformation. Robot-assisted surgical resection is an option that we have shown to be effective in managing these patients.",,,"2 methylacyl coenzyme A racemase (endogenous compound), cytokeratin 20 (endogenous compound), cytokeratin 7 (endogenous compound), kidney injury molecule 1 (endogenous compound), protein p53 (endogenous compound), transcription factor Cdx2 (endogenous compound), transcription factor PAX8 (endogenous compound)","colloid carcinoma (diagnosis), cyst (congenital disorder, diagnosis, surgery), enteric duplication cyst (congenital disorder, diagnosis, surgery), malignant transformation, radical nephrectomy, robotic surgical procedure","abdominal pain, adult, artery ligation, article, bleeding (diagnosis), cancer risk, case report, cell infiltration, clinical article, columnar epithelium, computer assisted tomography, cyst hemorrhage (diagnosis), disease exacerbation, fatigue, female, follow up, general anesthesia, histopathology, human, human tissue, middle aged, nuclear magnetic resonance imaging, priority journal, renal artery ligation, vascularization","abdominal pain, adult, artery ligation, article, bleeding (diagnosis), cancer risk, case report, cell infiltration, clinical article, columnar epithelium, computer assisted tomography, cyst hemorrhage (diagnosis), disease exacerbation, fatigue, female, follow up, general anesthesia, histopathology, human, human tissue, middle aged, nuclear magnetic resonance imaging, priority journal, renal artery ligation, vascularization",,,,,,http://sfx.library.uu.nl/utrecht?sid=EMBASE&issn=23799889&id=doi:10.1089%2Fcren.2017.0016&atitle=Completely+Isolated+Retroperitoneal+Enteric+Duplication+Cyst+with+Adenocarcinoma+Transformation+Managed+with+Robotic+Radical+Nephrectomy&stitle=J.+Endourol.+Case+Rep.&title=Journal+of+Endourology+Case+Reports&volume=3&issue=1&spage=31&epage=33&aulast=Faraj&aufirst=Kassem&auinit=K.&aufull=Faraj+K.&coden=&isbn=&pages=31-33&date=2017&auinit1=K&auinitm=,"Copyright 2018 Elsevier B.V., All rights reserved.",0
diff --git a/tests/test_metrics.py b/tests/test_metrics.py
new file mode 100644
index 0000000..3e54a1e
--- /dev/null
+++ b/tests/test_metrics.py
@@ -0,0 +1,93 @@
+from pathlib import Path
+
+from asreview import open_state
+
+from numpy.testing import assert_almost_equal
+
+from asreviewcontrib.insights.metrics import recall, _recall
+
+
+TEST_ASREVIEW_FILES = Path("tests", "asreview_files")
+TEST_FIGURES = Path("figures")
+
+
+def setup():
+
+    TEST_FIGURES.mkdir(exist_ok=True)
+
+
+def test_metric_recall_small_data():
+
+    labels = [1, 1, 1, 0]
+    r = _recall(labels, 0.5)
+    assert_almost_equal(r, 0.66666667)
+
+    r = _recall(labels, 2, x_relative=False)
+    assert_almost_equal(r, 0.66666667)
+
+    r = _recall(labels, 2, x_relative=False, y_relative=False)
+    assert_almost_equal(r, 2)
+
+    r = _recall(labels, 0.5, y_relative=False)
+    assert_almost_equal(r, 2)
+
+
+def test_metric_recall_max_values():
+
+    labels = [1, 1, 1, 0]
+    r = _recall(labels, 1)
+    assert_almost_equal(r, 1)
+
+    r = _recall(labels, 4, x_relative=False)
+    assert_almost_equal(r, 1)
+
+    r = _recall(labels, 4, x_relative=False, y_relative=False)
+    assert_almost_equal(r, 3)
+
+    r = _recall(labels, 1, y_relative=False)
+    assert_almost_equal(r, 3)
+
+
+def test_metric_recall_min_values():
+
+    labels = [1, 1, 1, 0]
+    r = _recall(labels, 0)
+    assert_almost_equal(r, 0)
+
+    r = _recall(labels, 0, x_relative=False)
+    assert_almost_equal(r, 0)
+
+    r = _recall(labels, 0, x_relative=False, y_relative=False)
+    assert_almost_equal(r, 0)
+
+    r = _recall(labels, 0, y_relative=False)
+    assert_almost_equal(r, 0)
+
+
+def test_metric_recall_invalid_values_min():
+
+    labels = [1, 1, 1, 0]
+    r = _recall(labels, -1)
+    assert_almost_equal(r, 0)
+
+    r = _recall(labels, 0.2, x_relative=False)
+    assert_almost_equal(r, 0)
+
+
+def test_metric_recall_invalid_values_max():
+
+    labels = [1, 1, 1, 0]
+    r = _recall(labels, 6, x_relative=False)
+    assert_almost_equal(r, 1)
+
+    r = _recall(labels, 1.2)
+    assert_almost_equal(r, 1)
+
+
+def test_metric_recall():
+
+    with open_state(
+            Path(TEST_ASREVIEW_FILES,
+                 "sim_van_de_schoot_2017_1.asreview")) as s:
+
+        assert_almost_equal(recall(s, 0.25), 1)
diff --git a/tests/test_plot.py b/tests/test_plot.py
index 6951daa..d97a603 100644
--- a/tests/test_plot.py
+++ b/tests/test_plot.py
@@ -1,84 +1,125 @@
-import os
 from pathlib import Path
 
-from pytest import mark
-
-from asreview.review.factory import review_simulate
-
-from asreviewcontrib.visualization.entrypoint import PlotEntryPoint
-
-
-def create_state_file(data_fp, state_fp):
-    try:
-        os.remove(state_fp)
-    except FileNotFoundError:
-        pass
-
-    review_simulate(str(data_fp),
-                    state_file=state_fp,
-                    n_prior_included=1,
-                    n_prior_excluded=1,
-                    model="nb",
-                    feature_extraction="tfidf")
-
-
-def plot_setup(data_fp, state_dirs, state_files):
-    for dir_ in state_dirs:
-        os.makedirs(dir_, exist_ok=True)
-    for file_ in state_files:
-        create_state_file(data_fp, file_)
-
-
-def plot_clean(dirs, files):
-    for f in files:
-        try:
-            os.remove(f)
-        except FileNotFoundError:
-            pass
-    for d in dirs:
-        try:
-            os.rmdir(d)
-        except (FileNotFoundError, OSError):
-            pass
-
-
-COMBINATIONS = [(pt, numbers)
-                for pt in ["inclusion", "progress", "discovery", "limit"]
-                for numbers in [True, False]]
-
-
-@mark.parametrize("plot_type,numbers", COMBINATIONS)
-def test_plots(request, plot_type, numbers):
-    test_dir = request.fspath.dirname
-    output_dir = Path(test_dir, "output")
-    state_dirs = [Path(output_dir, x) for x in ["h5", "json"]]
-    h5_dir = Path(output_dir, "h5")
-    json_dir = Path(output_dir, "json")
-    h5_files = [Path(output_dir, "h5", f"result_{x}.h5") for x in [1, 2]]
-    json_files = [Path(output_dir, "json", f"result_{x}.json") for x in [1, 2]]
-    data_fp = Path(test_dir, "data", "embase_labelled.csv")
-    picture_fp = Path(output_dir, "test.png")
-
-    if (plot_type, numbers) == COMBINATIONS[0]:
-        plot_setup(data_fp, state_dirs + [output_dir], h5_files + json_files)
-
-    data_combis = [
-        [h5_dir, *h5_files],
-        [json_dir, *json_files],
-        [h5_dir, json_dir],
-    ]
-    for combi in data_combis:
-        try:
-            os.remove(picture_fp)
-        except FileNotFoundError:
-            pass
-        args = [*[str(x) for x in combi], "-o", str(picture_fp)]
-        if numbers:
-            args += ["--show-absolute-values"]
-        entry = PlotEntryPoint()
-        entry.execute(args)
-        assert os.path.isfile(picture_fp)
-
-    if (plot_type, numbers) == COMBINATIONS[-1]:
-        plot_clean(state_dirs + [output_dir],
-                   h5_files + json_files + [picture_fp])
+import matplotlib.pyplot as plt
+
+from asreview import open_state
+
+from asreviewcontrib.insights.plot import plot_recall
+from asreviewcontrib.insights.plot import plot_erf
+from asreviewcontrib.insights.plot import plot_wss
+from asreviewcontrib.insights.plot import _plot_recall
+from asreviewcontrib.insights.plot import _plot_erf
+from asreviewcontrib.insights.plot import _plot_wss
+
+TEST_ASREVIEW_FILES = Path("tests", "asreview_files")
+TEST_FIGURES = Path("figures")
+
+SMALL_DATA = [1, 0, 1, 1, 0]
+
+
+def setup():
+
+    TEST_FIGURES.mkdir(exist_ok=True)
+
+
+def test_plot_erf_small_data():
+
+    fig, ax = plt.subplots()
+    _plot_erf(ax, SMALL_DATA)
+
+    fig.savefig(Path(TEST_FIGURES, "tests_small_dataset_erf.png"))
+
+
+def test_plot_wss_small_data():
+
+    fig, ax = plt.subplots()
+    _plot_wss(ax, SMALL_DATA)
+
+    fig.savefig(Path(TEST_FIGURES, "tests_small_dataset_wss.png"))
+
+
+def test_plot_recall_small_data():
+
+    fig, ax = plt.subplots()
+    _plot_recall(ax, SMALL_DATA)
+
+    fig.savefig(Path(TEST_FIGURES, "tests_small_dataset_recall.png"))
+
+
+def test_plot_recall():
+
+    with open_state(
+            Path(TEST_ASREVIEW_FILES,
+                 "sim_van_de_schoot_2017_1.asreview")) as s:
+
+        fig, ax = plt.subplots()
+        plot_recall(ax, s)
+
+        fig.savefig(
+            Path(TEST_FIGURES, "tests_recall_sim_van_de_schoot_2017_1.png"))
+
+
+def test_plot_wss():
+
+    with open_state(
+            Path(TEST_ASREVIEW_FILES,
+                 "sim_van_de_schoot_2017_1.asreview")) as s:
+        fig, ax = plt.subplots()
+        plot_wss(ax, s)
+        fig.savefig(
+            Path(TEST_FIGURES,
+                 "tests_wss_default_sim_van_de_schoot_2017_1.png"))
+
+        fig, ax = plt.subplots()
+        plot_wss(ax, s)
+        fig.savefig(
+            Path(TEST_FIGURES,
+                 "tests_wss_default_sim_van_de_schoot_2017_1.png"))
+
+        fig, ax = plt.subplots()
+        plot_wss(ax, s, x_relative=False)
+        fig.savefig(
+            Path(TEST_FIGURES, "tests_wss_xabs_sim_van_de_schoot_2017_1.png"))
+
+        fig, ax = plt.subplots()
+        plot_wss(ax, s, y_relative=False)
+        fig.savefig(
+            Path(TEST_FIGURES, "tests_wss_yabs_sim_van_de_schoot_2017_1.png"))
+
+        fig, ax = plt.subplots()
+        plot_wss(ax, s, x_relative=False, y_relative=False)
+        fig.savefig(
+            Path(TEST_FIGURES, "tests_wss_xyabs_sim_van_de_schoot_2017_1.png"))
+
+
+def test_plot_erf():
+
+    with open_state(
+            Path(TEST_ASREVIEW_FILES,
+                 "sim_van_de_schoot_2017_1.asreview")) as s:
+        fig, ax = plt.subplots()
+        plot_erf(ax, s)
+        fig.savefig(
+            Path(TEST_FIGURES,
+                 "tests_erf_default_sim_van_de_schoot_2017_1.png"))
+
+        fig, ax = plt.subplots()
+        plot_erf(ax, s)
+        fig.savefig(
+            Path(TEST_FIGURES,
+                 "tests_erf_default_sim_van_de_schoot_2017_1.png"))
+
+        fig, ax = plt.subplots()
+        plot_erf(ax, s, x_relative=False)
+        fig.savefig(
+            Path(TEST_FIGURES, "tests_erf_xabs_sim_van_de_schoot_2017_1.png"))
+
+        fig, ax = plt.subplots()
+        plot_erf(ax, s, y_relative=False)
+        fig.savefig(
+            Path(TEST_FIGURES, "tests_erf_yabs_sim_van_de_schoot_2017_1.png"))
+
+        fig, ax = plt.subplots()
+        plot_erf(ax, s, x_relative=False, y_relative=False)
+        fig.savefig(
+            Path(TEST_FIGURES, "tests_erf_xyabs_sim_van_de_schoot_2017_1.png"))