semvec/pypi-packages huggingface dataset:
jaraco.xkcd 3.0.3
Library for requesting XKCD comics from its API
Requires Python: >=3.8

.. image:: https://img.shields.io/pypi/v/jaraco.xkcd.svg
   :target: https://pypi.org/project/jaraco.xkcd

.. image:: https://img.shields.io/pypi/pyversions/jaraco.xkcd.svg

.. image:: https://github.com/jaraco/jaraco.xkcd/actions/workflows/main.yml/badge.svg
   :target: https://github.com/jaraco/jaraco.xkcd/actions?query=workflow%3A%22tests%22
   :alt: tests

.. image:: https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/charliermarsh/ruff/main/assets/badge/v2.json
    :target: https://github.com/astral-sh/ruff
    :alt: Ruff

.. .. image:: https://readthedocs.org/projects/PROJECT_RTD/badge/?version=latest
..    :target: https://PROJECT_RTD.readthedocs.io/en/latest/?badge=latest

.. image:: https://img.shields.io/badge/skeleton-2024-informational
   :target: https://blog.jaraco.com/skeleton
ConsistencyBench 0.1.3
Tools and Techniques for Consistency Benchmarking
Requires Python: <4.0,>=3.9

# ConsistencyBench

## Setup

First install the package from PyPI.

```bash
pip install consistencybench
```
Additionally, if you'd like to use the NER metric (`consistencybench.metrics.AgreementNER`), run the following first.
```bash
python -m spacy download en_core_web_sm
```

## To start generation and scoring

Set the arguments from `run_eval.py`

```bash
python run_eval.py --openai_api_key <OPENAI_API_KEY>
```

**Example Output file**: `consistencybench/result_gpt-3.5-turbo_paraphrasing.csv`

**Example Jupyter Notebook**: `example.ipynb`