## Testing

All of our packages have unit tests and integration tests, and we favor unit tests over integration tests.

Unit tests run on every pull request, so they should be fast and reliable.

Integration tests run once a day, and they require more setup, so they should be reserved for confirming interface points with external services.

## Unit Tests[​](#unit-tests "Direct link to Unit Tests")

Unit tests cover modular logic that does not require calls to outside APIs. If you add new logic, please add a unit test.

To install dependencies for unit tests:

```
poetry install --with test
```

To run unit tests:

To run unit tests in Docker:

To run a specific test:

```
TEST_FILE=tests/unit_tests/test_imports.py make test
```

## Integration Tests[​](#integration-tests "Direct link to Integration Tests")

Integration tests cover logic that requires making calls to outside APIs (often integration with other services). If you add support for a new external API, please add a new integration test.

**Warning:** Almost no tests should be integration tests.

Tests that require making network connections make it difficult for other developers to test the code.

Instead favor relying on `responses` library and/or mock.patch to mock requests using small fixtures.

To install dependencies for integration tests:

```
poetry install --with test,test_integration
```

To run integration tests:

### Prepare[​](#prepare "Direct link to Prepare")

The integration tests use several search engines and databases. The tests aim to verify the correct behavior of the engines and databases according to their specifications and requirements.

To run some integration tests, such as tests located in `tests/integration_tests/vectorstores/`, you will need to install the following software:

*   Docker
*   Python 3.8.1 or later

Any new dependencies should be added by running:

```
# add package and install it after adding:poetry add tiktoken@latest --group "test_integration" && poetry install --with test_integration
```

Before running any tests, you should start a specific Docker container that has all the necessary dependencies installed. For instance, we use the `elasticsearch.yml` container for `test_elasticsearch.py`:

```
cd tests/integration_tests/vectorstores/docker-composedocker-compose -f elasticsearch.yml up
```

For environments that requires more involving preparation, look for `*.sh`. For instance, `opensearch.sh` builds a required docker image and then launch opensearch.

### Prepare environment variables for local testing:[​](#prepare-environment-variables-for-local-testing "Direct link to Prepare environment variables for local testing:")

*   copy `tests/integration_tests/.env.example` to `tests/integration_tests/.env`
*   set variables in `tests/integration_tests/.env` file, e.g `OPENAI_API_KEY`

Additionally, it's important to note that some integration tests may require certain environment variables to be set, such as `OPENAI_API_KEY`. Be sure to set any required environment variables before running the tests to ensure they run correctly.

### Recording HTTP interactions with pytest-vcr[​](#recording-http-interactions-with-pytest-vcr "Direct link to Recording HTTP interactions with pytest-vcr")

Some of the integration tests in this repository involve making HTTP requests to external services. To prevent these requests from being made every time the tests are run, we use pytest-vcr to record and replay HTTP interactions.

When running tests in a CI/CD pipeline, you may not want to modify the existing cassettes. You can use the --vcr-record=none command-line option to disable recording new cassettes. Here's an example:

```
pytest --log-cli-level=10 tests/integration_tests/vectorstores/test_pinecone.py --vcr-record=nonepytest tests/integration_tests/vectorstores/test_elasticsearch.py --vcr-record=none
```

### Run some tests with coverage:[​](#run-some-tests-with-coverage "Direct link to Run some tests with coverage:")

```
pytest tests/integration_tests/vectorstores/test_elasticsearch.py --cov=langchain --cov-report=htmlstart "" htmlcov/index.html || open htmlcov/index.html
```

## Coverage[​](#coverage "Direct link to Coverage")

Code coverage (i.e. the amount of code that is covered by unit tests) helps identify areas of the code that are potentially more or less brittle.

Coverage requires the dependencies for integration tests:

```
poetry install --with test_integration
```

To get a report of current coverage, run the following: