Repository: opensearch-project/opensearch-py
Pull request #379 (2023-05-01 03:58:03)

Update sphinx requirement from <6.3 to <7.1
Updates the requirements on [sphinx](https://github.com/sphinx-doc/sphinx) to permit the latest version.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/sphinx-doc/sphinx/releases">sphinx's releases</a>.</em></p>
<blockquote>
<h2>v7.0.0</h2>
<p>Changelog: <a href="https://www.sphinx-doc.org/en/master/changes.html">https://www.sphinx-doc.org/en/master/changes.html</a></p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/sphinx-doc/sphinx/blob/master/CHANGES">sphinx's changelog</a>.</em></p>
<blockquote>
<h1>Release 7.0.0 (released Apr 29, 2023)</h1>
<h2>Incompatible changes</h2>
<ul>
<li><a href="https://redirect.github.com/sphinx-doc/sphinx/issues/11359">#11359</a>: Remove long-deprecated aliases for <code>MecabSplitter</code> and
<code>DefaultSplitter</code> in <code>sphinx.search.ja</code>.</li>
<li><a href="https://redirect.github.com/sphinx-doc/sphinx/issues/11360">#11360</a>: Remove deprecated <code>make_old_id</code> functions in domain object
description classes.</li>
<li><a href="https://redirect.github.com/sphinx-doc/sphinx/issues/11363">#11363</a>: Remove the Setuptools integration (<code>build_sphinx</code> hook in
<code>setup.py</code>).</li>
<li><a href="https://redirect.github.com/sphinx-doc/sphinx/issues/11364">#11364</a>: Remove deprecated <code>sphinx.ext.napoleon.iterators</code> module.</li>
<li><a href="https://redirect.github.com/sphinx-doc/sphinx/issues/11365">#11365</a>: Remove support for the <code>jsdump</code> format in <code>sphinx.search</code>.</li>
<li><a href="https://redirect.github.com/sphinx-doc/sphinx/issues/11366">#11366</a>: Make <code>locale</code> a required argument to
<code>sphinx.util.i18n.format_date()</code>.</li>
<li><a href="https://redirect.github.com/sphinx-doc/sphinx/issues/11370">#11370</a>: Remove deprecated <code>sphinx.util.stemmer</code> module.</li>
<li><a href="https://redirect.github.com/sphinx-doc/sphinx/issues/11371">#11371</a>: Remove deprecated <code>sphinx.pycode.ast.parse()</code> function.</li>
<li><a href="https://redirect.github.com/sphinx-doc/sphinx/issues/11372">#11372</a>: Remove deprecated <code>sphinx.io.read_doc()</code> function.</li>
<li><a href="https://redirect.github.com/sphinx-doc/sphinx/issues/11373">#11373</a>: Removed deprecated <code>sphinx.util.get_matching_files()</code> function.</li>
<li><a href="https://redirect.github.com/sphinx-doc/sphinx/issues/11378">#11378</a>: Remove deprecated <code>sphinx.util.docutils.is_html5_writer_available()</code>
function.</li>
<li><a href="https://redirect.github.com/sphinx-doc/sphinx/issues/11379">#11379</a>: Make the <code>env</code> argument to <code>Builder</code> subclasses required.</li>
<li><a href="https://redirect.github.com/sphinx-doc/sphinx/issues/11380">#11380</a>: autosummary: Always emit grouped import exceptions.</li>
<li><a href="https://redirect.github.com/sphinx-doc/sphinx/issues/11381">#11381</a>: Remove deprecated <code>style</code> key for HTML templates.</li>
<li><a href="https://redirect.github.com/sphinx-doc/sphinx/issues/11382">#11382</a>: Remove deprecated <code>sphinx.writers.latex.LaTeXTranslator.docclasses</code>
attribute.</li>
<li><a href="https://redirect.github.com/sphinx-doc/sphinx/issues/11383">#11383</a>: Remove deprecated <code>sphinx.builders.html.html5_ready</code> and
<code>sphinx.builders.html.HTMLTranslator</code> attributes.</li>
<li><a href="https://redirect.github.com/sphinx-doc/sphinx/issues/11385">#11385</a>: Remove support for HTML 4 output.</li>
</ul>
<h1>Release 6.2.1 (released Apr 25, 2023)</h1>
<h2>Bugs fixed</h2>
<ul>
<li><a href="https://redirect.github.com/sphinx-doc/sphinx/issues/11355">#11355</a>: Revert the default type of :confval:<code>nitpick_ignore</code> and
:confval:<code>nitpick_ignore_regex</code> to <code>list</code>.</li>
</ul>
<h1>Release 6.2.0 (released Apr 23, 2023)</h1>
<h2>Dependencies</h2>
<ul>
<li>Require Docutils 0.18.1 or greater.</li>
</ul>
<h2>Incompatible changes</h2>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="https://github.com/sphinx-doc/sphinx/commit/d568b2f4f7cca743fcbf70814d15602d8129b790"><code>d568b2f</code></a> Bump to 7.0.0 final</li>
<li><a href="https://github.com/sphinx-doc/sphinx/commit/ff79edf353f5cc6e02036f58e0295dc704c5e681"><code>ff79edf</code></a> Remove <code>jsdump</code> references post removal</li>
<li><a href="https://github.com/sphinx-doc/sphinx/commit/1a5133ac391931d8e0e2a271fe8ea9c505176908"><code>1a5133a</code></a> Bump to 7.0.0rc1 final</li>
<li><a href="https://github.com/sphinx-doc/sphinx/commit/5795fc7f9f7a5fe968cd11b961fc33fbd952a152"><code>5795fc7</code></a> Update <code>sphinx.deprecation</code> for Sphinx 7.0 (<a href="https://redirect.github.com/sphinx-doc/sphinx/issues/11386">#11386</a>)</li>
<li><a href="https://github.com/sphinx-doc/sphinx/commit/6202087f912a2ae32808865072f884ea44cd31cb"><code>6202087</code></a> Add a note to CHANGES for PR 11385</li>
<li><a href="https://github.com/sphinx-doc/sphinx/commit/ad473730a3763f241e9aaea1a87d1893f01b86fd"><code>ad47373</code></a> Remove HTML 4 support (<a href="https://redirect.github.com/sphinx-doc/sphinx/issues/11385">#11385</a>)</li>
<li><a href="https://github.com/sphinx-doc/sphinx/commit/3e3251d3ba4f1e5fa6d34d995505f4d3ace5acad"><code>3e3251d</code></a> Remove <code>HTMLTranslator</code> and <code>html5_ready</code> from <code>sphinx.builders.html</code> (...</li>
<li><a href="https://github.com/sphinx-doc/sphinx/commit/77fd819a09881102bdd48593cc69ec164b2495c8"><code>77fd819</code></a> Remove deprecated <code>LaTeXTranslator.docclasses</code> attribute (<a href="https://redirect.github.com/sphinx-doc/sphinx/issues/11382">#11382</a>)</li>
<li><a href="https://github.com/sphinx-doc/sphinx/commit/4be56f3b7ddf44265d2e7a8f26a33ba46be52ab9"><code>4be56f3</code></a> Remove deprecated <code>style</code> key for HTML templates (<a href="https://redirect.github.com/sphinx-doc/sphinx/issues/11381">#11381</a>)</li>
<li><a href="https://github.com/sphinx-doc/sphinx/commit/49027a9ae61d0d00016182025f68239a50ba4c09"><code>49027a9</code></a> Autosummary: Always emit grouped <code>ImportError</code> exceptions (<a href="https://redirect.github.com/sphinx-doc/sphinx/issues/11380">#11380</a>)</li>
<li>Additional commits viewable in <a href="https://github.com/sphinx-doc/sphinx/compare/v0.1.61611...v7.0.0">compare view</a></li>
</ul>
</details>
<br />


Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>


Patch:
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 7bda4f88..b7767643 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -34,45 +34,6 @@ jobs:
       - name: Build project for distribution
         run: |
           python3.7 -m build
-  test-linux:
-    strategy:
-      fail-fast: false
-      matrix:
-        python-version: ["2.7", "3.7", "3.8", "3.9", "3.10", "3.11"]
-        os: [ubuntu-latest]
-        experimental: [false]
-        include:
-          - os: "ubuntu-20.04"
-            python-version: "3.5"
-            experimental: false
-          - os: "ubuntu-20.04"
-            python-version: "3.6"
-            experimental: false
-
-    runs-on: ${{ matrix.os }}
-    name: test-${{ matrix.python-version }}
-    continue-on-error: ${{ matrix.experimental }}
-    steps:
-      - name: Checkout Repository
-        uses: actions/checkout@v3
-      - name: Set Up Python - ${{ matrix.python-version }}
-        uses: actions/setup-python@v4
-        with:
-          python-version: ${{ matrix.python-version }}
-        env:
-          PIP_NO_PYTHON_VERSION_WARNING: 1
-          PIP_DISABLE_PIP_VERSION_CHECK: 1
-      - name: Install Dependencies
-        run: |
-          python -m pip install -r dev-requirements.txt
-      - name: Run Tests
-        run: |
-          python setup.py test
-      - name: Upload coverage to Codecov
-        uses: codecov/codecov-action@v2
-        with:
-          token: ${{ secrets.CODECOV_TOKEN }}
-          files: ./junit/opensearch-py-codecov.xml
 
   twine-check:
     runs-on: ubuntu-latest
diff --git a/.github/workflows/integration.yml b/.github/workflows/integration.yml
index 96804474..8819d248 100644
--- a/.github/workflows/integration.yml
+++ b/.github/workflows/integration.yml
@@ -1,4 +1,4 @@
-name: Integration tests
+name: Integration Tests
 
 on: [push, pull_request]
 
@@ -15,5 +15,5 @@ jobs:
     steps:
       - name: Checkout
         uses: actions/checkout@v3
-      - name: Integ OpenSearch secured=${{ matrix.secured }} version=${{matrix.entry.opensearch_version}}
-        run: "./.ci/run-tests ${{ matrix.secured }} ${{ matrix.entry.opensearch_version }}"
+      - name: Integ OpenSearch secured=${{ matrix.secured }} version=${{ matrix.opensearch_version }}
+        run: "./.ci/run-tests ${{ matrix.secured }} ${{ matrix.opensearch_version }}"
diff --git a/.gitignore b/.gitignore
index 5bf425ef..f90ffbe6 100644
--- a/.gitignore
+++ b/.gitignore
@@ -53,6 +53,7 @@ coverage.xml
 .pytest_cache/
 cover/
 junit.xml
+junit/
 
 # Translations
 *.mo
@@ -148,4 +149,4 @@ test_opensearch/local.py
 
 #Vi text editor
 .*.swp
-*~
\ No newline at end of file
+*~
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 479a0ac3..8bbf825a 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -7,18 +7,22 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 - Added search.md to guides ([#356](https://github.com/opensearch-project/opensearch-py/pull/356))
 - Added index lifecycle guide ([#362](https://github.com/opensearch-project/opensearch-py/pull/362))
 - Added 'point in time' APIs to the pyi files in sync and async client ([#378](https://github.com/opensearch-project/opensearch-py/pull/378))
+- Added MacOS and Windows CI workflows ([#390](https://github.com/opensearch-project/opensearch-py/pull/390))
+- Compatibility with OpenSearch 2.1.0 - 2.6.0 ([#381](https://github.com/opensearch-project/opensearch-py/pull/381))
 ### Changed
 - Upgrading pytest-asyncio to latest version - 0.21.0 ([#339](https://github.com/opensearch-project/opensearch-py/pull/339))
+- Fixed flaky CI tests by replacing httpbin with a simple http_server ([#395](https://github.com/opensearch-project/opensearch-py/pull/395))
 ### Deprecated
 ### Removed
 ### Fixed
 - Fixed import cycle when importing async helpers ([#311](https://github.com/opensearch-project/opensearch-py/pull/311))
 - Fixed userguide for async client ([#340](https://github.com/opensearch-project/opensearch-py/pull/340))
+- Include parsed error info in TransportError in async connections (fixes #225) ([#226](https://github.com/opensearch-project/opensearch-py/pull/226)
 ### Security
 - Fixed CVE-2022-23491 reported in opensearch-dsl-py ([#295](https://github.com/opensearch-project/opensearch-py/pull/295))
 - Update ci workflows ([#318](https://github.com/opensearch-project/opensearch-py/pull/318))
 ### Dependencies
-- Bumps `sphinx` from <1.7 to <6.3
+- Bumps `sphinx` from <1.7 to <7.1
 
 ## [2.2.0]
 ### Added
@@ -69,6 +73,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 - Updates `master` to `cluster_manager` to be inclusive ([#242](https://github.com/opensearch-project/opensearch-py/pull/242))
 - Support a custom signing service name for AWS SigV4 ([#268](https://github.com/opensearch-project/opensearch-py/pull/268))
 - Updated CI tests to make them work locally ([#275](https://github.com/opensearch-project/opensearch-py/pull/275))
+- Fix bug with validation of 'timeout' parameter ([#387](Do not escape the "timeout" parameter.))
 ### Deprecated
 
 ### Removed
@@ -82,4 +87,5 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 [2.0.1]: https://github.com/opensearch-project/opensearch-py/compare/v2.0.0...v2.0.1
 [2.1.0]: https://github.com/opensearch-project/opensearch-py/compare/v2.0.1...v2.1.0
 [2.1.1]: https://github.com/opensearch-project/opensearch-py/compare/v2.1.0...v2.1.1
-[2.2.0]: https://github.com/opensearch-project/opensearch-py/compare/v2.1.1...v2.2.0
\ No newline at end of file
+[2.2.0]: https://github.com/opensearch-project/opensearch-py/compare/v2.1.1...v2.2.0
+
diff --git a/COMPATIBILITY.md b/COMPATIBILITY.md
index 59d8529b..bae1cafb 100644
--- a/COMPATIBILITY.md
+++ b/COMPATIBILITY.md
@@ -9,8 +9,9 @@ The below matrix shows the compatibility of the [`opensearch-py`](https://pypi.o
 | --- | --- | --- |
 | 1.0.0 | 1.0.0-1.2.4 | |
 | 1.1.0 | 1.3.0-1.3.7 | |
-| 2.0.0 | 1.0.0-2.4.1 | client works against Opensearch Version 1.x as long as features removed in 2.0 are not used |
-| 2.0.1 | 1.0.0-2.4.1 | client works against Opensearch Version 1.x as long as features removed in 2.0 are not used |
+| 2.0.x | 1.0.0-2.6.0 | client works against Opensearch Version 1.x as long as features removed in 2.0 are not used |
+| 2.1.x | 1.0.0-2.6.0 | client works against Opensearch Version 1.x as long as features removed in 2.0 are not used |
+| 2.2.0 | 1.0.0-2.6.0 | client works against Opensearch Version 1.x as long as features removed in 2.0 are not used |
 
 ## Upgrading
 
diff --git a/DEVELOPER_GUIDE.md b/DEVELOPER_GUIDE.md
index 1d8c1c3d..54df76dc 100644
--- a/DEVELOPER_GUIDE.md
+++ b/DEVELOPER_GUIDE.md
@@ -1,43 +1,74 @@
 - [Developer Guide](#developer-guide)
-  - [Linter and Test Suite](#linter-and-test-suite)
-  - [Integration Tests](#integration-tests)
-  - [Build the Documentation with Sphinx](#build-the-documentation-with-sphinx)
+  - [Prerequisites](#prerequisites)
+  - [Running Tests](#running-tests)
+  - [Linter](#linter)
+  - [Documentation](#documentation)
 
 # Developer Guide
 
-## Linter and Test Suite 
-Run the linter and test suite to ensure your changes do not break existing code:
+## Prerequisites
+
+Python 3.6 or newer is required.
+
+```
+$ python --version
+Python 3.11.1
+```
+
+Install [Nox](https://nox.thea.codes/en/stable/) for task management.
 
 ```
-# Install Nox for task management
 $ python -m pip install nox
+```
 
-# Auto-format and lint your changes
-$ nox -rs format
+## Running Tests
 
-# Run the test suite
+Tests require a live instance of OpenSearch running in docker.
+
+This will start a new instance and run tests against the latest version of OpenSearch.
+
+```
+./.ci/run-tests
+```
+
+If your OpenSearch docker instance is running, you can execute the test suite directly.
+
+```
 $ nox -rs test
 ```
 
-## Integration Tests
-To run the integration tests locally, run:
+To run tests against different versions of OpenSearch, use `run-tests [with/without security] [version]`:
 
 ```
 ./.ci/run-tests true 1.3.0
 ```
 
-The first argument tells whether to run server with security plugin enabled or not.
-The second argument specifies the version of OpenSearch the tests should run against, if not specified, the tests run against the latest version. 
+The first argument tells whether to run server with security plugin enabled or not. The second argument specifies the version of OpenSearch the tests should run against, if not specified, the tests run against the latest version. You can also run tests against a current SNAPSHOT.
+
+The following example runs tests against the latest SNAPSHOT build of OpenSearch without security.
+
+```
+./.ci/run-tests opensearch false SNAPSHOT
+```
 
 Note that integration tests require docker to be installed and running, and downloads quite a bit of data from over the internet and hence take few minutes to complete.
 
-## Build the Documentation with Sphinx
-This are the steps to build the documentation with [Sphinx](https://www.sphinx-doc.org/):
+## Linter
+
+Run the linter and test suite to ensure your changes do not break existing code. The following will auto-format your changes.
+
+```
+$ nox -rs format
+```
+
+## Documentation
+
+To build the documentation with [Sphinx](https://www.sphinx-doc.org/).
+
+```
+pip install -e .[docs]
+cd docs
+make html
+```
 
-1. change into the `opensearch-py` directory where `setup.py` is located
-2. install opensearch-py - we recommend [editable mode](https://pip.pypa.io/en/stable/cli/pip_install/#cmdoption-e
-    - with bash: `pip install -e .[docs]`
-    - with zsh: `pip install -e .\[docs]`
-4. change into the `docs` directory
-5. execute `make html`
-6. use your favorite web browser to open the file called `opensearch-py/docs/build/html/index.html`
+Open `opensearch-py/docs/build/html/index.html` to see results.
diff --git a/dev-requirements.txt b/dev-requirements.txt
index a2e0bd32..9fddad53 100644
--- a/dev-requirements.txt
+++ b/dev-requirements.txt
@@ -3,7 +3,7 @@ pytest
 pytest-cov
 coverage
 mock
-sphinx<6.3
+sphinx<7.1
 sphinx_rtd_theme
 jinja2
 
diff --git a/opensearchpy/_async/http_aiohttp.py b/opensearchpy/_async/http_aiohttp.py
index 6f96997e..f630a3bd 100644
--- a/opensearchpy/_async/http_aiohttp.py
+++ b/opensearchpy/_async/http_aiohttp.py
@@ -331,7 +331,11 @@ class AIOHttpConnection(AsyncConnection):
                 status_code=response.status,
                 response=raw_data,
             )
-            self._raise_error(response.status, raw_data)
+            self._raise_error(
+                response.status,
+                raw_data,
+                response.headers.get("content-type"),
+            )
 
         self.log_request_success(
             method, str(url), url_path, orig_body, response.status, raw_data, duration
diff --git a/opensearchpy/client/utils.py b/opensearchpy/client/utils.py
index e79b6e9e..a5e99b11 100644
--- a/opensearchpy/client/utils.py
+++ b/opensearchpy/client/utils.py
@@ -165,16 +165,17 @@ def query_params(*opensearch_query_params):
             elif api_key is not None:
                 headers["authorization"] = "ApiKey %s" % (_base64_auth_header(api_key),)
 
+            # don't escape ignore, request_timeout, or timeout
+            for p in ("ignore", "request_timeout", "timeout"):
+                if p in kwargs:
+                    params[p] = kwargs.pop(p)
+
             for p in opensearch_query_params + GLOBAL_PARAMS:
                 if p in kwargs:
                     v = kwargs.pop(p)
                     if v is not None:
                         params[p] = _escape(v)
 
-            # don't treat ignore, request_timeout, and opaque_id as other params to avoid escaping
-            for p in ("ignore", "request_timeout"):
-                if p in kwargs:
-                    params[p] = kwargs.pop(p)
             return func(*args, params=params, headers=headers, **kwargs)
 
         return _wrapped


Test patch:
diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
new file mode 100644
index 00000000..472b2e9d
--- /dev/null
+++ b/.github/workflows/test.yml
@@ -0,0 +1,43 @@
+name: Tests
+
+on: [push, pull_request]
+
+jobs:
+  test:
+    strategy:
+      matrix:
+        entry:
+          - { os: 'ubuntu-latest', python-version: "2.7" }
+          - { os: 'ubuntu-20.04', python-version: "3.5" }
+          - { os: 'ubuntu-20.04', python-version: "3.6" }
+          - { os: 'ubuntu-latest', python-version: "3.7" }
+          - { os: 'ubuntu-latest', python-version: "3.8" }
+          - { os: 'ubuntu-latest', python-version: "3.9" }
+          - { os: 'ubuntu-latest', python-version: "3.10" }
+          - { os: 'ubuntu-latest', python-version: "3.11" }
+          - { os: 'macos-latest', python-version: "3.11" }
+          
+    name: test (ruby=${{ matrix.entry.os }}, python=${{ matrix.entry.python-version }})
+    continue-on-error: ${{ matrix.entry.experimental || false }}
+    runs-on: ${{ matrix.entry.os }}
+    steps:
+      - name: Checkout Source Code
+        uses: actions/checkout@v3
+      - name: Set Up Python - ${{ matrix.entry.python-version }}
+        uses: actions/setup-python@v4
+        with:
+          python-version: ${{ matrix.entry.python-version }}
+        env:
+          PIP_NO_PYTHON_VERSION_WARNING: 1
+          PIP_DISABLE_PIP_VERSION_CHECK: 1
+      - name: Install Dependencies
+        run: |
+          python -m pip install -r dev-requirements.txt
+      - name: Run Tests
+        run: |
+          python setup.py test
+      - name: Upload coverage to Codecov
+        uses: codecov/codecov-action@v2
+        with:
+          token: ${{ secrets.CODECOV_TOKEN }}
+          files: ./junit/opensearch-py-codecov.xml
diff --git a/test_opensearchpy/TestHttpServer.py b/test_opensearchpy/TestHttpServer.py
new file mode 100644
index 00000000..4c35f621
--- /dev/null
+++ b/test_opensearchpy/TestHttpServer.py
@@ -0,0 +1,51 @@
+# SPDX-License-Identifier: Apache-2.0
+#
+# The OpenSearch Contributors require contributions made to
+# this file be licensed under the Apache-2.0 license or a
+# compatible open source license.
+#
+# Modifications Copyright OpenSearch Contributors. See
+# GitHub history for details.
+
+import json
+import threading
+from http.server import BaseHTTPRequestHandler, HTTPServer
+
+
+class TestHTTPRequestHandler(BaseHTTPRequestHandler):
+    def do_GET(self):
+        self.send_response(200)
+        headers = self.headers
+        self.send_header("Content-type", "application/json")
+        self.end_headers()
+
+        Headers = {}
+        for header, value in headers.items():
+            capitalized_header = "-".join([word.title() for word in header.split("-")])
+            Headers.update({capitalized_header: value})
+        if "Connection" in Headers:
+            Headers.pop("Connection")
+
+        data = {"method": "GET", "headers": Headers}
+        self.wfile.write(json.dumps(data).encode("utf-8"))
+
+
+class TestHTTPServer(HTTPServer):
+    def __init__(self, host="localhost", port=8080):
+        super().__init__((host, port), TestHTTPRequestHandler)
+        self._server_thread = None
+
+    def start(self):
+        if self._server_thread is not None:
+            return
+
+        self._server_thread = threading.Thread(target=self.serve_forever)
+        self._server_thread.start()
+
+    def stop(self):
+        if self._server_thread is None:
+            return
+        self.socket.close()
+        self.shutdown()
+        self._server_thread.join()
+        self._server_thread = None
diff --git a/test_opensearchpy/test_async/test_connection.py b/test_opensearchpy/test_async/test_connection.py
index d43b2bb5..0cda23d1 100644
--- a/test_opensearchpy/test_async/test_connection.py
+++ b/test_opensearchpy/test_async/test_connection.py
@@ -42,7 +42,8 @@ from pytest import raises
 from opensearchpy import AIOHttpConnection, AsyncOpenSearch, __versionstr__, serializer
 from opensearchpy.compat import reraise_exceptions
 from opensearchpy.connection import Connection, async_connections
-from opensearchpy.exceptions import ConnectionError
+from opensearchpy.exceptions import ConnectionError, TransportError
+from test_opensearchpy.TestHttpServer import TestHTTPServer
 
 pytestmark = pytest.mark.asyncio
 
@@ -53,7 +54,13 @@ def gzip_decompress(data):
 
 
 class TestAIOHttpConnection:
-    async def _get_mock_connection(self, connection_params={}, response_body=b"{}"):
+    async def _get_mock_connection(
+        self,
+        connection_params={},
+        response_code=200,
+        response_body=b"{}",
+        response_headers={},
+    ):
         con = AIOHttpConnection(**connection_params)
         await con._create_aiohttp_session()
 
@@ -69,8 +76,8 @@ class TestAIOHttpConnection:
                     return response_body.decode("utf-8", "surrogatepass")
 
             dummy_response = DummyResponse()
-            dummy_response.headers = CIMultiDict()
-            dummy_response.status = 200
+            dummy_response.headers = CIMultiDict(**response_headers)
+            dummy_response.status = response_code
             _dummy_request.call_args = (args, kwargs)
             return dummy_response
 
@@ -298,73 +305,95 @@ class TestAIOHttpConnection:
             await conn.perform_request("GET", "/")
         assert str(e.value) == "Wasn't modified!"
 
+    async def test_json_errors_are_parsed(self):
+        con = await self._get_mock_connection(
+            response_code=400,
+            response_body=b'{"error": {"type": "snapshot_in_progress_exception"}}',
+            response_headers={"Content-Type": "application/json;"},
+        )
+        try:
+            with pytest.raises(TransportError) as e:
+                await con.perform_request("POST", "/", body=b'{"some": "json"')
+
+            assert e.value.error == "snapshot_in_progress_exception"
+        finally:
+            await con.close()
 
-class TestConnectionHttpbin:
+
+class TestConnectionHttpServer:
     """Tests the HTTP connection implementations against a live server E2E"""
 
-    async def httpbin_anything(self, conn, **kwargs):
-        status, headers, data = await conn.perform_request("GET", "/anything", **kwargs)
+    @classmethod
+    def setup_class(cls):
+        # Start server
+        cls.server = TestHTTPServer(port=8081)
+        cls.server.start()
+
+    @classmethod
+    def teardown_class(cls):
+        # Stop server
+        cls.server.stop()
+
+    async def httpserver(self, conn, **kwargs):
+        status, headers, data = await conn.perform_request("GET", "/", **kwargs)
         data = json.loads(data)
-        data["headers"].pop(
-            "X-Amzn-Trace-Id", None
-        )  # Remove this header as it's put there by AWS.
         return (status, data)
 
     async def test_aiohttp_connection(self):
         # Defaults
-        conn = AIOHttpConnection("httpbin.org", port=443, use_ssl=True)
+        conn = AIOHttpConnection("localhost", port=8081, use_ssl=False)
         user_agent = conn._get_default_user_agent()
-        status, data = await self.httpbin_anything(conn)
+        status, data = await self.httpserver(conn)
         assert status == 200
         assert data["method"] == "GET"
         assert data["headers"] == {
             "Content-Type": "application/json",
-            "Host": "httpbin.org",
+            "Host": "localhost:8081",
             "User-Agent": user_agent,
         }
 
         # http_compress=False
         conn = AIOHttpConnection(
-            "httpbin.org", port=443, use_ssl=True, http_compress=False
+            "localhost", port=8081, use_ssl=False, http_compress=False
         )
-        status, data = await self.httpbin_anything(conn)
+        status, data = await self.httpserver(conn)
         assert status == 200
         assert data["method"] == "GET"
         assert data["headers"] == {
             "Content-Type": "application/json",
-            "Host": "httpbin.org",
+            "Host": "localhost:8081",
             "User-Agent": user_agent,
         }
 
         # http_compress=True
         conn = AIOHttpConnection(
-            "httpbin.org", port=443, use_ssl=True, http_compress=True
+            "localhost", port=8081, use_ssl=False, http_compress=True
         )
-        status, data = await self.httpbin_anything(conn)
+        status, data = await self.httpserver(conn)
         assert status == 200
         assert data["headers"] == {
             "Accept-Encoding": "gzip,deflate",
             "Content-Type": "application/json",
-            "Host": "httpbin.org",
+            "Host": "localhost:8081",
             "User-Agent": user_agent,
         }
 
         # Headers
         conn = AIOHttpConnection(
-            "httpbin.org",
-            port=443,
-            use_ssl=True,
+            "localhost",
+            port=8081,
+            use_ssl=False,
             http_compress=True,
             headers={"header1": "value1"},
         )
-        status, data = await self.httpbin_anything(
+        status, data = await self.httpserver(
             conn, headers={"header2": "value2", "header1": "override!"}
         )
         assert status == 200
         assert data["headers"] == {
             "Accept-Encoding": "gzip,deflate",
             "Content-Type": "application/json",
-            "Host": "httpbin.org",
+            "Host": "localhost:8081",
             "Header1": "override!",
             "Header2": "value2",
             "User-Agent": user_agent,
diff --git a/test_opensearchpy/test_client/test_utils.py b/test_opensearchpy/test_client/test_utils.py
index 2b5e5a39..1a4b6809 100644
--- a/test_opensearchpy/test_client/test_utils.py
+++ b/test_opensearchpy/test_client/test_utils.py
@@ -85,6 +85,31 @@ class TestQueryParams(TestCase):
         self.func_to_wrap(headers={"X": "y"})
         self.assertEqual(self.calls[-1], ((), {"params": {}, "headers": {"x": "y"}}))
 
+    def test_non_escaping_params(self):
+        # the query_params decorator doesn't validate "timeout" it simply avoids escaping as it did
+        self.func_to_wrap(simple_param="x", timeout="4s")
+        self.assertEqual(
+            self.calls[-1],
+            ((), {"params": {"simple_param": b"x", "timeout": "4s"}, "headers": {}}),
+        )
+
+        self.func_to_wrap(simple_param="x", timeout=4, ignore=5, request_timeout=6)
+        self.assertEqual(
+            self.calls[-1],
+            (
+                (),
+                {
+                    "params": {
+                        "simple_param": b"x",
+                        "timeout": 4,
+                        "ignore": 5,
+                        "request_timeout": 6,
+                    },
+                    "headers": {},
+                },
+            ),
+        )
+
     def test_per_call_authentication(self):
         self.func_to_wrap(api_key=("name", "key"))
         self.assertEqual(
diff --git a/test_opensearchpy/test_connection.py b/test_opensearchpy/test_connection.py
index 3ed1cf0b..8d501ffe 100644
--- a/test_opensearchpy/test_connection.py
+++ b/test_opensearchpy/test_connection.py
@@ -72,6 +72,9 @@ from pytest import raises
 from opensearchpy import OpenSearch, serializer
 from opensearchpy.connection import connections
 
+if sys.version_info > (3, 0):
+    from test_opensearchpy.TestHttpServer import TestHTTPServer
+
 
 def gzip_decompress(data):
     buf = gzip.GzipFile(fileobj=io.BytesIO(data), mode="rb")
@@ -857,76 +860,87 @@ class TestRequestsConnection(TestCase):
         assert str(e.value) == "Wasn't modified!"
 
 
-class TestConnectionHttpbin:
+@pytest.mark.skipif(
+    sys.version_info < (3, 0),
+    reason="http_server is only available from python 3.x",
+)
+class TestConnectionHttpServer:
     """Tests the HTTP connection implementations against a live server E2E"""
 
-    def httpbin_anything(self, conn, **kwargs):
-        status, headers, data = conn.perform_request("GET", "/anything", **kwargs)
+    @classmethod
+    def setup_class(cls):
+        # Start server
+        cls.server = TestHTTPServer(port=8080)
+        cls.server.start()
+
+    @classmethod
+    def teardown_class(cls):
+        # Stop server
+        cls.server.stop()
+
+    def httpserver(self, conn, **kwargs):
+        status, headers, data = conn.perform_request("GET", "/", **kwargs)
         data = json.loads(data)
-        data["headers"].pop(
-            "X-Amzn-Trace-Id", None
-        )  # Remove this header as it's put there by AWS.
         return (status, data)
 
     def test_urllib3_connection(self):
         # Defaults
-        # httpbin.org can be slow sometimes. Hence the timeout
-        conn = Urllib3HttpConnection("httpbin.org", port=443, use_ssl=True, timeout=60)
+        conn = Urllib3HttpConnection("localhost", port=8080, use_ssl=False, timeout=60)
         user_agent = conn._get_default_user_agent()
-        status, data = self.httpbin_anything(conn)
+        status, data = self.httpserver(conn)
         assert status == 200
         assert data["method"] == "GET"
         assert data["headers"] == {
             "Accept-Encoding": "identity",
             "Content-Type": "application/json",
-            "Host": "httpbin.org",
+            "Host": "localhost:8080",
             "User-Agent": user_agent,
         }
 
         # http_compress=False
         conn = Urllib3HttpConnection(
-            "httpbin.org", port=443, use_ssl=True, http_compress=False, timeout=60
+            "localhost", port=8080, use_ssl=False, http_compress=False, timeout=60
         )
-        status, data = self.httpbin_anything(conn)
+        status, data = self.httpserver(conn)
         assert status == 200
         assert data["method"] == "GET"
         assert data["headers"] == {
             "Accept-Encoding": "identity",
             "Content-Type": "application/json",
-            "Host": "httpbin.org",
+            "Host": "localhost:8080",
             "User-Agent": user_agent,
         }
 
         # http_compress=True
         conn = Urllib3HttpConnection(
-            "httpbin.org", port=443, use_ssl=True, http_compress=True, timeout=60
+            "localhost", port=8080, use_ssl=False, http_compress=True, timeout=60
         )
-        status, data = self.httpbin_anything(conn)
+        status, data = self.httpserver(conn)
         assert status == 200
         assert data["headers"] == {
             "Accept-Encoding": "gzip,deflate",
             "Content-Type": "application/json",
-            "Host": "httpbin.org",
+            "Host": "localhost:8080",
             "User-Agent": user_agent,
         }
 
         # Headers
         conn = Urllib3HttpConnection(
-            "httpbin.org",
-            port=443,
-            use_ssl=True,
+            "localhost",
+            port=8080,
+            use_ssl=False,
             http_compress=True,
             headers={"header1": "value1"},
             timeout=60,
         )
-        status, data = self.httpbin_anything(
+        status, data = self.httpserver(
             conn, headers={"header2": "value2", "header1": "override!"}
         )
         assert status == 200
         assert data["headers"] == {
             "Accept-Encoding": "gzip,deflate",
             "Content-Type": "application/json",
-            "Host": "httpbin.org",
+            "Host": "localhost:8080",
             "Header1": "override!",
             "Header2": "value2",
             "User-Agent": user_agent,
@@ -939,62 +953,62 @@ class TestConnectionHttpbin:
 
     def test_requests_connection(self):
         # Defaults
-        conn = RequestsHttpConnection("httpbin.org", port=443, use_ssl=True, timeout=60)
+        conn = RequestsHttpConnection("localhost", port=8080, use_ssl=False, timeout=60)
         user_agent = conn._get_default_user_agent()
-        status, data = self.httpbin_anything(conn)
+        status, data = self.httpserver(conn)
         assert status == 200
         assert data["method"] == "GET"
         assert data["headers"] == {
             "Accept-Encoding": "identity",
             "Content-Type": "application/json",
-            "Host": "httpbin.org",
+            "Host": "localhost:8080",
             "User-Agent": user_agent,
         }
 
         # http_compress=False
         conn = RequestsHttpConnection(
-            "httpbin.org", port=443, use_ssl=True, http_compress=False, timeout=60
+            "localhost", port=8080, use_ssl=False, http_compress=False, timeout=60
         )
-        status, data = self.httpbin_anything(conn)
+        status, data = self.httpserver(conn)
         assert status == 200
         assert data["method"] == "GET"
         assert data["headers"] == {
             "Accept-Encoding": "identity",
             "Content-Type": "application/json",
-            "Host": "httpbin.org",
+            "Host": "localhost:8080",
             "User-Agent": user_agent,
         }
 
         # http_compress=True
         conn = RequestsHttpConnection(
-            "httpbin.org", port=443, use_ssl=True, http_compress=True, timeout=60
+            "localhost", port=8080, use_ssl=False, http_compress=True, timeout=60
         )
-        status, data = self.httpbin_anything(conn)
+        status, data = self.httpserver(conn)
         assert status == 200
         assert data["headers"] == {
             "Accept-Encoding": "gzip,deflate",
             "Content-Type": "application/json",
-            "Host": "httpbin.org",
+            "Host": "localhost:8080",
             "User-Agent": user_agent,
         }
 
         # Headers
         conn = RequestsHttpConnection(
-            "httpbin.org",
-            port=443,
-            use_ssl=True,
+            "localhost",
+            port=8080,
+            use_ssl=False,
             http_compress=True,
             headers={"header1": "value1"},
             timeout=60,
         )
-        status, data = self.httpbin_anything(
+        status, data = self.httpserver(
             conn, headers={"header2": "value2", "header1": "override!"}
         )
         assert status == 200
         assert data["headers"] == {
             "Accept-Encoding": "gzip,deflate",
             "Content-Type": "application/json",
-            "Host": "httpbin.org",
+            "Host": "localhost:8080",
             "Header1": "override!",
             "Header2": "value2",
             "User-Agent": user_agent,