Repository: kedro-org/kedro

Issue:
Additional options for APIDataSet (e.g. proxies)
## Description
Add proxies parameter to APIDataSet. Perhaps other options could also interesting?

## Context
When comparing the list of options available from [requests](https://requests.readthedocs.io/en/master/api/) with those available in the [APIDataSet](https://kedro.readthedocs.io/en/stable/kedro.extras.datasets.api.APIDataSet.html), there are few missing that might be interesting to add.
Personally, the most interesting to have would be the proxies option as discussed [here](https://discourse.kedro.community/t/an-elegant-way-of-setting-up-http-https-proxies/197). Before I used to set my proxies in my global env variables but I stumble upon cases where my pipeline has two APIs requiring two different proxies. Having the proxies options in the data catalog would have the added benefit that the proxies could be loaded using the TemplatedConfigLoader, thereby keeping the proxies confidential in the credentials.yml.

Other than proxies, I was wondering if there are other options that users might find to be of interest? Should we consider adding all options available in requests that are not yet available in the APIDataSet so that users have full flexibility?

## Possible Implementation
I would like to try solving this one if thats ok. Hopefully it requires just adding these options in extras/datasets/api/APIDataSet.py. I am not certain if these options would require extra tests. Any inputs?


Discussion:
Thanks for opening the issue, and for your enthusiasm in trying to solve it! This sounds like an eminently reasonable idea to me, but would definitely appreciate the input from people who know `APIDataSet` better or who were involved with [its initial addition to kedro](https://github.com/quantumblacklabs/kedro/pull/284) - maybe @limdauto?

Just to be clear, here's where we currently stand. Below are all the options available in `requests`:

Available in kedro:
```- method
- url
- params
- data
- json (added in #424 after original implementation)
- headers
- auth
- timeout
```
Not available in kedro:
```
- cookies
- files
- allow_redirects
- proxies
- verify
- stream
- cert
```

Personally I don't know which of these options are or aren't important, but I don't see why we shouldn't support all of them. However, I do wonder whether our current implementation would be the correct way to do this (having many different options in the `__init__` method). Should we instead treat these as `load_args`? This way we could support all the options without needing to explicitly list and document them.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

Hi @lucasjamar, is this still a feature you're interested in? If so would you be able to make a contribution PR? 
Hi,

Definitely useful. As @AntonyMilneQB mentionned, should we just wrap up the other options inside load_args or add them like the options?
> Definitely useful. As @AntonyMilneQB mentionned, should we just wrap up the other options inside load_args or add them like the options?

I'd be in favour of wrapping them in `load_args` instead of adding each option in the `__init__`

Hi, I can pick this one up. The API specification of the `requests` library is the following: 
```python
def request(method, url, **kwargs): ...
```
Hence, my proposal is to keep the interface (almost) same in the data catalog: `method`, `url`, `load_args`.

Since authentication while using the `requests` library requires `BaseAuth` instances and this is not possible from a catalog, I'd like to add the parameter `auth`.
This auth parameter allows to configure complex authentication instances. 

```python
if auth is not None:
    if 'type' not in auth:
        raise DataSetError("'type' is missing from Authenticator configuration")
    auth_type = auth.pop('type')
    self._auth = BaseAuthFactory.create(auth_type, **auth, **credentials)
```

There is a PR open #1445  for this implementation, but since my implementation is quite deviant from the current implementation, I'd like to open a new one. 

Also fyi @afuetterer as you are working on the subclass of the general API class.

@daBlesr Sounds fine at a high level. Not entirely sure how you're planning to handle auth? If it's not an object, the default handling seems to be user/password as in https://github.com/psf/requests/blob/2a6f290bc09324406708a4d404a88a45d848ddf9/requests/models.py#L589-L609, and the current `APIDataSet` implementation also looks to be geared towards creating an auth object from tuple, if possible.
Hi @deepyaman, I think since most commonly credentials are provided in dictionary type, it would be better to stay consistent with that across the project. Hence, in my approach I would not support tuples. 

For instance, if you want to support access tokens, you can currently write something like this:

```yaml
url: example.com
method: GET
auth_type: tests.extras.datasets.api.test_api_auth.AccessTokenAuth 
credentials: access_token_cred
load_args:
    headers: 
        some_header: some_value

-- credentials.yaml
access_token_cred:
    token: abc
```
Custom Access Token class:
```python
class AccessTokenAuth(AuthBase):
    """Attaches Access Token Authentication to the given Request object."""

    def __init__(self, token):
        self.token = token

    def __call__(self, r):
        r.headers['Authorization'] = f'access_token {self.token}'
        return r
```
In my tests I have verified that indeed the following is the case:
```python
requests_mocker.last_request.headers['Authorization'] == 'access_token abc' 
```
The default for `auth_type` is `requests.auth.HTTPBasicAuth`, so username and password are generally expected, if credentials are actually provided. Let me know what you think.
Thank you for continuing this PR, @daBlesr. I am looking forward for your update on the "base" `APIDataSet`. I will gladly adapt my subclass to your approach here.

A question: The `APIDataSet` `s main use case is to make a single request in a pipeline run, right? It would not make sense to instantiate a `requests.Session` in this class, right? It would be more consistent with the subclass, that is used for multiple API calls and therefore needs a Session object. If you adapt the "base" class here, I thought I might ask.

Resolving patch:
diff --git a/kedro/extras/datasets/api/api_dataset.py b/kedro/extras/datasets/api/api_dataset.py
--- a/kedro/extras/datasets/api/api_dataset.py
+++ b/kedro/extras/datasets/api/api_dataset.py
@@ -1,9 +1,10 @@
 """``APIDataSet`` loads the data from HTTP(S) APIs.
 It uses the python requests library: https://requests.readthedocs.io/en/latest/
 """
-from typing import Any, Dict, Iterable, List, NoReturn, Union
+from typing import Any, Dict, List, NoReturn, Tuple, Union
 
 import requests
+from requests import Session, sessions
 from requests.auth import AuthBase
 
 from kedro.io.core import AbstractDataSet, DataSetError
@@ -21,83 +22,84 @@ class APIDataSet(AbstractDataSet[None, requests.Response]):
         >>>
         >>> data_set = APIDataSet(
         >>>     url="https://quickstats.nass.usda.gov",
-        >>>     params={
-        >>>         "key": "SOME_TOKEN",
-        >>>         "format": "JSON",
-        >>>         "commodity_desc": "CORN",
-        >>>         "statisticcat_des": "YIELD",
-        >>>         "agg_level_desc": "STATE",
-        >>>         "year": 2000
-        >>>     }
+        >>>     load_args={
+        >>>         "params": {
+        >>>             "key": "SOME_TOKEN",
+        >>>             "format": "JSON",
+        >>>             "commodity_desc": "CORN",
+        >>>             "statisticcat_des": "YIELD",
+        >>>             "agg_level_desc": "STATE",
+        >>>             "year": 2000
+        >>>         }
+        >>>     },
+        >>>     credentials=("username", "password")
         >>> )
         >>> data = data_set.load()
     """
 
-    # pylint: disable=too-many-arguments
     def __init__(
         self,
         url: str,
         method: str = "GET",
-        data: Any = None,
-        params: Dict[str, Any] = None,
-        headers: Dict[str, Any] = None,
-        auth: Union[Iterable[str], AuthBase] = None,
-        json: Union[List, Dict[str, Any]] = None,
-        timeout: int = 60,
-        credentials: Union[Iterable[str], AuthBase] = None,
+        load_args: Dict[str, Any] = None,
+        credentials: Union[Tuple[str, str], List[str], AuthBase] = None,
     ) -> None:
         """Creates a new instance of ``APIDataSet`` to fetch data from an API endpoint.
 
         Args:
             url: The API URL endpoint.
             method: The Method of the request, GET, POST, PUT, DELETE, HEAD, etc...
-            data: The request payload, used for POST, PUT, etc requests
-                https://requests.readthedocs.io/en/latest/user/quickstart/#more-complicated-post-requests
-            params: The url parameters of the API.
-                https://requests.readthedocs.io/en/latest/user/quickstart/#passing-parameters-in-urls
-            headers: The HTTP headers.
-                https://requests.readthedocs.io/en/latest/user/quickstart/#custom-headers
-            auth: Anything ``requests`` accepts. Normally it's either ``('login', 'password')``,
-                or ``AuthBase``, ``HTTPBasicAuth`` instance for more complex cases. Any
-                iterable will be cast to a tuple.
-            json: The request payload, used for POST, PUT, etc requests, passed in
-                to the json kwarg in the requests object.
-                https://requests.readthedocs.io/en/latest/user/quickstart/#more-complicated-post-requests
-            timeout: The wait time in seconds for a response, defaults to 1 minute.
-                https://requests.readthedocs.io/en/latest/user/quickstart/#timeouts
-            credentials: same as ``auth``. Allows specifying ``auth`` secrets in
-                credentials.yml.
-
+            load_args: Additional parameters to be fed to requests.request.
+                https://requests.readthedocs.io/en/latest/api/#requests.request
+            credentials: Allows specifying secrets in credentials.yml.
+                Expected format is ``('login', 'password')`` if given as a tuple or list.
+                An ``AuthBase`` instance can be provided for more complex cases.
         Raises:
-            ValueError: if both ``credentials`` and ``auth`` are specified.
+            ValueError: if both ``auth`` in ``load_args`` and ``credentials`` are specified.
         """
         super().__init__()
 
-        if credentials is not None and auth is not None:
+        self._load_args = load_args or {}
+        self._load_args_auth = self._load_args.pop("auth", None)
+
+        if credentials is not None and self._load_args_auth is not None:
             raise ValueError("Cannot specify both auth and credentials.")
 
-        auth = credentials or auth
+        self._auth = credentials or self._load_args_auth
 
-        if isinstance(auth, Iterable):
-            auth = tuple(auth)
+        if "cert" in self._load_args:
+            self._load_args["cert"] = self._convert_type(self._load_args["cert"])
+
+        if "timeout" in self._load_args:
+            self._load_args["timeout"] = self._convert_type(self._load_args["timeout"])
 
         self._request_args: Dict[str, Any] = {
             "url": url,
             "method": method,
-            "data": data,
-            "params": params,
-            "headers": headers,
-            "auth": auth,
-            "json": json,
-            "timeout": timeout,
+            "auth": self._convert_type(self._auth),
+            **self._load_args,
         }
 
+    @staticmethod
+    def _convert_type(value: Any):
+        """
+        From the Data Catalog, iterables are provided as Lists.
+        However, for some parameters in the Python requests library,
+        only Tuples are allowed.
+        """
+        if isinstance(value, List):
+            return tuple(value)
+        return value
+
     def _describe(self) -> Dict[str, Any]:
-        return dict(**self._request_args)
+        # prevent auth from logging
+        request_args_cp = self._request_args.copy()
+        request_args_cp.pop("auth", None)
+        return request_args_cp
 
-    def _execute_request(self) -> requests.Response:
+    def _execute_request(self, session: Session) -> requests.Response:
         try:
-            response = requests.request(**self._request_args)
+            response = session.request(**self._request_args)
             response.raise_for_status()
         except requests.exceptions.HTTPError as exc:
             raise DataSetError("Failed to fetch data", exc) from exc
@@ -107,12 +109,13 @@ def _execute_request(self) -> requests.Response:
         return response
 
     def _load(self) -> requests.Response:
-        return self._execute_request()
+        with sessions.Session() as session:
+            return self._execute_request(session)
 
     def _save(self, data: None) -> NoReturn:
         raise DataSetError(f"{self.__class__.__name__} is a read only data set type")
 
     def _exists(self) -> bool:
-        response = self._execute_request()
-
+        with sessions.Session() as session:
+            response = self._execute_request(session)
         return response.ok


Test patch:
diff --git a/tests/extras/datasets/api/test_api_dataset.py b/tests/extras/datasets/api/test_api_dataset.py
--- a/tests/extras/datasets/api/test_api_dataset.py
+++ b/tests/extras/datasets/api/test_api_dataset.py
@@ -1,10 +1,10 @@
 # pylint: disable=no-member
-import json
+import base64
 import socket
 
 import pytest
 import requests
-import requests_mock
+from requests.auth import HTTPBasicAuth
 
 from kedro.extras.datasets.api import APIDataSet
 from kedro.io.core import DataSetError
@@ -13,96 +13,191 @@
 
 TEST_URL = "http://example.com/api/test"
 TEST_TEXT_RESPONSE_DATA = "This is a response."
-TEST_JSON_RESPONSE_DATA = [{"key": "value"}]
+TEST_JSON_REQUEST_DATA = [{"key": "value"}]
 
 TEST_PARAMS = {"param": "value"}
 TEST_URL_WITH_PARAMS = TEST_URL + "?param=value"
-
+TEST_METHOD = "GET"
 TEST_HEADERS = {"key": "value"}
 
 
-@pytest.mark.parametrize("method", POSSIBLE_METHODS)
 class TestAPIDataSet:
-    @pytest.fixture
-    def requests_mocker(self):
-        with requests_mock.Mocker() as mock:
-            yield mock
+    @pytest.mark.parametrize("method", POSSIBLE_METHODS)
+    def test_request_method(self, requests_mock, method):
+        api_data_set = APIDataSet(url=TEST_URL, method=method)
+        requests_mock.register_uri(method, TEST_URL, text=TEST_TEXT_RESPONSE_DATA)
+
+        response = api_data_set.load()
+        assert response.text == TEST_TEXT_RESPONSE_DATA
 
-    def test_successfully_load_with_response(self, requests_mocker, method):
+    @pytest.mark.parametrize(
+        "parameters_in, url_postfix",
+        [
+            ({"param": "value"}, "?param=value"),
+            (bytes("a=1", "latin-1"), "?a=1"),
+        ],
+    )
+    def test_params_in_request(self, requests_mock, parameters_in, url_postfix):
         api_data_set = APIDataSet(
-            url=TEST_URL, method=method, params=TEST_PARAMS, headers=TEST_HEADERS
+            url=TEST_URL, method=TEST_METHOD, load_args={"params": parameters_in}
         )
-        requests_mocker.register_uri(
-            method,
-            TEST_URL_WITH_PARAMS,
-            headers=TEST_HEADERS,
-            text=TEST_TEXT_RESPONSE_DATA,
+        requests_mock.register_uri(
+            TEST_METHOD, TEST_URL + url_postfix, text=TEST_TEXT_RESPONSE_DATA
         )
 
         response = api_data_set.load()
         assert isinstance(response, requests.Response)
         assert response.text == TEST_TEXT_RESPONSE_DATA
 
-    def test_successful_json_load_with_response(self, requests_mocker, method):
+    def test_json_in_request(self, requests_mock):
         api_data_set = APIDataSet(
             url=TEST_URL,
-            method=method,
-            json=TEST_JSON_RESPONSE_DATA,
-            headers=TEST_HEADERS,
+            method=TEST_METHOD,
+            load_args={"json": TEST_JSON_REQUEST_DATA},
         )
-        requests_mocker.register_uri(
-            method,
+        requests_mock.register_uri(TEST_METHOD, TEST_URL)
+
+        response = api_data_set.load()
+        assert response.request.json() == TEST_JSON_REQUEST_DATA
+
+    def test_headers_in_request(self, requests_mock):
+        api_data_set = APIDataSet(
+            url=TEST_URL, method=TEST_METHOD, load_args={"headers": TEST_HEADERS}
+        )
+        requests_mock.register_uri(TEST_METHOD, TEST_URL, headers={"pan": "cake"})
+
+        response = api_data_set.load()
+
+        assert response.request.headers["key"] == "value"
+        assert response.headers["pan"] == "cake"
+
+    def test_api_cookies(self, requests_mock):
+        api_data_set = APIDataSet(
+            url=TEST_URL, method=TEST_METHOD, load_args={"cookies": {"pan": "cake"}}
+        )
+        requests_mock.register_uri(TEST_METHOD, TEST_URL, text="text")
+
+        response = api_data_set.load()
+        assert response.request.headers["Cookie"] == "pan=cake"
+
+    def test_credentials_auth_error(self):
+        """
+        If ``auth`` in ``load_args`` and ``credentials`` are both provided,
+        the constructor should raise a ValueError.
+        """
+        with pytest.raises(ValueError, match="both auth and credentials"):
+            APIDataSet(
+                url=TEST_URL, method=TEST_METHOD, load_args={"auth": []}, credentials={}
+            )
+
+    @staticmethod
+    def _basic_auth(username, password):
+        encoded = base64.b64encode(f"{username}:{password}".encode("latin-1"))
+        return f"Basic {encoded.decode('latin-1')}"
+
+    @pytest.mark.parametrize(
+        "auth_kwarg",
+        [
+            {"load_args": {"auth": ("john", "doe")}},
+            {"load_args": {"auth": ["john", "doe"]}},
+            {"load_args": {"auth": HTTPBasicAuth("john", "doe")}},
+            {"credentials": ("john", "doe")},
+            {"credentials": ["john", "doe"]},
+            {"credentials": HTTPBasicAuth("john", "doe")},
+        ],
+    )
+    def test_auth_sequence(self, requests_mock, auth_kwarg):
+        api_data_set = APIDataSet(url=TEST_URL, method=TEST_METHOD, **auth_kwarg)
+        requests_mock.register_uri(
+            TEST_METHOD,
             TEST_URL,
-            headers=TEST_HEADERS,
-            text=json.dumps(TEST_JSON_RESPONSE_DATA),
+            text=TEST_TEXT_RESPONSE_DATA,
         )
 
         response = api_data_set.load()
         assert isinstance(response, requests.Response)
-        assert response.json() == TEST_JSON_RESPONSE_DATA
+        assert response.request.headers["Authorization"] == TestAPIDataSet._basic_auth(
+            "john", "doe"
+        )
+        assert response.text == TEST_TEXT_RESPONSE_DATA
 
-    def test_http_error(self, requests_mocker, method):
+    @pytest.mark.parametrize(
+        "timeout_in, timeout_out",
+        [
+            (1, 1),
+            ((1, 2), (1, 2)),
+            ([1, 2], (1, 2)),
+        ],
+    )
+    def test_api_timeout(self, requests_mock, timeout_in, timeout_out):
         api_data_set = APIDataSet(
-            url=TEST_URL, method=method, params=TEST_PARAMS, headers=TEST_HEADERS
+            url=TEST_URL, method=TEST_METHOD, load_args={"timeout": timeout_in}
         )
-        requests_mocker.register_uri(
-            method,
-            TEST_URL_WITH_PARAMS,
-            headers=TEST_HEADERS,
-            text="Nope, not found",
-            status_code=requests.codes.FORBIDDEN,
+        requests_mock.register_uri(TEST_METHOD, TEST_URL)
+        response = api_data_set.load()
+        assert response.request.timeout == timeout_out
+
+    def test_stream(self, requests_mock):
+        text = "I am being streamed."
+
+        api_data_set = APIDataSet(
+            url=TEST_URL, method=TEST_METHOD, load_args={"stream": True}
         )
 
-        with pytest.raises(DataSetError, match="Failed to fetch data"):
-            api_data_set.load()
+        requests_mock.register_uri(TEST_METHOD, TEST_URL, text=text)
+
+        response = api_data_set.load()
+        assert isinstance(response, requests.Response)
+        assert response.request.stream
 
-    def test_socket_error(self, requests_mocker, method):
+        chunks = list(response.iter_content(chunk_size=2, decode_unicode=True))
+        assert chunks == ["I ", "am", " b", "ei", "ng", " s", "tr", "ea", "me", "d."]
+
+    def test_proxy(self, requests_mock):
         api_data_set = APIDataSet(
-            url=TEST_URL, method=method, params=TEST_PARAMS, headers=TEST_HEADERS
+            url="ftp://example.com/api/test",
+            method=TEST_METHOD,
+            load_args={"proxies": {"ftp": "ftp://127.0.0.1:3000"}},
+        )
+        requests_mock.register_uri(
+            TEST_METHOD,
+            "ftp://example.com/api/test",
         )
-        requests_mocker.register_uri(method, TEST_URL_WITH_PARAMS, exc=socket.error)
 
-        with pytest.raises(DataSetError, match="Failed to connect"):
-            api_data_set.load()
+        response = api_data_set.load()
+        assert response.request.proxies.get("ftp") == "ftp://127.0.0.1:3000"
 
-    def test_read_only_mode(self, method):
-        """
-        Saving is disabled on the data set.
-        """
-        api_data_set = APIDataSet(url=TEST_URL, method=method)
-        with pytest.raises(DataSetError, match="is a read only data set type"):
-            api_data_set.save({})
+    @pytest.mark.parametrize(
+        "cert_in, cert_out",
+        [
+            (("cert.pem", "privkey.pem"), ("cert.pem", "privkey.pem")),
+            (["cert.pem", "privkey.pem"], ("cert.pem", "privkey.pem")),
+            ("some/path/to/file.pem", "some/path/to/file.pem"),
+            (None, None),
+        ],
+    )
+    def test_certs(self, requests_mock, cert_in, cert_out):
 
-    def test_exists_http_error(self, requests_mocker, method):
+        api_data_set = APIDataSet(
+            url=TEST_URL, method=TEST_METHOD, load_args={"cert": cert_in}
+        )
+        requests_mock.register_uri(TEST_METHOD, TEST_URL)
+
+        response = api_data_set.load()
+        assert response.request.cert == cert_out
+
+    def test_exists_http_error(self, requests_mock):
         """
         In case of an unexpected HTTP error,
         ``exists()`` should not silently catch it.
         """
         api_data_set = APIDataSet(
-            url=TEST_URL, method=method, params=TEST_PARAMS, headers=TEST_HEADERS
+            url=TEST_URL,
+            method=TEST_METHOD,
+            load_args={"params": TEST_PARAMS, "headers": TEST_HEADERS},
         )
-        requests_mocker.register_uri(
-            method,
+        requests_mock.register_uri(
+            TEST_METHOD,
             TEST_URL_WITH_PARAMS,
             headers=TEST_HEADERS,
             text="Nope, not found",
@@ -111,16 +206,18 @@ def test_exists_http_error(self, requests_mocker, method):
         with pytest.raises(DataSetError, match="Failed to fetch data"):
             api_data_set.exists()
 
-    def test_exists_ok(self, requests_mocker, method):
+    def test_exists_ok(self, requests_mock):
         """
         If the file actually exists and server responds 200,
         ``exists()`` should return True
         """
         api_data_set = APIDataSet(
-            url=TEST_URL, method=method, params=TEST_PARAMS, headers=TEST_HEADERS
+            url=TEST_URL,
+            method=TEST_METHOD,
+            load_args={"params": TEST_PARAMS, "headers": TEST_HEADERS},
         )
-        requests_mocker.register_uri(
-            method,
+        requests_mock.register_uri(
+            TEST_METHOD,
             TEST_URL_WITH_PARAMS,
             headers=TEST_HEADERS,
             text=TEST_TEXT_RESPONSE_DATA,
@@ -128,43 +225,38 @@ def test_exists_ok(self, requests_mocker, method):
 
         assert api_data_set.exists()
 
-    def test_credentials_auth_error(self, method):
-        """
-        If ``auth`` and ``credentials`` are both provided,
-        the constructor should raise a ValueError.
-        """
-        with pytest.raises(ValueError, match="both auth and credentials"):
-            APIDataSet(url=TEST_URL, method=method, auth=[], credentials=[])
-
-    @pytest.mark.parametrize("auth_kwarg", ["auth", "credentials"])
-    @pytest.mark.parametrize(
-        "auth_seq",
-        [
-            ("username", "password"),
-            ["username", "password"],
-            (e for e in ["username", "password"]),  # Generator.
-        ],
-    )
-    def test_auth_sequence(self, requests_mocker, method, auth_seq, auth_kwarg):
-        """
-        ``auth`` and ``credentials`` should be able to be any Iterable.
-        """
-        kwargs = {
-            "url": TEST_URL,
-            "method": method,
-            "params": TEST_PARAMS,
-            "headers": TEST_HEADERS,
-            auth_kwarg: auth_seq,
-        }
-
-        api_data_set = APIDataSet(**kwargs)
-        requests_mocker.register_uri(
-            method,
+    def test_http_error(self, requests_mock):
+        api_data_set = APIDataSet(
+            url=TEST_URL,
+            method=TEST_METHOD,
+            load_args={"params": TEST_PARAMS, "headers": TEST_HEADERS},
+        )
+        requests_mock.register_uri(
+            TEST_METHOD,
             TEST_URL_WITH_PARAMS,
             headers=TEST_HEADERS,
-            text=TEST_TEXT_RESPONSE_DATA,
+            text="Nope, not found",
+            status_code=requests.codes.FORBIDDEN,
         )
 
-        response = api_data_set.load()
-        assert isinstance(response, requests.Response)
-        assert response.text == TEST_TEXT_RESPONSE_DATA
+        with pytest.raises(DataSetError, match="Failed to fetch data"):
+            api_data_set.load()
+
+    def test_socket_error(self, requests_mock):
+        api_data_set = APIDataSet(
+            url=TEST_URL,
+            method=TEST_METHOD,
+            load_args={"params": TEST_PARAMS, "headers": TEST_HEADERS},
+        )
+        requests_mock.register_uri(TEST_METHOD, TEST_URL_WITH_PARAMS, exc=socket.error)
+
+        with pytest.raises(DataSetError, match="Failed to connect"):
+            api_data_set.load()
+
+    def test_read_only_mode(self):
+        """
+        Saving is disabled on the data set.
+        """
+        api_data_set = APIDataSet(url=TEST_URL, method=TEST_METHOD)
+        with pytest.raises(DataSetError, match="is a read only data set type"):
+            api_data_set.save({})