Repository: RasaHQ/rasa
Pull request: Add ability to point to a certificate in endpoints.yml
**Proposed changes**:
Fixes #9085 

Adds support for the parameter `cafile` in `endpoints.yml`.
If this points to a local certificate, this will be used when making a request to that endpoint.

**Status (please check what you already did)**:
- [x] added some tests for the functionality
- [ ] updated the documentation
- [x] updated the changelog (please check [changelog](https://github.com/RasaHQ/rasa/tree/main/changelog) for instructions)
- [x] reformat files using `black` (please check [Readme](https://github.com/RasaHQ/rasa#code-style) for instructions)


Problem statement:
Create custom SSL context in Rasa OSS
**Description of Problem**:
When an action server is deployed with a custom SSL certificate the necessary CA certificates also need to be available within Rasa OSS.

**Definition of Done**:
<!-- What needs to be there to consider this feature as done?
- [ ] Tests are added
- [ ] Feature described the docs
- [ ] Feature mentioned in the changelog
- [ ] ... -->

Hints from issue discussion:

Commit message: Add cert file functionality to Endpoint

Patch to review:
diff --git a/changelog/9085.improvement.md b/changelog/9085.improvement.md
new file mode 100644
index 000000000000..20acbfb972b9
--- /dev/null
+++ b/changelog/9085.improvement.md
@@ -0,0 +1,2 @@
+Add support for `cafile` parameter in `endpoints.yaml`.
+This will load a custom local certificate file and use it when making requests to that endpoint.
diff --git a/data/test_endpoints/cert.pem b/data/test_endpoints/cert.pem
new file mode 100644
index 000000000000..cd25e0551178
--- /dev/null
+++ b/data/test_endpoints/cert.pem
@@ -0,0 +1,32 @@
+-----BEGIN CERTIFICATE-----
+MIIFiDCCA3ACCQCgeYtdshQ3bjANBgkqhkiG9w0BAQsFADCBhTELMAkGA1UEBhMC
+ZGUxDzANBgNVBAgMBmJlcmxpbjEPMA0GA1UEBwwGYmVybGluMQ0wCwYDVQQKDARy
+YXNhMQ0wCwYDVQQLDARyYXNhMRYwFAYDVQQDDA1teWFjdGlvbnMuZGV2MR4wHAYJ
+KoZIhvcNAQkBFg9qLmp1emxAcmFzYS5jb20wHhcNMjEwNzEzMTAwNTI5WhcNMjIw
+NzEzMTAwNTI5WjCBhTELMAkGA1UEBhMCZGUxDzANBgNVBAgMBmJlcmxpbjEPMA0G
+A1UEBwwGYmVybGluMQ0wCwYDVQQKDARyYXNhMQ0wCwYDVQQLDARyYXNhMRYwFAYD
+VQQDDA1teWFjdGlvbnMuZGV2MR4wHAYJKoZIhvcNAQkBFg9qLmp1emxAcmFzYS5j
+b20wggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQDH4D74AQ0vlk7McR+a
+syG/38maXyqFM1XKybxxjUnBqooay4v8+wiByn3LK3IjVdnUBeuJLe8jAhbbDXgK
+okbjuQb8U7o3T/rW4rrFKoT3bpF+VOfULmzW51jREYB/mtd+TvhBLhMlxytxPVzU
+Rq3xT5NV6UILaTuWUFrQNP/CcwzjCPJXfPHwQaY8G5Ua31PqOBe2rNxHInIcctbg
+DSwUxtcCCroIOIC5THoiYDzMMwGryZIabtdRbp/XAvcnrDxqXZGLxNEKKM7xOLTG
+HfJfqqlLhCMn7KOie8OC0NGX36rIDUJHV+JVenDqgFdu7h4sRNFal/w0dstcZQBy
+68A5GcQuuyggLv2XvTWThc1WycDAQ/gCoOsLTnuDW2KS1v/W1Rckyvm0R45FwIv/
+BomnAEDWpUao8Jy4k8/b10xaX1kDyikz3uIeM0AtDY1cxnBowk+82yL1uC0UIcRp
+8bCxp9U9y+uk80+1+OHnb8CIOjR3jzVSdLnnMeZyld+5g2qJ8WbPvXJmzo5sh9yb
+PKMx8zwODM3YyJm6WgRkZXNRCEytIeNMFBr4gfa+iBvCn2PBERMDrG5gzOV7VSY8
+JJTCpbmAkq9DnxLLMO1PqK+l+yD3xtE5xIx4lF4vMoy29w1K046pyORcRBDAOHyN
+WA3YrlWVghZPCPH3eMaZ8H+gPQIDAQABMA0GCSqGSIb3DQEBCwUAA4ICAQBGAKk5
+0Rts2Dob9DBh0/lVTSeIa9tmjlpeBf1ZsqUVCkmAs5nhE+3ydGeNYTw1zoJ7hiHn
+27IiB6yq+zu1qUoHFLv7W0xrZ2cW2M2df0mYVG1bFH9pmSeCQt4/ekDjnztZLgNb
+fQiKpVWk7GLrAKhAhdoYE/t5uKT5Ns5vp9g5AsIIlMNPZqQAYdhX7/6zI755xuHp
+2Dp232zKpmh+6bYLYeuI6bLDQ2HLSBhxmwEf4YZCi3R1LwxjnomXT9yTyd2PDN/B
+WrV7G6zrPRcYq/QeLVKHDCNbUtZrwko5dR2Siz4ds8vLN1pX4v5y88WkIniZox+Q
+9GyYvQ/m11rB+0kp9LKfCswW/F2AIznLgnQxfgXgrw/DUiUkQ0hQ/S5trwSQPHpO
+Uf4g4yXh1MZg38kadSslqmkwMg0lI4MYdyCNjsVbH+ESqc5FkfFfR6TOS1wuevIQ
+OFbF6BfkA/b4Faq9Uf+pxA1+9bt6EI5No2SnZ0InGvmbqnPkU78SCa7qTmq2UHZ1
+fvLejsbOTHZpu72lj/mXVZAgrnXzcuj+M0uc+k/vAWp1sx7fAapsMlO+8Unn6QGq
+/BE3rSj1OnYfb6c7w+yF/8mRx2t0OWyJinkNrOAScFdDWalnprc9woattsRunhvF
+xdPTIWctCZpcmgDvh7XJMP47CGvOHCSFtgsEIg==
+-----END CERTIFICATE-----
diff --git a/data/test_endpoints/example_endpoints.yml b/data/test_endpoints/example_endpoints.yml
index 5ca07a8dcab5..049d6cd32b89 100644
--- a/data/test_endpoints/example_endpoints.yml
+++ b/data/test_endpoints/example_endpoints.yml
@@ -23,4 +23,7 @@ tracker_store:
   #db: rasa
   #user: username
   #password: password
+action_endpoint:
+  url: http://localhost:5055/webhook
+  cafile: ./some_test_file
 empty:
diff --git a/rasa/utils/endpoints.py b/rasa/utils/endpoints.py
index 9e040188a692..89a6ec882d52 100644
--- a/rasa/utils/endpoints.py
+++ b/rasa/utils/endpoints.py
@@ -1,3 +1,5 @@
+import ssl
+
 import aiohttp
 import logging
 import os
@@ -5,6 +7,7 @@
 from sanic.request import Request
 from typing import Any, Optional, Text, Dict
 
+from rasa.shared.exceptions import FileNotFoundException
 import rasa.shared.utils.io
 import rasa.utils.io
 from rasa.constants import DEFAULT_REQUEST_TIMEOUT
@@ -79,6 +82,7 @@ def __init__(
         basic_auth: Dict[Text, Text] = None,
         token: Optional[Text] = None,
         token_name: Text = "token",
+        cafile: Text = None,
         **kwargs: Any,
     ) -> None:
         self.url = url
@@ -88,9 +92,11 @@ def __init__(
         self.token = token
         self.token_name = token_name
         self.type = kwargs.pop("store_type", kwargs.pop("type", None))
+        self.cafile = cafile
         self.kwargs = kwargs
 
     def session(self) -> aiohttp.ClientSession:
+        """Create and return a configured aiohttp client session."""
         # create authentication parameters
         if self.basic_auth:
             auth = aiohttp.BasicAuth(
@@ -142,12 +148,24 @@ async def request(
             del kwargs["headers"]
 
         url = concat_url(self.url, subpath)
+
+        sslcontext = None
+        if self.cafile:
+            try:
+                sslcontext = ssl.create_default_context(cafile=self.cafile)
+            except FileNotFoundError:
+                raise FileNotFoundException(
+                    f"Failed to find certificate file, "
+                    f"'{os.path.abspath(self.cafile)}' does not exist."
+                )
+
         async with self.session() as session:
             async with session.request(
                 method,
                 url,
                 headers=headers,
                 params=self.combine_parameters(kwargs),
+                ssl=sslcontext,
                 **kwargs,
             ) as response:
                 if response.status >= 400:
diff --git a/tests/utils/test_endpoints.py b/tests/utils/test_endpoints.py
index 5532e1d75e96..19065527e723 100644
--- a/tests/utils/test_endpoints.py
+++ b/tests/utils/test_endpoints.py
@@ -5,6 +5,7 @@
 import pytest
 from aioresponses import aioresponses
 
+from rasa.shared.exceptions import FileNotFoundException
 from tests.utilities import latest_request, json_of_latest_request
 import rasa.utils.endpoints as endpoint_utils
 
@@ -92,6 +93,38 @@ async def test_endpoint_config():
             assert s._default_auth.password == "pass"
 
 
+async def test_endpoint_config_with_cafile(tmp_path):
+    cafile = "data/test_endpoints/cert.pem"
+
+    with aioresponses() as mocked:
+        endpoint = endpoint_utils.EndpointConfig(
+            "https://example.com/", cafile=str(cafile),
+        )
+
+        mocked.post(
+            "https://example.com/", status=200,
+        )
+
+        await endpoint.request("post",)
+
+        request = latest_request(mocked, "post", "https://example.com/")[-1]
+
+        ssl_context = request.kwargs["ssl"]
+        certs = ssl_context.get_ca_certs()
+        assert certs[0]["subject"][4][0] == ("organizationalUnitName", "rasa")
+
+
+async def test_endpoint_config_with_non_existent_cafile(tmp_path):
+    cafile = "data/test_endpoints/no_file.pem"
+
+    endpoint = endpoint_utils.EndpointConfig(
+        "https://example.com/", cafile=str(cafile),
+    )
+
+    with pytest.raises(FileNotFoundException):
+        await endpoint.request("post",)
+
+
 def test_endpoint_config_default_token_name():
     test_data = {"url": "http://test", "token": "token"}
 
@@ -133,6 +166,17 @@ def test_read_endpoint_config(filename: Text, endpoint_type: Text):
     assert isinstance(conf, endpoint_utils.EndpointConfig)
 
 
+@pytest.mark.parametrize(
+    "endpoint_type, cafile",
+    [("action_endpoint", "./some_test_file"), ("tracker_store", None)],
+)
+def test_read_endpoint_config_with_cafile(endpoint_type, cafile):
+    conf = endpoint_utils.read_endpoint_config(
+        "data/test_endpoints/example_endpoints.yml", endpoint_type
+    )
+    assert conf.cafile == cafile
+
+
 @pytest.mark.parametrize(
     "filename, endpoint_type",
     [


Code review comments:
diff_hunk: @@ -92,6 +93,38 @@ async def test_endpoint_config():
             assert s._default_auth.password == "pass"
 
 
+async def test_endpoint_config_with_cafile(tmp_path):
+    cafile = "data/test_endpoints/cert.pem"
+
+    with aioresponses() as mocked:
+        endpoint = endpoint_utils.EndpointConfig(
+            "https://example.com/", cafile=str(cafile),
+        )
+
+        mocked.post(
+            "https://example.com/", status=200,
+        )
+
+        await endpoint.request("post",)
+
+        request = latest_request(mocked, "post", "https://example.com/")[-1]
+
+        ssl_context = request.kwargs["ssl"]
+        certs = ssl_context.get_ca_certs()
+        assert certs[0]["subject"][4][0] == ("organizationalUnitName", "rasa")
+
+
+async def test_endpoint_config_with_non_existent_cafile(tmp_path):
path: tests/utils/test_endpoints.py
text: @user1:
```suggestion
async def test_endpoint_config_with_non_existent_cafile(tmp_path: Path):
```

diff_hunk: @@ -133,6 +166,17 @@ def test_read_endpoint_config(filename: Text, endpoint_type: Text):
     assert isinstance(conf, endpoint_utils.EndpointConfig)
 
 
+@pytest.mark.parametrize(
+    "endpoint_type, cafile",
+    [("action_endpoint", "./some_test_file"), ("tracker_store", None)],
+)
+def test_read_endpoint_config_with_cafile(endpoint_type, cafile):
path: tests/utils/test_endpoints.py
text: @user1:
```suggestion
def test_read_endpoint_config_with_cafile(endpoint_type: Text, cafile: Optional[Text]):
```

diff_hunk: @@ -142,12 +148,24 @@ async def request(
             del kwargs["headers"]
 
         url = concat_url(self.url, subpath)
+
+        sslcontext = None
+        if self.cafile:
+            try:
+                sslcontext = ssl.create_default_context(cafile=self.cafile)
+            except FileNotFoundError:
+                raise FileNotFoundException(
+                    f"Failed to find certificate file, "
+                    f"'{os.path.abspath(self.cafile)}' does not exist."
+                )
path: rasa/utils/endpoints.py
text: @user1:
```suggestion
            except FileNotFoundError as e:
                raise FileNotFoundException(
                    f"Failed to find certificate file, "
                    f"'{os.path.abspath(self.cafile)}' does not exist."
                ) from e
```

diff_hunk: @@ -79,6 +82,7 @@ def __init__(
         basic_auth: Dict[Text, Text] = None,
         token: Optional[Text] = None,
         token_name: Text = "token",
+        cafile: Text = None,
path: rasa/utils/endpoints.py
text: @user1:
```suggestion
        cafile: Optional[Text] = None,
```