Repository: elastic/apm-agent-python

Issue:
IndexError when "url" not found in args or kwargs.
### Overview

I've found an issue in line 46 of `call()` method in `AioHttpClientInstrumentation(...)` class.

https://github.com/elastic/apm-agent-python/blob/da93e7af448abcac367d216e2d20a584051f6e50/elasticapm/instrumentation/packages/asyncio/aiohttp_client.py#L44-L47

I'm getting an `IndexError` exception due to lack or "url" in both kwargs and in args[1]. The the reason is that the argument containing urls is called "str_or_url".

https://github.com/aio-libs/aiohttp/blob/4b59d55e9e79f5a0b1932d6dc9f6b12a33d19266/aiohttp/client.py#L325-L328

By default the code is running fine, but this issue will appear in cases where someone will try to use `ClientSession._request()` method directly AND use keyword-arguments.

### How to recreate the bug?

This is a general example on how to recreate the bug. Lets assume that somewhere in my code I want to connect to some external http rest-api service using aiohttp library. I'll be using custom made session object based on ClientSession object from aiohttp library.

```python
from aiohttp import ClientSession, ClientResponse

class CustomSession(ClientSession):

    async def _request(self, method: str, str_or_url: StrOrURL, **kwargs: Any) -> ClientResponse:
        # put some extra code here, like add retry functionality
        client_response = await super()._request(method=method, str_or_url=url, **kwargs) # << ISSUE IS HERE
        return client_response
```
the above code is valid and it works as long as there is no apm agent running as middleware. With apm agent added, the code has to be written as below:
```python
        client_response = await super()._request(method, url, **kwargs)
```

### How to fix it?

Do search for both `url` and `str_or_url` in kwargs. It's simple fix that can fit in same line, I bet.


Discussion:
Thanks for the find! Would you be willing to submit a PR for this fix?
I can't promise it will be soon, but I'll try to put something together and submit it as PR. After all it should be a really simple fix.
Excellent! Looking forward to your PR. No hurry. :)

Resolving patch:
diff --git a/elasticapm/instrumentation/packages/asyncio/aiohttp_client.py b/elasticapm/instrumentation/packages/asyncio/aiohttp_client.py
--- a/elasticapm/instrumentation/packages/asyncio/aiohttp_client.py
+++ b/elasticapm/instrumentation/packages/asyncio/aiohttp_client.py
@@ -42,8 +42,10 @@ class AioHttpClientInstrumentation(AsyncAbstractInstrumentedModule):
     instrument_list = [("aiohttp.client", "ClientSession._request")]
 
     async def call(self, module, method, wrapped, instance, args, kwargs):
-        method = kwargs["method"] if "method" in kwargs else args[0]
-        url = kwargs["url"] if "url" in kwargs else args[1]
+        method = kwargs.get("method", args[0])
+        url = kwargs.get("url", kwargs.get("str_or_url", None))
+        if url is None:
+            url = args[1]
         url = str(url)
 
         signature = " ".join([method.upper(), get_host_from_url(url)])


Test patch:
diff --git a/tests/instrumentation/asyncio_tests/aiohttp_client_tests.py b/tests/instrumentation/asyncio_tests/aiohttp_client_tests.py
--- a/tests/instrumentation/asyncio_tests/aiohttp_client_tests.py
+++ b/tests/instrumentation/asyncio_tests/aiohttp_client_tests.py
@@ -73,6 +73,38 @@ async def test_http_get(instrument, event_loop, elasticapm_client, waiting_https
     assert spans[0]["outcome"] == "success"
 
 
+@pytest.mark.parametrize("use_yarl", [True, False])
+async def test_http_get_via_request(instrument, event_loop, elasticapm_client, waiting_httpserver, use_yarl):
+    assert event_loop.is_running()
+    elasticapm_client.begin_transaction("test")
+
+    url = waiting_httpserver.url
+    url = yarl.URL(url) if use_yarl else url
+
+    async with aiohttp.ClientSession() as session:
+        await session._request("GET", str_or_url=waiting_httpserver.url)
+
+    elasticapm_client.end_transaction()
+    transaction = elasticapm_client.events[constants.TRANSACTION][0]
+    spans = elasticapm_client.spans_for_transaction(transaction)
+    assert len(spans) == 1
+    span = spans[0]
+    assert span["name"] == "GET %s:%s" % waiting_httpserver.server_address
+    assert span["type"] == "external"
+    assert span["subtype"] == "http"
+    assert span["sync"] is False
+    assert span["context"]["http"]["url"] == waiting_httpserver.url
+    assert span["context"]["http"]["status_code"] == 204
+    assert spans[0]["context"]["destination"]["service"] == {
+        "name": "",
+        "resource": "127.0.0.1:%d" % waiting_httpserver.server_address[1],
+        "type": "",
+    }
+    assert spans[0]["context"]["service"]["target"]["type"] == "http"
+    assert spans[0]["context"]["service"]["target"]["name"] == f"127.0.0.1:{waiting_httpserver.server_address[1]}"
+    assert spans[0]["outcome"] == "success"
+
+
 @pytest.mark.parametrize("status_code", [400, 500])
 async def test_http_get_error(instrument, event_loop, elasticapm_client, waiting_httpserver, status_code):
     assert event_loop.is_running()