internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: elastic/apm-agent-python

Issue:
dictionary changed size during iteration
Same problem to https://github.com/elastic/apm-agent-python/issues/717

```
  File "/elasticapm/base.py", line 180, in <lambda>
    local_var,
  File "/elasticapm/utils/__init__.py", line 64, in varmap
    ret = func(name, dict((k, varmap(func, v, context, k, **kwargs)) for k, v in compat.iteritems(var)), **kwargs)
  File "/elasticapm/utils/__init__.py", line 64, in <genexpr>
    ret = func(name, dict((k, varmap(func, v, context, k, **kwargs)) for k, v in compat.iteritems(var)), **kwargs)

RuntimeError: dictionary changed size during iteration
```

Environment (please complete the following information)

OS: Ubuntu 20.04 stable
Python version: 2.7.18
Framework and version Django 1.3
APM Server version: 7.6.0
Agent version: 5.8
dictionary changed size during iteration
Same problem to https://github.com/elastic/apm-agent-python/issues/717

```
  File "/elasticapm/base.py", line 180, in <lambda>
    local_var,
  File "/elasticapm/utils/__init__.py", line 64, in varmap
    ret = func(name, dict((k, varmap(func, v, context, k, **kwargs)) for k, v in compat.iteritems(var)), **kwargs)
  File "/elasticapm/utils/__init__.py", line 64, in <genexpr>
    ret = func(name, dict((k, varmap(func, v, context, k, **kwargs)) for k, v in compat.iteritems(var)), **kwargs)

RuntimeError: dictionary changed size during iteration
```

Environment (please complete the following information)

OS: Ubuntu 20.04 stable
Python version: 2.7.18
Framework and version Django 1.3
APM Server version: 7.6.0
Agent version: 5.8


Discussion:
Hi @mrkiril 

Good catch! The solution from #717 should also work in this case. I'll open a PR.
Hi @mrkiril 

Good catch! The solution from #717 should also work in this case. I'll open a PR.

Resolving patch:
diff --git a/elasticapm/utils/__init__.py b/elasticapm/utils/__init__.py
--- a/elasticapm/utils/__init__.py
+++ b/elasticapm/utils/__init__.py
@@ -61,7 +61,10 @@ def varmap(func, var, context=None, name=None, **kwargs):
         return func(name, "<...>", **kwargs)
     context.add(objid)
     if isinstance(var, dict):
-        ret = func(name, dict((k, varmap(func, v, context, k, **kwargs)) for k, v in compat.iteritems(var)), **kwargs)
+        # iterate over a copy of the dictionary to avoid "dictionary changed size during iteration" issues
+        ret = func(
+            name, dict((k, varmap(func, v, context, k, **kwargs)) for k, v in compat.iteritems(var.copy())), **kwargs
+        )
     elif isinstance(var, (list, tuple)):
         ret = func(name, [varmap(func, f, context, name, **kwargs) for f in var], **kwargs)
     else:


Test patch:
Repository: elastic/apm-agent-python

Issue:
Expand k8s pod ID discovery regex
Implementing elastic/apm#344


Discussion:

Resolving patch:
diff --git a/elasticapm/utils/cgroup.py b/elasticapm/utils/cgroup.py
--- a/elasticapm/utils/cgroup.py
+++ b/elasticapm/utils/cgroup.py
@@ -36,7 +36,7 @@
 SYSTEMD_SCOPE_SUFFIX = ".scope"
 
 kubepods_regexp = re.compile(
-    r"(?:^/kubepods[\S]*/pod([^/]+)$)|(?:^/kubepods\.slice/kubepods-[^/]+\.slice/kubepods-[^/]+-pod([^/]+)\.slice$)"
+    r"(?:^/kubepods[\S]*/pod([^/]+)$)|(?:^/kubepods\.slice/(kubepods-[^/]+\.slice/)?kubepods[^/]*-pod([^/]+)\.slice$)"
 )
 
 container_id_regexp = re.compile(
@@ -97,9 +97,9 @@ def parse_cgroups(filehandle):
                 container_id = container_id.split("-", 1)[1]
         kubepods_match = kubepods_regexp.match(directory)
         if kubepods_match:
-            pod_id = kubepods_match.group(1)
+            pod_id = kubepods_match.group(1)  # if first part of kubepods_regexp matched
             if not pod_id:
-                pod_id = kubepods_match.group(2)
+                pod_id = kubepods_match.group(3)  # if second part of kubepods_regexp matched
                 if pod_id:
                     pod_id = pod_id.replace("_", "-")
             return {"container": {"id": container_id}, "kubernetes": {"pod": {"uid": pod_id}}}


Test patch:
diff --git a/tests/utils/cgroup_tests.py b/tests/utils/cgroup_tests.py
--- a/tests/utils/cgroup_tests.py
+++ b/tests/utils/cgroup_tests.py
@@ -77,6 +77,13 @@
                 "kubernetes": {"pod": {"uid": "5eadac96-ab58-11ea-b82b-0242ac110009"}},
             },
         ),
+        (
+            "9:freezer:/kubepods.slice/kubepods-pod22949dce_fd8b_11ea_8ede_98f2b32c645c.slice/docker-b15a5bdedd2e7645c3be271364324321b908314e4c77857bbfd32a041148c07f.scope",
+            {
+                "container": {"id": "b15a5bdedd2e7645c3be271364324321b908314e4c77857bbfd32a041148c07f"},
+                "kubernetes": {"pod": {"uid": "22949dce-fd8b-11ea-8ede-98f2b32c645c"}},
+            },
+        ),
     ],
 )
 def test_cgroup_parsing(test_input, expected):
Repository: elastic/apm-agent-python

Issue:
Remove Python 3.5 support
Python 3.5 hit EOL September 13, 2020. Support will be removed in our next major release.


Discussion:

Resolving patch:
diff --git a/elasticapm/__init__.py b/elasticapm/__init__.py
--- a/elasticapm/__init__.py
+++ b/elasticapm/__init__.py
@@ -36,8 +36,8 @@
     capture_span,
     get_span_id,
     get_trace_id,
-    get_transaction_id,
     get_trace_parent_header,
+    get_transaction_id,
     label,
     set_context,
     set_custom_context,
@@ -60,5 +60,7 @@
     VERSION = "unknown"
 
 
-if sys.version_info >= (3, 5):
-    from elasticapm.contrib.asyncio.traces import async_capture_span  # noqa: F401
+if sys.version_info <= (3, 5):
+    raise DeprecationWarning("The Elastic APM agent requires Python 3.6+")
+
+from elasticapm.contrib.asyncio.traces import async_capture_span  # noqa: F401


Test patch:
Repository: elastic/apm-agent-python

Issue:
[META 382] Zero-configuration support
Implement zero-configuration support for the agent as described in: https://github.com/elastic/apm/issues/382

SPEC issue: https://github.com/elastic/apm/issues/401


Discussion:

Resolving patch:
diff --git a/elasticapm/base.py b/elasticapm/base.py
--- a/elasticapm/base.py
+++ b/elasticapm/base.py
@@ -108,6 +108,8 @@ def __init__(self, config=None, **inline):
             for msg in config.errors.values():
                 self.error_logger.error(msg)
             config.disable_send = True
+        if config.service_name == "python_service":
+            self.logger.warning("No custom SERVICE_NAME was set -- using non-descript default 'python_service'")
         self.config = VersionedConfig(config, version=None)
 
         # Insert the log_record_factory into the logging library
diff --git a/elasticapm/conf/__init__.py b/elasticapm/conf/__init__.py
--- a/elasticapm/conf/__init__.py
+++ b/elasticapm/conf/__init__.py
@@ -472,7 +472,9 @@ def copy(self):
 
 
 class Config(_ConfigBase):
-    service_name = _ConfigValue("SERVICE_NAME", validators=[RegexValidator("^[a-zA-Z0-9 _-]+$")], required=True)
+    service_name = _ConfigValue(
+        "SERVICE_NAME", validators=[RegexValidator("^[a-zA-Z0-9 _-]+$")], default="python_service", required=True
+    )
     service_node_name = _ConfigValue("SERVICE_NODE_NAME")
     environment = _ConfigValue("ENVIRONMENT")
     secret_token = _ConfigValue("SECRET_TOKEN")


Test patch:
diff --git a/tests/contrib/django/django_tests.py b/tests/contrib/django/django_tests.py
--- a/tests/contrib/django/django_tests.py
+++ b/tests/contrib/django/django_tests.py
@@ -1181,16 +1181,6 @@ def test_subcommand_not_known(argv_mock):
     assert 'No such command "foo"' in output
 
 
-def test_settings_missing():
-    stdout = compat.StringIO()
-    with override_settings(ELASTIC_APM={}):
-        call_command("elasticapm", "check", stdout=stdout)
-    output = stdout.getvalue()
-    assert "Configuration errors detected" in output
-    assert "SERVICE_NAME not set" in output
-    assert "optional SECRET_TOKEN not set" in output
-
-
 def test_settings_missing_secret_token_no_https():
     stdout = compat.StringIO()
     with override_settings(ELASTIC_APM={"SERVER_URL": "http://foo"}):
@@ -1255,7 +1245,6 @@ def test_settings_server_url_not_http_nor_https():
     with override_settings(ELASTIC_APM={"SERVER_URL": "xhttp://dev.brwnppr.com:8000/"}):
         call_command("elasticapm", "check", stdout=stdout)
     output = stdout.getvalue()
-    assert "Configuration errors detected" in output
     assert "SERVER_URL has scheme xhttp and we require http or https" in output
Repository: elastic/apm-agent-python

Issue:
Using use_elastic_excepthook throws TypeError
**Describe the bug**:
When using the option [use_elastic_excepthook](https://www.elastic.co/guide/en/apm/agent/python/master/configuration.html#config-use-elastic-excepthook) every uncaught exception also produces the following `TypeError`:

```bash
Error in sys.excepthook:
TypeError: _excepthook() takes 1 positional argument but 4 were given
```

**To Reproduce**

Run the following Python snippet
```python
import elasticapm


def run():
    raise Exception("hello")

client = elasticapm.Client(
    {"SERVICE_NAME": "test"},
    use_elastic_excepthook=True,
)
run()
```
It will produce this output:

```bash
Error in sys.excepthook:
TypeError: _excepthook() takes 1 positional argument but 4 were given

Original exception was:
Traceback (most recent call last):
  File "t.py", line 13, in <module>
    run()
  File "t.py", line 5, in run
    raise Exception("hello")
Exception: hello
```

**Environment (please complete the following information)**
- OS: MacOS 10.15.3
- Python version: 3.8.5
- Agent version: 6.0.0


**Additional context**

The problem seems to be with the function signature of the [Client._excepthook](https://github.com/elastic/apm-agent-python/blob/b7e408a7b8c9554c0ee74a5b7e7c6347b4b814f2/elasticapm/base.py#L567) function that is used to override the `sys.excepthook`.

The [sys.excepthook](https://docs.python.org/3/library/sys.html#sys.excepthook) expects three arguments (type, value, traceback), while the `Client._excepthook` only expects the `self` argument.




Discussion:
Thanks for reporting this! We'll get it fixed.

Resolving patch:
diff --git a/elasticapm/base.py b/elasticapm/base.py
--- a/elasticapm/base.py
+++ b/elasticapm/base.py
@@ -571,14 +571,13 @@ def _get_stack_info_for_trace(
             locals_processor_func=locals_processor_func,
         )
 
-    def _excepthook(self):
-        exec_info = sys.exc_info()
+    def _excepthook(self, type_, value, traceback):
         try:
-            self.original_excepthook(*exec_info)
+            self.original_excepthook(type_, value, traceback)
         except Exception:
             self.capture_exception(handled=False)
         finally:
-            self.capture_exception(exec_info, handled=False)
+            self.capture_exception(exc_info=(type_, value, traceback), handled=False)
 
     def load_processors(self):
         """


Test patch:
diff --git a/tests/client/client_tests.py b/tests/client/client_tests.py
--- a/tests/client/client_tests.py
+++ b/tests/client/client_tests.py
@@ -885,3 +885,13 @@ def test_client_enabled(elasticapm_client):
         assert not elasticapm_client.config.is_recording
         for manager in elasticapm_client._thread_managers.values():
             assert not manager.is_started()
+
+
+def test_excepthook(elasticapm_client):
+    try:
+        raise Exception("hi!")
+    except Exception:
+        type_, value, traceback = sys.exc_info()
+        elasticapm_client._excepthook(type_, value, traceback)
+
+    assert elasticapm_client.events[ERROR]
Repository: elastic/apm-agent-python

Issue:
Traceback from agent on max spans
**Describe the bug**: ...

We are seeing the following traceback coming out of the ElasticAPM agent via Sentry frequently on many of our environments.

```
KeyError: 'TRANSACTION_MAX_SPANS'
  File "elasticapm/utils/threading.py", line 80, in run
    rval = self._function(*self._args, **self._kwargs)
  File "elasticapm/conf/__init__.py", line 701, in update_config
    self.reset()
  File "elasticapm/conf/__init__.py", line 650, in reset
    if self._config.values[key] != self._first_config.values[key]:
```

**To Reproduce**

Unknown. However, Sentry shows a lot of POST requests going to the APM server before this happens. Our assumption is that we are hitting TRANSACTION_MAX_SPANS, but this is not being handled gracefully, and causing this exception instead.

**Environment (please complete the following information)**
- OS: Linux
- Python version: 3.8
- Framework and version: Django 3.1.7
- APM Server version: 7.6
- Agent version: 6.0.0



Discussion:

Resolving patch:
diff --git a/elasticapm/conf/__init__.py b/elasticapm/conf/__init__.py
--- a/elasticapm/conf/__init__.py
+++ b/elasticapm/conf/__init__.py
@@ -689,7 +689,7 @@ def reset(self):
         """
         callbacks = []
         for key in compat.iterkeys(self._config.values):
-            if self._config.values[key] != self._first_config.values[key]:
+            if key in self._first_config.values and self._config.values[key] != self._first_config.values[key]:
                 callbacks.append((key, self._config.values[key], self._first_config.values[key]))
 
         with self._lock:


Test patch:
diff --git a/tests/config/tests.py b/tests/config/tests.py
--- a/tests/config/tests.py
+++ b/tests/config/tests.py
@@ -396,6 +396,19 @@ class MyConfig(_ConfigBase):
     assert test_var["foo"] == 3
 
 
+def test_reset_after_adding_config():
+    class MyConfig(_ConfigBase):
+        foo = _ConfigValue("foo")
+        bar = _ConfigValue("bar")
+
+    c = VersionedConfig(MyConfig({"foo": "baz"}), version=1)
+
+    c.update(version=2, bar="bazzinga")
+
+    c.reset()
+    assert c.bar is None
+
+
 def test_valid_values_validator():
     # Case sensitive
     v = EnumerationValidator(["foo", "Bar", "baz"], case_sensitive=False)
Repository: elastic/apm-agent-python

Issue:
FastAPI: Calls to undefined routes result in 500
**Description**
When using the apm agent with FastAPI, calls to any undefined route results in a 500 internal server error. Without apm, 404 not found is returned correctly.

**To Reproduce**

Run the following simple FastAPI app:
```python
import uvicorn
from elasticapm.contrib.starlette import ElasticAPM, make_apm_client
from fastapi import FastAPI

app = FastAPI()

apm = make_apm_client(
    {
        "SERVICE_NAME": "undefined-route",
    }
)

app.add_middleware(ElasticAPM, client=apm)

@app.get("/hi")
async def hi():
    return "hi"

if __name__ == "__main__":
    uvicorn.run(app, host="0.0.0.0", port=80)
```

**Observed behavior**
Calling `http://localhost/bye` results in a 500 internal server error.

**Expected behavior**
Calling  `http://localhost/bye` results in a 404 not found error.

**Environment**

- OS:
  - Client: Windows
  - Server: Ubuntu
- Python version: `3.7.3`
- Framework and version: `fastapi==0.61.2`
- APM Server version: docker image `elasticsearch/elasticsearch:7.10.2`
- Agent version: `elastic-apm==master`


Discussion:

Resolving patch:
diff --git a/elasticapm/contrib/starlette/__init__.py b/elasticapm/contrib/starlette/__init__.py
--- a/elasticapm/contrib/starlette/__init__.py
+++ b/elasticapm/contrib/starlette/__init__.py
@@ -225,7 +225,8 @@ def get_route_name(self, request: Request) -> str:
                 trim = False
 
             route_name = self._get_route_name(redirect_scope, routes)
-            route_name = route_name + "/" if trim else route_name[:-1]
+            if route_name is not None:
+                route_name = route_name + "/" if trim else route_name[:-1]
         return route_name
 
     def _get_route_name(self, scope, routes, route_name=None):


Test patch:
diff --git a/tests/contrib/asyncio/starlette_tests.py b/tests/contrib/asyncio/starlette_tests.py
--- a/tests/contrib/asyncio/starlette_tests.py
+++ b/tests/contrib/asyncio/starlette_tests.py
@@ -329,6 +329,27 @@ def test_transaction_name_is_route_for_mounts(app, elasticapm_client):
     assert transaction["context"]["request"]["url"]["pathname"] == "/sub/subsub/hihi/shay"
 
 
+def test_undefined_route(app, elasticapm_client):
+    client = TestClient(app)
+
+    response = client.get("/undefined")
+
+    assert response.status_code == 404
+    assert len(elasticapm_client.events[constants.TRANSACTION]) == 1
+    transaction = elasticapm_client.events[constants.TRANSACTION][0]
+    assert transaction["name"] == "GET /undefined"
+
+
+def test_undefined_mounted_route(app, elasticapm_client):
+    client = TestClient(app)
+
+    response = client.get("/sub/subsub/undefined")
+
+    assert response.status_code == 404
+    assert len(elasticapm_client.events[constants.TRANSACTION]) == 1
+    transaction = elasticapm_client.events[constants.TRANSACTION][0]
+    assert transaction["name"] == "GET /sub/subsub/undefined"
+
 
 @pytest.mark.parametrize(
     "elasticapm_client",
Repository: elastic/apm-agent-python

Issue:
Python agent should show database as a dependency, shows up in span
**Is your feature request related to a problem? Please describe.**
I can see Postgres under **Time spent by span type** graph but it is not shown as a dependency. Rails agent is able to show this as Postgresql

<img width="397" alt="Screenshot 2021-04-24 at 12 03 10 PM" src="https://user-images.githubusercontent.com/36618391/115949791-14edbe80-a4f5-11eb-999a-9b2880460abc.png">

I am using asyncpg==0.21.0

**Describe the solution you'd like**
The database(Postgres) should show up as a dependency just like it shows up for ruby agent. Screenshot attached below.

<img width="979" alt="Screenshot 2021-04-24 at 11 40 09 AM" src="https://user-images.githubusercontent.com/36618391/115949754-d526d700-a4f4-11eb-9d43-bad114aad4eb.png">



Discussion:

Resolving patch:
diff --git a/elasticapm/instrumentation/packages/asyncio/asyncpg.py b/elasticapm/instrumentation/packages/asyncio/asyncpg.py
--- a/elasticapm/instrumentation/packages/asyncio/asyncpg.py
+++ b/elasticapm/instrumentation/packages/asyncio/asyncpg.py
@@ -31,6 +31,7 @@
 from elasticapm.contrib.asyncio.traces import async_capture_span
 from elasticapm.instrumentation.packages.asyncio.base import AsyncAbstractInstrumentedModule
 from elasticapm.instrumentation.packages.dbapi2 import extract_signature
+from elasticapm.utils import default_ports
 
 
 class AsyncPGInstrumentation(AsyncAbstractInstrumentedModule):
@@ -56,6 +57,12 @@ async def call(self, module, method, wrapped, instance, args, kwargs):
         name = extract_signature(query)
         context = {"db": {"type": "sql", "statement": query}}
         action = "query"
+        destination_info = {
+            "address": kwargs.get("host", "localhost"),
+            "port": int(kwargs.get("port", default_ports.get("postgresql"))),
+            "service": {"name": "postgres", "resource": "postgres", "type": "db"},
+        }
+        context["destination"] = destination_info
         async with async_capture_span(
             name, leaf=True, span_type="db", span_subtype="postgres", span_action=action, extra=context
         ):


Test patch:
Repository: elastic/apm-agent-python

Issue:
RemovedInDjango41Warning
**Describe the bug**: ...

```shell
../../../../.pyenv/versions/3.8.2/lib/python3.8/site-packages/django/apps/registry.py:91
  /Users/mingyu.wu/.pyenv/versions/3.8.2/lib/python3.8/site-packages/django/apps/registry.py:91: RemovedInDjango41Warning: 'elasticapm.contrib.django' defines default_app_config = 'elasticapm.contrib.django.apps.ElasticAPMConfig'. Django now detects this configuration automatically. You can remove default_app_config.
    app_config = AppConfig.create(entry)

-- Docs: https://docs.pytest.org/en/stable/warnings.html
```

**To Reproduce**

1. run unit test or start server

**Environment (please complete the following information)**
- OS: Linux/MacOS
- Python version:3.8.2
- Framework and version: Django 3.2.2
- APM Server version: *
- Agent version: 6.1.3



Discussion:

Resolving patch:
diff --git a/elasticapm/contrib/django/__init__.py b/elasticapm/contrib/django/__init__.py
--- a/elasticapm/contrib/django/__init__.py
+++ b/elasticapm/contrib/django/__init__.py
@@ -27,7 +27,9 @@
 #  CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
 #  OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
 #  OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+from django import VERSION as DJANGO_VERSION
 
 from elasticapm.contrib.django.client import *  # noqa E401
 
-default_app_config = "elasticapm.contrib.django.apps.ElasticAPMConfig"
+if DJANGO_VERSION < (3, 2):
+    default_app_config = "elasticapm.contrib.django.apps.ElasticAPMConfig"


Test patch:
diff --git a/tests/contrib/django/django_tests.py b/tests/contrib/django/django_tests.py
--- a/tests/contrib/django/django_tests.py
+++ b/tests/contrib/django/django_tests.py
@@ -1604,3 +1604,14 @@ def test_django_ignore_transaction_urls(client, django_elasticapm_client):
         django_elasticapm_client.config.update(1, transaction_ignore_urls="/no*")
         client.get("/no-error")
     assert len(django_elasticapm_client.events[TRANSACTION]) == 1
+
+
+def test_default_app_config_present_by_version():
+    # The default_app_config attribute was deprecated in Django 3.2
+    import elasticapm.contrib.django
+
+    default_app_config_is_defined = hasattr(elasticapm.contrib.django, "default_app_config")
+    if django.VERSION < (3, 2):
+        assert default_app_config_is_defined
+    else:
+        assert not default_app_config_is_defined