internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: dynaconf/dynaconf

Issue:
[bug] reload() function does not clear old hooks
**Describe the bug**
the `reload()` function of the `LazySettings` class does not clear the `_loaded_hooks` attribute. Because of that, when trying to reload and rerun the loaders, the hook won't run again.

**To Reproduce**
Steps to reproduce the behavior:

Configure a post hook just like in the docs. Create a `DynaConf` object and then run `reload()` on the object.

**Expected behavior**
The post hook should run after every reload.

**Additional context**
I'll be glad to make a PR for this myself.


Discussion:

Resolving patch:
diff --git a/dynaconf/base.py b/dynaconf/base.py
--- a/dynaconf/base.py
+++ b/dynaconf/base.py
@@ -1124,6 +1124,7 @@ def loaders(self):  # pragma: no cover
     def reload(self, env=None, silent=None):  # pragma: no cover
         """Clean end Execute all loaders"""
         self.clean()
+        self._loaded_hooks.clear()
         self.execute_loaders(env, silent)
 
     def execute_loaders(


Test patch:
diff --git a/tests_functional/issues/850_reload-hooks/app.py b/tests_functional/issues/850_reload-hooks/app.py
new file mode 100644
--- /dev/null
+++ b/tests_functional/issues/850_reload-hooks/app.py
@@ -0,0 +1,12 @@
+"""
+Assert settings.reload() re-reruns hooks.
+"""
+from __future__ import annotations
+
+from dynaconf import Dynaconf
+
+settings = Dynaconf(settings_file="settings.toml")
+
+assert settings.counter == 1
+settings.reload()
+assert settings.counter == 1
diff --git a/tests_functional/issues/850_reload-hooks/dynaconf_hooks.py b/tests_functional/issues/850_reload-hooks/dynaconf_hooks.py
new file mode 100644
--- /dev/null
+++ b/tests_functional/issues/850_reload-hooks/dynaconf_hooks.py
@@ -0,0 +1,5 @@
+from __future__ import annotations
+
+
+def post(settings):
+    return {"counter": settings.counter + 1}
diff --git a/tests_functional/issues/850_reload-hooks/settings.toml b/tests_functional/issues/850_reload-hooks/settings.toml
new file mode 100644
--- /dev/null
+++ b/tests_functional/issues/850_reload-hooks/settings.toml
@@ -0,0 +1 @@
+counter=0
Repository: dynaconf/dynaconf

Issue:
Deprecation warning raised by hvac
**Describe the bug**
Using the latest version of dynaconf along with the Hashicorp Vault integration, we get the following deprecation warning.
```
/opt/app-root/lib64/python3.11/site-packages/dynaconf/loaders/vault_loader.py:101: DeprecationWarning: The raise_on_deleted parameter will change its default value to False in hvac v3.0.0. The current default of True will presere previous behavior. To use the old behavior with no warning, explicitly set this value to True. See https://github.com/hvac/hvac/pull/907
      data = client.secrets.kv.v2.read_secret_version(
```

**To Reproduce**
Steps to reproduce the behavior:
Create a LazySettings object using the Vault integration.


**Expected behavior**
no deprecation warning

**Environment (please complete the following information):**
hvac==1.0.2
dynaconf==3.1.12

**Additional context**
This is not exactly a bug, but something that should be addressed before hvac 3.0.0 gets out



Discussion:

Resolving patch:
diff --git a/dynaconf/loaders/redis_loader.py b/dynaconf/loaders/redis_loader.py
--- a/dynaconf/loaders/redis_loader.py
+++ b/dynaconf/loaders/redis_loader.py
@@ -94,7 +94,7 @@ def write(obj, data=None, **kwargs):
     redis_data = {
         upperfy(key): unparse_conf_data(value) for key, value in data.items()
     }
-    client.hmset(holder.upper(), redis_data)
+    client.hset(holder.upper(), mapping=redis_data)
     load(obj)
 
 
diff --git a/dynaconf/loaders/vault_loader.py b/dynaconf/loaders/vault_loader.py
--- a/dynaconf/loaders/vault_loader.py
+++ b/dynaconf/loaders/vault_loader.py
@@ -109,7 +109,9 @@ def load(obj, env=None, silent=None, key=None, validate=False):
         try:
             if obj.VAULT_KV_VERSION_FOR_DYNACONF == 2:
                 data = client.secrets.kv.v2.read_secret_version(
-                    path, mount_point=obj.VAULT_MOUNT_POINT_FOR_DYNACONF
+                    path,
+                    mount_point=obj.VAULT_MOUNT_POINT_FOR_DYNACONF,
+                    raise_on_deleted_version=True,  # keep default behavior
                 )
             else:
                 data = client.secrets.kv.read_secret(


Test patch:
Repository: dynaconf/dynaconf

Issue:
[bug] Error handling on dynaconf get CLI 
**Describe the bug**
When using get to read a non existent variable it outputs the python traceback to the CLI

**To Reproduce**
```console
$ dynaconf get DONTEXIST
Traceback (most recent call last):
  File "/venv/bin/dynaconf", line 8, in <module>
    sys.exit(main())
  File "/src/dynaconf/dynaconf/vendor/click/core.py", line 857, in __call__
    return self.main(*args, **kwargs)
  File "/src/dynaconf/dynaconf/vendor/click/core.py", line 810, in main
    rv = self.invoke(ctx)
  File "/src/dynaconf/dynaconf/vendor/click/core.py", line 1292, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/src/dynaconf/dynaconf/vendor/click/core.py", line 1099, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/src/dynaconf/dynaconf/vendor/click/core.py", line 613, in invoke
    return callback(*args, **kwargs)
  File "/src/dynaconf/dynaconf/cli.py", line 476, in get
    result = settings[key]  # let the keyerror raises
  File "/src/dynaconf/dynaconf/utils/functional.py", line 19, in inner
    return func(self._wrapped, *args)
  File "/src/dynaconf/dynaconf/base.py", line 315, in __getitem__
    raise KeyError(f"{item} does not exist")
KeyError: 'DONTEXIST does not exist'
```

**Expected behavior**

```console
$ dynaconf get DONTEXIST
Key not found.  # stderr
$ echo $?   # proper retcode
1
```

The same behavior as we have in `list`

```console
$ dynaconf list -k DONTEXIST
Django app detected
Working in development environment 
Key not found
```



Discussion:

Resolving patch:
diff --git a/dynaconf/cli.py b/dynaconf/cli.py
--- a/dynaconf/cli.py
+++ b/dynaconf/cli.py
@@ -473,7 +473,11 @@ def get(key, default, env, unparse):
     if default is not empty:
         result = settings.get(key, default)
     else:
-        result = settings[key]  # let the keyerror raises
+        try:
+            result = settings[key]
+        except KeyError:
+            click.echo("Key not found", nl=False, err=True)
+            sys.exit(1)
 
     if unparse:
         result = unparse_conf_data(result)
@@ -596,7 +600,7 @@ def format_setting(_k, _v):
             value = empty
 
         if value is empty:
-            click.echo(click.style("Key not found", bg="red", fg="white"))
+            click.secho("Key not found", bg="red", fg="white", err=True)
             return
 
         click.echo(format_setting(key, value))


Test patch:
diff --git a/tests/test_cli.py b/tests/test_cli.py
--- a/tests/test_cli.py
+++ b/tests/test_cli.py
@@ -126,6 +126,17 @@ def test_get(testdir):
     assert result == "test_value"
 
 
+def test_negative_get(testdir):
+    """Tests get command erroring when key does not exist"""
+    cmd = ["get", "DONTEXIST"]
+    env = {
+        "ROOT_PATH_FOR_DYNACONF": testdir,
+        "INSTANCE_FOR_DYNACONF": "tests.config.settings",
+    }
+    assert run(cmd, env=env, attr="stdout") == "Key not found"
+    assert run(cmd, env=env, attr="exit_code") == 1
+
+
 def test_get_json_dict(testdir):
     """Tests get command printing json"""
     env = env = {
Repository: dynaconf/dynaconf

Issue:
Field '_bypass_evaluation' provided as part of the value of a setting
**Describe the bug**
When a BoxList containing a dictionary is created, when looking up the value of the BoxList, the field `_bypass_evaluation` is present in the dictionary.  This is unexpected.

**To Reproduce**
Steps to reproduce the behavior:

Using Dynaconf v3.2.0:

```
from dynaconf import Dynaconf

settings = Dynaconf()
settings.set("key", [{"d": "v"}])
settings.key
```
This provides the value of the key as:
`<BoxList: [{'d': 'v', '_bypass_evaluation': False}]>`

However, it is expected to be:
`<BoxList: [{'d': 'v'}]>`

**Executing under the following environment**
This happens in Ubuntu 20.04, python3, using  Dynaconf v3.2.0

**Proposed solution**
The following line:
```self._store._box_config["_bypass_evaluation"] = False```

at 
https://github.com/dynaconf/dynaconf/blob/678d29b7383f03934a47e21b1bcdf68946d5c501/dynaconf/base.py#L1174

can be replaced with:
```self._store._box_config.pop("_bypass_evaluation")```

This has the same result as setting it to False with the benefit that it is removed completely.



Discussion:

Resolving patch:
diff --git a/dynaconf/base.py b/dynaconf/base.py
--- a/dynaconf/base.py
+++ b/dynaconf/base.py
@@ -1171,7 +1171,7 @@ def execute_loaders(
         execute_module_hooks("post", self, env, silent=silent, key=key)
         execute_instance_hooks(self, "post", self._post_hooks)
 
-        self._store._box_config["_bypass_evaluation"] = False
+        self._store._box_config.pop("_bypass_evaluation")
 
     def pre_load(self, env, silent, key):
         """Do we have any file to pre-load before main settings file?"""


Test patch:
diff --git a/tests/test_base.py b/tests/test_base.py
--- a/tests/test_base.py
+++ b/tests/test_base.py
@@ -1423,3 +1423,10 @@ def test_get_with_sysenv_fallback_local_overrides():
     assert settings.sysenv_fallback_for_dynaconf is True
     assert settings.get("another_test") == "ANOTHER_VALUE"
     assert not settings.get("another_test", sysenv_fallback=False)
+
+
+# issue #965
+def test_no_extra_values_in_nested_structure():
+    settings = Dynaconf()
+    settings.set("key", [{"d": "v"}])
+    assert settings.key == [{"d": "v"}]
Repository: dynaconf/dynaconf

Issue:
[bug] KeyError: '_bypass_evaluation' in Dynaconf 3.2.1
**Describe the bug**
The change introduced in PR #966 also introduced a KeyError in our tests.

**To Reproduce**
Steps to reproduce the behavior:
Not currently clear without digging further into the cause, but this was triggered in our tests using multi-threading. You can see the failures in the github action link here: https://github.com/SatelliteQE/broker/actions/runs/5928071069/job/16074442428?pr=227

**Expected behavior**
This KeyError should not be triggered. Instead, there should likely be a check that the key exists before attempting to pop or give a default pop value of None.
```python
self._store._box_config.pop("_bypass_evaluation", None)
```



Discussion:
@pedro-psb this should be an easy fix :)

Resolving patch:
diff --git a/dynaconf/base.py b/dynaconf/base.py
--- a/dynaconf/base.py
+++ b/dynaconf/base.py
@@ -1171,7 +1171,7 @@ def execute_loaders(
         execute_module_hooks("post", self, env, silent=silent, key=key)
         execute_instance_hooks(self, "post", self._post_hooks)
 
-        self._store._box_config.pop("_bypass_evaluation")
+        self._store._box_config.pop("_bypass_evaluation", None)
 
     def pre_load(self, env, silent, key):
         """Do we have any file to pre-load before main settings file?"""


Test patch:
Repository: dynaconf/dynaconf

Issue:
[bug] TypeError for older versions of HVAC in read_secret_version method
**Describe the bug**
A combination of newer versions of Dynaconf with older versions of HVAC result in an incompatible mix of expected vs available arguments. Specifically you can get the following traceback.

```python
    109 try:
    110     if obj.VAULT_KV_VERSION_FOR_DYNACONF == 2:
--> 111         data = client.secrets.kv.v2.read_secret_version(
    112             path,
    113             mount_point=obj.VAULT_MOUNT_POINT_FOR_DYNACONF,
    114             raise_on_deleted_version=True,  # keep default behavior
    115         )
    116     else:
    117         data = client.secrets.kv.read_secret(
    118             "data/" + path,
    119             mount_point=obj.VAULT_MOUNT_POINT_FOR_DYNACONF,
    120         )

TypeError: KvV2.read_secret_version() got an unexpected keyword argument 'raise_on_deleted_version'
```

The PR introducing this feature was included in HVAC 1.1.0: https://github.com/hvac/hvac/pull/907 

**To Reproduce**
Steps to reproduce the behavior:

1. Have a version of HVAC older than 1.1.0
2. Trigger a vault version read





Discussion:

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -33,7 +33,7 @@ def read(*names, **kwargs):
     "python-dotenv",
     "toml",
     "redis",
-    "hvac",
+    "hvac>=1.1.0",
     "configobj",
 ]
 


Test patch:
Repository: elastic/apm-agent-python

Issue:
Add ELASTIC_APM_SERVICE_NODE_NAME config
See https://github.com/elastic/apm/issues/141 for implementation details



Discussion:

Resolving patch:
diff --git a/elasticapm/base.py b/elasticapm/base.py
--- a/elasticapm/base.py
+++ b/elasticapm/base.py
@@ -317,6 +317,8 @@ def get_service_info(self):
                 "name": keyword_field(self.config.framework_name),
                 "version": keyword_field(self.config.framework_version),
             }
+        if self.config.service_node_name:
+            result["node"] = {"configured_name": keyword_field(self.config.service_node_name)}
         self._service_info = result
         return result
 
diff --git a/elasticapm/conf/__init__.py b/elasticapm/conf/__init__.py
--- a/elasticapm/conf/__init__.py
+++ b/elasticapm/conf/__init__.py
@@ -252,6 +252,7 @@ def errors(self):
 
 class Config(_ConfigBase):
     service_name = _ConfigValue("SERVICE_NAME", validators=[RegexValidator("^[a-zA-Z0-9 _-]+$")], required=True)
+    service_node_name = _ConfigValue("SERVICE_NODE_NAME", default=None)
     environment = _ConfigValue("ENVIRONMENT", default=None)
     secret_token = _ConfigValue("SECRET_TOKEN")
     api_key = _ConfigValue("API_KEY")


Test patch:
diff --git a/tests/client/client_tests.py b/tests/client/client_tests.py
--- a/tests/client/client_tests.py
+++ b/tests/client/client_tests.py
@@ -63,6 +63,18 @@ def test_service_info(elasticapm_client):
     assert service_info["agent"]["name"] == "python"
 
 
+@pytest.mark.parametrize(
+    "elasticapm_client", [{"environment": "production", "service_node_name": "my_node"}], indirect=True
+)
+def test_service_info_node_name(elasticapm_client):
+    service_info = elasticapm_client.get_service_info()
+    assert service_info["name"] == elasticapm_client.config.service_name
+    assert service_info["environment"] == elasticapm_client.config.environment == "production"
+    assert service_info["language"] == {"name": "python", "version": platform.python_version()}
+    assert service_info["agent"]["name"] == "python"
+    assert service_info["node"]["configured_name"] == "my_node"
+
+
 def test_process_info(elasticapm_client):
     with mock.patch.object(sys, "argv", ["a", "b", "c"]):
         process_info = elasticapm_client.get_process_info()
Repository: elastic/apm-agent-python

Issue:
'AsyncTransport' has no attribute '_start_event_processor'
**Describe the bug**: ...

After upgrading to `elastic-apm==5.4.1` I now get an error when Celery starts:

```
<function _register_worker_signals.<locals>.worker_startup at 0x7feae4beb620> raised: AttributeError("'AsyncTransport' object has no attribute '_start_event_processor'",)
Traceback (most recent call last):
  File "/venv/lib/python3.6/site-packages/celery/utils/dispatch/signal.py", line 288, in send
    response = receiver(signal=self, sender=sender, **named)
  File "/venv/1.37.1/lib/python3.6/site-packages/elasticapm/contrib/celery/__init__.py", line 80, in worker_startup
    client._transport._start_event_processor()
AttributeError: 'AsyncTransport' object has no attribute '_start_event_processor' 
```

**Environment (please complete the following information)**
- OS: Linux-3.10.0-1062.9.1.el7.x86_64-x86_64-with-centos-7.7.1908-Core 2020-02-18 16:24:31
- Python version: Python 3.6.8
- Framework and version: 
  celery 4.4.0
  Django 3.0.3
- APM Server version: ?
- Agent version: 5.4.1

I see the same error mentioned in issue #704, but I don't seem to have an issue with restarting Celery workers.


Discussion:
@jaap3 thanks for opening the issue! We are already working on a fix, hopefully to be released later today.

Sorry for the inconvenience!

Resolving patch:
diff --git a/elasticapm/contrib/celery/__init__.py b/elasticapm/contrib/celery/__init__.py
--- a/elasticapm/contrib/celery/__init__.py
+++ b/elasticapm/contrib/celery/__init__.py
@@ -76,14 +76,10 @@ def end_transaction(task_id, task, *args, **kwargs):
 
 
 def _register_worker_signals(client):
-    def worker_startup(*args, **kwargs):
-        client._transport._start_event_processor()
-
     def worker_shutdown(*args, **kwargs):
         client.close()
 
     def connect_worker_process_init(*args, **kwargs):
-        signals.worker_process_init.connect(worker_startup, dispatch_uid="elasticapm-start-worker", weak=False)
         signals.worker_process_shutdown.connect(worker_shutdown, dispatch_uid="elasticapm-shutdown-worker", weak=False)
 
     signals.worker_init.connect(


Test patch:
diff --git a/tests/fixtures.py b/tests/fixtures.py
--- a/tests/fixtures.py
+++ b/tests/fixtures.py
@@ -233,7 +233,10 @@ def queue(self, event_type, data, flush=False):
         self.events[event_type].append(data)
         self._flushed.set()
 
-    def _start_event_processor(self):
+    def start_thread(self):
+        pass
+
+    def stop_thread(self):
         pass
 
     def get_config(self, current_version=None, keys=None):
Repository: elastic/apm-agent-python

Issue:
Coerce HTTP headers to string before reporting HTTP requests to APM Server
Hi,

I started with elastic-apm and no data is captured, in case of error

`TransportException elasticapm.transport.http in send`

`document against schema: I[#] S[#] doesn\'t validate with \\"transaction#\\"\\n  I[#] S[#/allOf/3] allOf failed\\n    I[#/context/request/headers/last-region] S[#/allOf/3/properties/context/properties/request/properties/headers/patternProperties/%5B.%2A%5D%2A$/type] expected string or array or null, but got number",`

header last-region have integer value, and this cause the problem

**Environment (please complete the following information)**
- OS: [e.g. Linux] Ubuntu 18.04
- Python version: 3.6
- Framework and version [e.g. Django 2.1]: Django 2.0
- APM Server version: 7.6.2
- Agent version: 5.5.2

looks similar to https://github.com/elastic/apm-agent-ruby/issues/672



Discussion:

Resolving patch:
diff --git a/elasticapm/contrib/django/client.py b/elasticapm/contrib/django/client.py
--- a/elasticapm/contrib/django/client.py
+++ b/elasticapm/contrib/django/client.py
@@ -122,7 +122,13 @@ def get_data_from_request(self, request, event_type):
             "cookies": dict(request.COOKIES),
         }
         if self.config.capture_headers:
-            result["headers"] = dict(get_headers(request.META))
+            request_headers = dict(get_headers(request.META))
+
+            for key, value in request_headers.items():
+                if isinstance(value, (int, float)):
+                    request_headers[key] = str(value)
+
+            result["headers"] = request_headers
 
         if request.method in constants.HTTP_WITH_BODY:
             content_type = request.META.get("CONTENT_TYPE")
@@ -162,7 +168,14 @@ def get_data_from_response(self, response, event_type):
         result = {"status_code": response.status_code}
 
         if self.config.capture_headers and hasattr(response, "items"):
-            result["headers"] = dict(response.items())
+            response_headers = dict(response.items())
+
+            for key, value in response_headers.items():
+                if isinstance(value, (int, float)):
+                    response_headers[key] = str(value)
+
+            result["headers"] = response_headers
+
         return result
 
     def capture(self, event_type, request=None, **kwargs):


Test patch:
Repository: elastic/apm-agent-python

Issue:
dictionary changed size during iteration
I get this error pretty often in `TransactionsMetricSet`:

```
RuntimeError: dictionary changed size during iteration
  File "elasticapm/utils/threading.py", line 79, in run
    rval = self._function(*self._args, **self._kwargs)
  File "elasticapm/metrics/base_metrics.py", line 97, in collect
    for data in metricset.collect():
  File "elasticapm/metrics/base_metrics.py", line 215, in collect
    for (name, labels), t in compat.iteritems(self._timers):
```

**Environment (please complete the following information)**
- OS: Debian stable
- Python version: 2.7.13
- Framework and version [e.g. Django 2.1]: 1.11.20
- APM Server version: 7.6.0
- Agent version: 5.3.3



Discussion:
@starenka that's interesting. It probably happens because while we iterate over the metrics in the background thread, the foreground thread (the one that is handling Django requests) is adding new metrics data.

There are two ways to fix this: we could either take the lock before iterating, or turns the items iterator into a list before iterating over it.

Taking a lock would probably the cleanest way, but it'll block the main thread, even if it is for a very short amount of time. OTOH, turning the iterator into a list would consume a little bit more memory, but wouldn't block the main thread.

@basepi WDYT?
I guess I could go either way. I'm loathe to block the main thread -- I feel like I'd rather (briefly) lose a little memory during the operation. But I don't think either option is particularly bad.

Resolving patch:
diff --git a/elasticapm/metrics/base_metrics.py b/elasticapm/metrics/base_metrics.py
--- a/elasticapm/metrics/base_metrics.py
+++ b/elasticapm/metrics/base_metrics.py
@@ -192,7 +192,8 @@ def collect(self):
         timestamp = int(time.time() * 1000000)
         samples = defaultdict(dict)
         if self._counters:
-            for (name, labels), c in compat.iteritems(self._counters):
+            # iterate over a copy of the dict to avoid threading issues, see #717
+            for (name, labels), c in compat.iteritems(self._counters.copy()):
                 if c is not noop_metric:
                     val = c.val
                     if val or not c.reset_on_collect:
@@ -200,7 +201,7 @@ def collect(self):
                     if c.reset_on_collect:
                         c.reset()
         if self._gauges:
-            for (name, labels), g in compat.iteritems(self._gauges):
+            for (name, labels), g in compat.iteritems(self._gauges.copy()):
                 if g is not noop_metric:
                     val = g.val
                     if val or not g.reset_on_collect:
@@ -208,7 +209,7 @@ def collect(self):
                     if g.reset_on_collect:
                         g.reset()
         if self._timers:
-            for (name, labels), t in compat.iteritems(self._timers):
+            for (name, labels), t in compat.iteritems(self._timers.copy()):
                 if t is not noop_metric:
                     val, count = t.val
                     if val or not t.reset_on_collect:


Test patch: