internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: scoutapp/scout_apm_python

Issue:
Installation seems to be broken on python3.6.4
<img width="1125" alt="Screen Shot 2020-02-26 at 12 31 00 PM" src="https://user-images.githubusercontent.com/17484350/75380353-e2224900-58a4-11ea-96b3-2629b94c7107.png">



Discussion:
Hi @jackton1 !

Thanks for the report.

I think you're compiling with quite an old macOS / X Code / clang there. It does look however like we're using the non-standard C `__thread` keyword. Let me look into this and come back to you!

Thanks,

Adam
This seems to be linked to using 

`/Library/Frameworks/Python.framework/Versions/3.6/bin/python3.6` if that helps switch to `pyenv` which seems to work fine.
I've done a bit more research. It won't help to switch pyenv. The best thing to do is upgrade your macOS, or if you can't do that, at least your C compiler `clang`. It turns out `__thread` *is* standard.

I'm also going to look into a way for Scout to install itself without the C extension if it won't compile. But I can't say when that will be done.

Thanks,

Adam
Already tested  using pyenv's `python 3.6.4` and didn't have any issues.

Upgrading Mac didn't seem to help it was still pointing to the same path.

```
$ sw_vers 
ProductName:    Mac OS X
ProductVersion: 10.15.3
BuildVersion:   19D76

```
How did you install Python 3.6.4 previously?

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -23,7 +23,9 @@
 if compile_extensions:
     ext_modules = [
         Extension(
-            str("scout_apm.core._objtrace"), [str("src/scout_apm/core/_objtrace.c")]
+            name=str("scout_apm.core._objtrace"),
+            sources=[str("src/scout_apm/core/_objtrace.c")],
+            optional=True,
         )
     ]
 else:


Test patch:
Repository: scoutapp/scout_apm_python

Issue:
Ensure cutoff date is updated
figure out when the cutoff date for ambiguous timestamps needs updating and either calculate it dynamically or add a unit test that fails when it needs adjusting.

https://github.com/scoutapp/scout_apm_python/blob/cf2246e6ff0dc1b69ffff25e10cd83782895ee27/src/scout_apm/core/web_requests.py#L149-L173


Discussion:

Resolving patch:
diff --git a/src/scout_apm/core/web_requests.py b/src/scout_apm/core/web_requests.py
--- a/src/scout_apm/core/web_requests.py
+++ b/src/scout_apm/core/web_requests.py
@@ -1,6 +1,7 @@
 # coding=utf-8
 from __future__ import absolute_import, division, print_function, unicode_literals
 
+import datetime as dt
 import time
 
 from scout_apm.compat import datetime_to_timestamp, parse_qsl, urlencode
@@ -146,9 +147,8 @@ def track_amazon_request_queue_time(header_value, tracked_request):
     return True
 
 
-# Cutoff epoch is used for determining ambiguous timestamp boundaries, and is
-# just over 10 years ago at time of writing
-CUTOFF_EPOCH_S = time.mktime((2009, 6, 1, 0, 0, 0, 0, 0, 0))
+# Cutoff epoch is used for determining ambiguous timestamp boundaries
+CUTOFF_EPOCH_S = time.mktime((dt.date.today().year - 10, 1, 1, 0, 0, 0, 0, 0, 0))
 CUTOFF_EPOCH_MS = CUTOFF_EPOCH_S * 1000.0
 CUTOFF_EPOCH_US = CUTOFF_EPOCH_S * 1000000.0
 CUTOFF_EPOCH_NS = CUTOFF_EPOCH_S * 1000000000.0


Test patch:
Repository: scoutapp/scout_apm_python

Issue:
Remove SCOUT_LOG_LEVEL deprecation warning
The Heroku addon sets this environment variable automatically, and it can't vary based on language, so we shouldn't emit a deprecation warning (on Heroku only?) there since there's nothing users can do about it.


Discussion:

Resolving patch:
diff --git a/src/scout_apm/core/core_agent_manager.py b/src/scout_apm/core/core_agent_manager.py
--- a/src/scout_apm/core/core_agent_manager.py
+++ b/src/scout_apm/core/core_agent_manager.py
@@ -8,7 +8,6 @@
 import subprocess
 import tarfile
 import time
-import warnings
 
 from urllib3.exceptions import HTTPError
 
@@ -89,15 +88,7 @@ def socket_path(self):
     def log_level(self):
         # Old deprecated name "log_level"
         log_level = scout_config.value("log_level")
-        if log_level is not None:
-            warnings.warn(
-                "The config name 'log_level' is deprecated - "
-                + "please use the new name 'core_agent_log_level' instead. "
-                + "This might be configured in your environment variables or "
-                + "framework settings as SCOUT_LOG_LEVEL.",
-                DeprecationWarning,
-            )
-        else:
+        if log_level is None:
             log_level = scout_config.value("core_agent_log_level")
         return ["--log-level", log_level]
 


Test patch:
diff --git a/tests/integration/core/test_core_agent_manager.py b/tests/integration/core/test_core_agent_manager.py
--- a/tests/integration/core/test_core_agent_manager.py
+++ b/tests/integration/core/test_core_agent_manager.py
@@ -100,25 +100,17 @@ def test_launch_error(caplog, core_agent_manager):
     assert caplog.records[0].exc_info[1] is exception
 
 
-def test_log_level(caplog, core_agent_manager):
+def test_log_level(core_agent_manager):
     scout_config.set(core_agent_log_level="foo")
 
     result = core_agent_manager.log_level()
 
     assert result == ["--log-level", "foo"]
-    assert caplog.record_tuples == []
 
 
-def test_log_level_deprecated(core_agent_manager, recwarn):
+def test_log_level_old_name_takes_precedence(core_agent_manager):
     scout_config.set(log_level="foo", core_agent_log_level="bar")
 
     result = core_agent_manager.log_level()
 
     assert result == ["--log-level", "foo"]
-    assert len(recwarn) == 1
-    warning = recwarn.pop(DeprecationWarning)
-    assert str(warning.message) == (
-        "The config name 'log_level' is deprecated - please use the new name "
-        + "'core_agent_log_level' instead. This might be configured in your "
-        + "environment variables or framework settings as SCOUT_LOG_LEVEL."
-    )
Repository: scoutapp/scout_apm_python

Issue:
Default app name
Use a default app name like "Python App" rather than the empty string, so if users forget to set it it still appears on the consle.


Discussion:

Resolving patch:
diff --git a/src/scout_apm/core/config.py b/src/scout_apm/core/config.py
--- a/src/scout_apm/core/config.py
+++ b/src/scout_apm/core/config.py
@@ -218,7 +218,7 @@ def __init__(self):
             "hostname": None,
             "key": "",
             "monitor": False,
-            "name": "",
+            "name": "Python App",
             "revision_sha": self._git_revision_sha(),
             "scm_subdirectory": "",
             "shutdown_timeout_seconds": 2.0,


Test patch:
Repository: scoutapp/scout_apm_python

Issue:
Support Django BASE_DIR being a pathlib.Path
As per [my blog post today](https://adamj.eu/tech/2020/03/16/use-pathlib-in-your-django-project/), this is changing in Django 3.1 but users may already be changing it now.


Discussion:

Resolving patch:
diff --git a/src/scout_apm/django/apps.py b/src/scout_apm/django/apps.py
--- a/src/scout_apm/django/apps.py
+++ b/src/scout_apm/django/apps.py
@@ -38,8 +38,10 @@ def update_scout_config_from_django_settings(self, **kwargs):
             self.on_setting_changed(name)
 
     def on_setting_changed(self, setting, **kwargs):
+        cast = None
         if setting == "BASE_DIR":
             scout_name = "application_root"
+            cast = str
         elif setting.startswith("SCOUT_"):
             scout_name = setting.replace("SCOUT_", "").lower()
         else:
@@ -51,6 +53,8 @@ def on_setting_changed(self, setting, **kwargs):
             # It was removed
             scout_config.unset(scout_name)
         else:
+            if cast is not None:
+                value = cast(value)
             scout_config.set(**{scout_name: value})
 
     def install_middleware(self):


Test patch:
diff --git a/tests/integration/test_django.py b/tests/integration/test_django.py
--- a/tests/integration/test_django.py
+++ b/tests/integration/test_django.py
@@ -40,6 +40,12 @@
     from django.core.urlresolvers import resolve
 
 
+if sys.version_info >= (3,):
+    from pathlib import Path
+else:
+    Path = None
+
+
 skip_unless_new_style_middleware = pytest.mark.skipif(
     django.VERSION < (1, 10), reason="new-style middleware was added in Django 1.10"
 )
@@ -112,6 +118,15 @@ def test_on_setting_changed_application_root():
     assert scout_config.value("application_root") == ""
 
 
+@skip_if_python_2
+def test_on_setting_changed_application_root_pathlib():
+    with app_with_scout(BASE_DIR=Path("/tmp/foobar")):
+        value = scout_config.value("application_root")
+        assert isinstance(value, str)
+        assert value == "/tmp/foobar"
+    assert scout_config.value("application_root") == ""
+
+
 def test_on_setting_changed_monitor():
     with app_with_scout(SCOUT_MONITOR=True):
         assert scout_config.value("monitor") is True
Repository: scoutapp/scout_apm_python

Issue:
Django log levels are ignored by core agent
No matter what I did to the Django log levels, I kept getting this message when running Django management tasks:

```
[2020-04-02T09:36:06][core_agent][INFO] Initializing logger with log level: Info
```

This is super annoying, because this causes programs in my crontab to start sending out lots of e-mails because previously quiet tasks would now start producing output.

After a lot of trial and error, I figured out that you can set `SCOUT_LOG_LEVEL` in Django settings to suppress this message.

This really should be fixed so the agent also honors the Django logging settings, or at least documented under the logging section in the Python documentation. It's really non-obvious that this the agent is a separate thing which ignores the Django logging settings.


Discussion:
Hi @sjamaan 

Sorry about this. The core agent shouldn't ened to restart with every program in your crontab.

The variable to set is `SCOUT_CORE_AGENT_LOG_LEVEL` - `SCOUT_LOG_LEVEL` is the deprecated alias (it invites further confusion with Python log levels). See https://docs.scoutapm.com/#python-configuration .

Agree it would be good to have the core agent respect log silencing, or not forward its output to the Django process. I'll chat with the core agent developers and investigate this to see what we can do.

Thanks,

Adam

Resolving patch:
diff --git a/src/scout_apm/core/core_agent_manager.py b/src/scout_apm/core/core_agent_manager.py
--- a/src/scout_apm/core/core_agent_manager.py
+++ b/src/scout_apm/core/core_agent_manager.py
@@ -58,17 +58,19 @@ def download(self):
 
     def run(self):
         try:
-            subprocess.check_call(
-                (
-                    self.agent_binary()
-                    + self.daemonize_flag()
-                    + self.log_level()
-                    + self.log_file()
-                    + self.config_file()
-                    + self.socket_path()
-                ),
-                close_fds=True,
-            )
+            with open(os.devnull) as devnull:
+                subprocess.check_call(
+                    (
+                        self.agent_binary()
+                        + self.daemonize_flag()
+                        + self.log_level()
+                        + self.log_file()
+                        + self.config_file()
+                        + self.socket_path()
+                    ),
+                    close_fds=True,
+                    stdout=devnull,
+                )
         except Exception:
             # TODO detect failure of launch properly
             logger.exception("Error running Core Agent")


Test patch:
Repository: scoutapp/scout_apm_python

Issue:
Core agent update to 1.2.9
We've had a few bug fixes and improvements with the CA. Please bump to 1.2.9 in python CA manager.


Discussion:

Resolving patch:
diff --git a/src/scout_apm/core/config.py b/src/scout_apm/core/config.py
--- a/src/scout_apm/core/config.py
+++ b/src/scout_apm/core/config.py
@@ -213,7 +213,7 @@ def __init__(self):
             "core_agent_launch": True,
             "core_agent_log_level": "info",
             "core_agent_permissions": 700,
-            "core_agent_version": "v1.2.6",  # can be an exact tag name, or 'latest'
+            "core_agent_version": "v1.2.9",  # can be an exact tag name, or 'latest'
             "disabled_instruments": [],
             "download_url": "https://s3-us-west-1.amazonaws.com/scout-public-downloads/apm_core_agent/release",  # noqa: E501
             "framework": "",


Test patch:
Repository: scoutapp/scout_apm_python

Issue:
Add RQ subclass of HerokuWorker
The "Using RQ on Heroku" docs section ( https://python-rq.org/patterns/ ) shows using a subclass of `Worker` specialized for Heroku. Unfortunateely using that, rather than the Scout RQ Worker subclass means that scout isn't instrumented. We should also provide a `ScoutHerokuWorker` class.


Discussion:

Resolving patch:
diff --git a/src/scout_apm/rq.py b/src/scout_apm/rq.py
--- a/src/scout_apm/rq.py
+++ b/src/scout_apm/rq.py
@@ -7,6 +7,7 @@
 from rq import SimpleWorker as RqSimpleWorker
 from rq import Worker as RqWorker
 from rq.job import Job
+from rq.worker import HerokuWorker as RqHerokuWorker
 
 import scout_apm.core
 from scout_apm.core.tracked_request import TrackedRequest
@@ -40,6 +41,10 @@ class SimpleWorker(WorkerMixin, RqSimpleWorker):
     pass
 
 
+class HerokuWorker(WorkerMixin, RqHerokuWorker):
+    pass
+
+
 job_instrumented = False
 
 


Test patch:
Repository: scoutapp/scout_apm_python

Issue:
Support Python 3.9
Python 3.9 will be released 2020-10-05.

Here are some steps before its release:

* Start testing with prerelease

After release:
* Ensure tests run with released version
* Add 3.9 PyPI classifier
* Enable PYthon wheel building in release


Discussion:

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -98,5 +98,6 @@
         "Programming Language :: Python :: 3.6",
         "Programming Language :: Python :: 3.7",
         "Programming Language :: Python :: 3.8",
+        "Programming Language :: Python :: 3.9",
     ],
 )


Test patch:
diff --git a/tests/compat.py b/tests/compat.py
--- a/tests/compat.py
+++ b/tests/compat.py
@@ -44,4 +44,17 @@ def nullcontext(obj):
         yield obj
 
 
-__all__ = ["mock", "nullcontext", "TemporaryDirectory"]
+if sys.version_info >= (3, 4):
+    from contextlib import suppress
+else:
+    from contextlib import contextmanager
+
+    @contextmanager
+    def suppress(*exceptions):
+        try:
+            yield
+        except exceptions:
+            pass
+
+
+__all__ = ["mock", "nullcontext", "suppress", "TemporaryDirectory"]
diff --git a/tests/integration/django_app.py b/tests/integration/django_app.py
--- a/tests/integration/django_app.py
+++ b/tests/integration/django_app.py
@@ -1,11 +1,15 @@
 # coding: utf-8
 from __future__ import absolute_import, division, print_function, unicode_literals
 
+import sys
+
 import django
 import wrapt
 from django.conf import settings
 from django.template.response import TemplateResponse
 
+from tests.compat import suppress
+
 config = {
     "ALLOWED_HOSTS": ["*"],
     "DATABASES": {
@@ -118,22 +122,22 @@ def sql_kwargs(request):
 
 def sql_type_errors(request):
     with connection.cursor() as cursor:
-        try:
+        with suppress(TypeError):
             cursor.execute()
-        except TypeError:
-            pass
-        try:
+
+        if sys.version_info >= (3, 9):
+            exc_type = TypeError
+        else:
+            exc_type = ValueError
+
+        with suppress(exc_type):
             cursor.execute(sql=None)
-        except ValueError:
-            pass
-        try:
+
+        with suppress(TypeError):
             cursor.executemany()
-        except TypeError:
-            pass
-        try:
+
+        with suppress(TypeError):
             cursor.executemany(sql=None, param_list=[(1,)])
-        except TypeError:
-            pass
     return HttpResponse("Done")
Repository: scoutapp/scout_apm_python

Issue:
Lower span limit to a debug level message
Currently we log these as a warn message, and can be quite loud for logging

https://github.com/scoutapp/scout_apm_python/blob/master/src/scout_apm/core/tracked_request.py#L92-L96


Discussion:

Resolving patch:
diff --git a/src/scout_apm/core/tracked_request.py b/src/scout_apm/core/tracked_request.py
--- a/src/scout_apm/core/tracked_request.py
+++ b/src/scout_apm/core/tracked_request.py
@@ -91,7 +91,7 @@ def start_span(
 
         if len(self.complete_spans) >= self.MAX_COMPLETE_SPANS:
             if not self.hit_max:
-                logger.warning(
+                logger.debug(
                     "Hit the maximum number of spans, this trace will be incomplete."
                 )
                 self.hit_max = True


Test patch:
diff --git a/tests/unit/core/test_tracked_request.py b/tests/unit/core/test_tracked_request.py
--- a/tests/unit/core/test_tracked_request.py
+++ b/tests/unit/core/test_tracked_request.py
@@ -133,7 +133,7 @@ def test_start_span_at_max_ignores_span(caplog, tracked_request):
     assert caplog.record_tuples == [
         (
             "scout_apm.core.tracked_request",
-            logging.WARNING,
+            logging.DEBUG,
             "Hit the maximum number of spans, this trace will be incomplete.",
         )
     ]
Repository: scoutapp/scout_apm_python

Issue:
Track when an exception occurs in a Celery task
Similar to how we do this in other libraries
`tracked_request.tag("error", "true")`



Discussion:
I'll need to check how [`task_failure` signal](https://docs.celeryproject.org/en/stable/userguide/signals.html?highlight=task_postrun#task-failure) completes in comparison to `task_postrun`. Though we might be able to key off of postrun's `state` parameter.

Resolving patch:
diff --git a/src/scout_apm/celery.py b/src/scout_apm/celery.py
--- a/src/scout_apm/celery.py
+++ b/src/scout_apm/celery.py
@@ -3,7 +3,7 @@
 
 import datetime as dt
 
-from celery.signals import before_task_publish, task_postrun, task_prerun
+from celery.signals import before_task_publish, task_failure, task_postrun, task_prerun
 
 import scout_apm.core
 from scout_apm.compat import datetime_to_timestamp
@@ -52,6 +52,11 @@ def task_postrun_callback(task=None, **kwargs):
     tracked_request.stop_span()
 
 
+def task_failure_callback(task_id=None, **kwargs):
+    tracked_request = TrackedRequest.instance()
+    tracked_request.tag("error", "true")
+
+
 def install(app=None):
     if app is not None:
         copy_configuration(app)
@@ -62,6 +67,7 @@ def install(app=None):
 
     before_task_publish.connect(before_task_publish_callback)
     task_prerun.connect(task_prerun_callback)
+    task_failure.connect(task_failure_callback)
     task_postrun.connect(task_postrun_callback)
 
 


Test patch:
diff --git a/tests/integration/test_celery.py b/tests/integration/test_celery.py
--- a/tests/integration/test_celery.py
+++ b/tests/integration/test_celery.py
@@ -50,6 +50,10 @@ def app_with_scout(celery_config=None, app=None, config=None):
     def hello():
         return "Hello World!"
 
+    @app.task
+    def crash():
+        raise ValueError("Boom!")
+
     # Setup according to https://docs.scoutapm.com/#celery
     Config.set(**config)
     scout_apm.celery.install(app)
@@ -92,6 +96,20 @@ def test_hello_eager(tracked_requests):
     assert span.operation == "Job/tests.integration.test_celery.hello"
 
 
+def test_error_task(tracked_requests):
+    with app_with_scout() as app:
+        result = app.tasks["tests.integration.test_celery.crash"].apply()
+
+    assert isinstance(result.result, ValueError)
+    assert len(tracked_requests) == 1
+    tracked_request = tracked_requests[0]
+    assert tracked_request.active_spans == []
+    assert len(tracked_request.complete_spans) == 1
+    span = tracked_request.complete_spans[0]
+    assert span.operation == "Job/tests.integration.test_celery.crash"
+    assert tracked_request.tags["error"]
+
+
 @skip_unless_celery_4_plus
 def test_hello_worker(celery_app, celery_worker, tracked_requests):
     with app_with_scout(app=celery_app) as app:
Repository: scoutapp/scout_apm_python

Issue:
Support ElasticSearch 7.14
The python package `elasticsearch-py` introduced the `terms_enum` parameter from ElasticSearch 7.14. This is currently not being instrumented and breaking tests.


Discussion:

Resolving patch:
diff --git a/src/scout_apm/instruments/elasticsearch.py b/src/scout_apm/instruments/elasticsearch.py
--- a/src/scout_apm/instruments/elasticsearch.py
+++ b/src/scout_apm/instruments/elasticsearch.py
@@ -71,6 +71,7 @@ def ensure_installed():
     ClientMethod("search_shards", True),
     ClientMethod("search_template", True),
     ClientMethod("termvectors", True),
+    ClientMethod("terms_enum", True),
     ClientMethod("update", True),
     ClientMethod("update_by_query", True),
     ClientMethod("update_by_query_rethrottle", False),


Test patch:
Repository: scoutapp/scout_apm_python

Issue:
Disconnect receiver from celery task_failure signal
Celery instrumentation's `uninstall` should call `task_failure.disconnect(task_failure_callback)`.


Discussion:

Resolving patch:
diff --git a/src/scout_apm/celery.py b/src/scout_apm/celery.py
--- a/src/scout_apm/celery.py
+++ b/src/scout_apm/celery.py
@@ -151,3 +151,4 @@ def uninstall():
     before_task_publish.disconnect(before_task_publish_callback)
     task_prerun.disconnect(task_prerun_callback)
     task_postrun.disconnect(task_postrun_callback)
+    task_failure.disconnect(task_failure_callback)


Test patch:
Repository: scoutapp/scout_apm_python

Issue:
Bump core agent version to 1.4.0
Please update the Python agent dependency of core agent to core agent v1.4.0.


Discussion:

Resolving patch:
diff --git a/src/scout_apm/core/config.py b/src/scout_apm/core/config.py
--- a/src/scout_apm/core/config.py
+++ b/src/scout_apm/core/config.py
@@ -220,7 +220,7 @@ def __init__(self):
             "core_agent_log_level": "info",
             "core_agent_permissions": 700,
             "core_agent_socket_path": "tcp://127.0.0.1:6590",
-            "core_agent_version": "v1.3.1",  # can be an exact tag name, or 'latest'
+            "core_agent_version": "v1.4.0",  # can be an exact tag name, or 'latest'
             "disabled_instruments": [],
             "download_url": "https://s3-us-west-1.amazonaws.com/scout-public-downloads/apm_core_agent/release",  # noqa: B950
             "errors_batch_size": 5,


Test patch: