Repository: scoutapp/scout_apm_python (python)

Issue:
Rename, document, and provide defaults for extra core agent configuration variables
It's possible to set these config vars to change the way the core agent is launched:

* `log_level`
* `log_file`
* `socket_path`
* `config_file`

These should be prefixed with `core_agent` , like `core_agent_log_level`, and be documented. Would also be useful for the [logging guide](https://docs.scoutapm.com/#python-logging) to point to setting `core_agent_log_file` to capture the logs - by default they go to stdout, which is nowhere, and it makes debugging harder.

Pull request:
Rename and alias some core agent config variables
Fixes #497.

Patch:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index b09c491..5c826a1 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -6,8 +6,13 @@
 
 - Added config value `shutdown_message_enabled` /
   `SCOUT_SHUTDOWN_MESSAGE_ENABLED` which can be set to "false" to disable the
-  shutdown message "Scout draining N events..."that was added in version 2.11.0
-  ([Issue #498](https://github.com/scoutapp/scout_apm_python/issues/498)).
+  shutdown message "Scout draining N events..."that was added in version
+  2.11.0.
+  ([Issue #498](https://github.com/scoutapp/scout_apm_python/issues/498))
+- Renamed `config_file`, `log_file`, and `socket_path` to have the
+  `core_agent_` prefix. This is to clarify that they apply only to the core
+  agent process. The old names continue to work as aliases.
+  ([Issue #497](https://github.com/scoutapp/scout_apm_python/issues/497))
 
 ## [2.12.0] 2020-03-03
 
@@ -15,56 +20,57 @@
 
 - Added ability to read configuration from your Celery app. This can help share
   configuration between Django and Celery. Pass your application as `app` to
-  have the configuration copied: `scout_apm.celery.install(app=app)`
-  ([Issue #477](https://github.com/scoutapp/scout_apm_python/issues/477)).
+  have the configuration copied: `scout_apm.celery.install(app=app)` .
+  ([Issue #477](https://github.com/scoutapp/scout_apm_python/issues/477))
 - Added maximum span limit of 1500. This prevents memory exhaustion when
-  tracing long running tasks as Scout no longer tries to record everything
-  ([Issue #472](https://github.com/scoutapp/scout_apm_python/issues/472)).
+  tracing long running tasks as Scout no longer tries to record everything.
+  ([Issue #472](https://github.com/scoutapp/scout_apm_python/issues/472))
 
 ### Fixed
 
 - Made installation of the objtrace C extension optional, so that if it fails
-  due to your C compiler being old, Scout can still install
-  ([Issue #488](https://github.com/scoutapp/scout_apm_python/issues/488)).
+  due to your C compiler being old, Scout can still install.
+  ([Issue #488](https://github.com/scoutapp/scout_apm_python/issues/488))
 - Removed warning for setting the `SCOUT_LOG_LEVEL` setting, since this is set
   by default by the Heroku add-on.
+  ([Issue #486](https://github.com/scoutapp/scout_apm_python/issues/486]))
 
 ## [2.11.0] 2020-02-17
 
 ### Added
 
 - Improve Falcon integration: tracing is automatically disabled if
-  installation fails, track middleware, and support class-based responders
+  installation fails, track middleware, and support class-based responders.
   ([PR #453](https://github.com/scoutapp/scout_apm_python/pull/453),
-  [PR #460](https://github.com/scoutapp/scout_apm_python/pull/460)).
-- Add Hug integration
-  ([PR #460](https://github.com/scoutapp/scout_apm_python/pull/460)).
-- Limit size of recorded tracebacks to reduce memory usage
-  ([PR #476](https://github.com/scoutapp/scout_apm_python/pull/476)).
+  [PR #460](https://github.com/scoutapp/scout_apm_python/pull/460))
+- Add Hug integration.
+  ([PR #460](https://github.com/scoutapp/scout_apm_python/pull/460))
+- Limit size of recorded tracebacks to reduce memory usage.
+  ([PR #476](https://github.com/scoutapp/scout_apm_python/pull/476))
 - Scout now flushes any queued commands to the core agent at shutdown, with a
   configurable timeout `shutdown_timeout_seconds`, defaulting to 2 seconds.
   This ensures that all traces are recorded, rather than lost, especially
   useful for single-run background commands.
-  ([Issue # 473](https://github.com/scoutapp/scout_apm_python/issues/473)).
+  ([Issue # 473](https://github.com/scoutapp/scout_apm_python/issues/473))
 - Move from **requests** to underlying **urllib3** for downloads to reduce
-  dependency footprint
-  ([PR #480](https://github.com/scoutapp/scout_apm_python/pull/480)).
+  dependency footprint.
+  ([PR #480](https://github.com/scoutapp/scout_apm_python/pull/480))
 - Retry downloading the core agent a few times, and use a timeout to prevent
-  startup hangs
-  ([PR #480](https://github.com/scoutapp/scout_apm_python/pull/480)).
+  startup hangs.
+  ([PR #480](https://github.com/scoutapp/scout_apm_python/pull/480))
 
 ### Fixed
 
 - Moved some user-facing warnings, such as for deprecated features, from
-  `logging` to `warnings`, so they can be caught during testing
-  ([PR #468](https://github.com/scoutapp/scout_apm_python/pull/468)).
-- Fix Jinja2 asynchronous rendering instrumentation for Jinja2 2.11.0+
-  ([PR #462](https://github.com/scoutapp/scout_apm_python/pull/462)).
+  `logging` to `warnings`, so they can be caught during testing.
+  ([PR #468](https://github.com/scoutapp/scout_apm_python/pull/468))
+- Fix Jinja2 asynchronous rendering instrumentation for Jinja2 2.11.0+.
+  ([PR #462](https://github.com/scoutapp/scout_apm_python/pull/462))
 - Stop patching `elasticsearch`'s `scripts_painless_context()`, because it has
-  been removed in version 7.5.1
-  ([Issue #454](https://github.com/scoutapp/scout_apm_python/issues/454)).
-- Fix for `elasticsearch` version 7.5.1's change of positional argument order
-  ([Issue #456](https://github.com/scoutapp/scout_apm_python/issues/456)).
+  been removed in version 7.5.1.
+  ([Issue #454](https://github.com/scoutapp/scout_apm_python/issues/454))
+- Fix for `elasticsearch` version 7.5.1's change of positional argument order.
+  ([Issue #456](https://github.com/scoutapp/scout_apm_python/issues/456))
 
 ## [2.10.0] 2020-01-02
 
@@ -74,60 +80,61 @@
   determine package versions, rather than `pkg_resources`. The [backport
   package
   `importlib_metadata`](https://pypi.org/project/importlib-metadata/) is
-  required on older Python versions
-  ([PR #430](https://github.com/scoutapp/scout_apm_python/pull/430)).
-- Add CherryPy integration
-  ([PR #431](https://github.com/scoutapp/scout_apm_python/pull/431)).
-- Add tracking for missing PyMongo Collection methods
-  ([PR #436](https://github.com/scoutapp/scout_apm_python/pull/436)).
-- Add tracking for missing Elasticsearch client methods
-  ([PR #438](https://github.com/scoutapp/scout_apm_python/pull/438)).
+  required on older Python versions.
+  ([PR #430](https://github.com/scoutapp/scout_apm_python/pull/430))
+- Add CherryPy integration.
+  ([PR #431](https://github.com/scoutapp/scout_apm_python/pull/431))
+- Add tracking for missing PyMongo Collection methods.
+  ([PR #436](https://github.com/scoutapp/scout_apm_python/pull/436))
+- Add tracking for missing Elasticsearch client methods.
+  ([PR #438](https://github.com/scoutapp/scout_apm_python/pull/438))
 
 ### Removed
 
 - Remove the "name" context tag from Flask requests - it only duplicated the
-  request name
-  ([PR #432](https://github.com/scoutapp/scout_apm_python/pull/432)).
+  request name.
+  ([PR #432](https://github.com/scoutapp/scout_apm_python/pull/432))
 
 ## [2.9.1] 2019-12-13
 
 ### Added
 
-- Update Core Agent to version 1.2.6, fixing several bugs
-  ([PR #415](https://github.com/scoutapp/scout_apm_python/pull/415)):
+- Update Core Agent to version 1.2.6, fixing several bugs:
     - Improve SQL statement names in Timeline Traces
     - Standardize on a `language_version` key in metadata
     - Autoclose spans that are still running when a Request completes
 
+  ([PR #415](https://github.com/scoutapp/scout_apm_python/pull/415))
+
 ### Fixed
 
-- Fix recording of path for unicode query parameters on Python 2.7
-  ([PR #419](https://github.com/scoutapp/scout_apm_python/pull/419)).
+- Fix recording of path for unicode query parameters on Python 2.7.
+  ([PR #419](https://github.com/scoutapp/scout_apm_python/pull/419))
 
 ## [2.9.0] 2019-11-30
 
 ### Added
 
-- Add RQ integration
-  ([PR #375](https://github.com/scoutapp/scout_apm_python/pull/375)).
-- Track 5XX responses as errors from all web frameworks
-  ([Issue #383](https://github.com/scoutapp/scout_apm_python/issues/383)).
-- Track Flask's request preprocessors as a span `PreprocessRequest`
-  ([PR #390](https://github.com/scoutapp/scout_apm_python/pull/390)).
-- Track Jinja2 asynchronous template rendering
-  ([PR #398](https://github.com/scoutapp/scout_apm_python/pull/398)).
-- Add Django-Huey integration
-  ([PR #401](https://github.com/scoutapp/scout_apm_python/pull/401)).
+- Add RQ integration.
+  ([PR #375](https://github.com/scoutapp/scout_apm_python/pull/375))
+- Track 5XX responses as errors from all web frameworks.
+  ([Issue #383](https://github.com/scoutapp/scout_apm_python/issues/383))
+- Track Flask's request preprocessors as a span `PreprocessRequest`.
+  ([PR #390](https://github.com/scoutapp/scout_apm_python/pull/390))
+- Track Jinja2 asynchronous template rendering.
+  ([PR #398](https://github.com/scoutapp/scout_apm_python/pull/398))
+- Add Django-Huey integration.
+  ([PR #401](https://github.com/scoutapp/scout_apm_python/pull/401))
 
 ### Fixed
 
-- Track better names for Django REST Framework `ViewSet`s
-  ([PR #380](https://github.com/scoutapp/scout_apm_python/pull/380)).
+- Track better names for Django REST Framework `ViewSet`s.
+  ([PR #380](https://github.com/scoutapp/scout_apm_python/pull/380))
 - Always capture SQL tracebacks for `executemany` queries going over the
-  built-in count threshold
-  ([PR #409](https://github.com/scoutapp/scout_apm_python/pull/409)).
-- Don't capture internal traceback function's frame in tracebacks
-  ([PR #410](https://github.com/scoutapp/scout_apm_python/pull/410)).
+  built-in count threshold.
+  ([PR #409](https://github.com/scoutapp/scout_apm_python/pull/409))
+- Don't capture internal traceback function's frame in tracebacks.
+  ([PR #410](https://github.com/scoutapp/scout_apm_python/pull/410))
 
 ## [2.8.1] 2019-11-19
 
@@ -141,46 +148,46 @@
 ### Added
 
 - Move all decorators to the [`wrapt`
-  library](https://wrapt.readthedocs.io/en/latest/) which is more transparent
-  ([PR #324](https://github.com/scoutapp/scout_apm_python/pull/324)).
-- Move context tracking to be async-compatble with `asgiref`'s `Local`
-  ([PR #349](https://github.com/scoutapp/scout_apm_python/pull/349)).
-- Add Starlette integration for Python 3.6+
-  ([PR #349](https://github.com/scoutapp/scout_apm_python/pull/349)).
-- Add Huey integration
-  ([PR #367](https://github.com/scoutapp/scout_apm_python/pull/367)).
-
-### Fixed
-
-- Track SQL Alchemy `executemany` calls as multi-queries
-  ([PR #340](https://github.com/scoutapp/scout_apm_python/pull/340)).
-- Track Elasticsearch index name when it's not passed as a keyword argument
-  ([PR #348](https://github.com/scoutapp/scout_apm_python/pull/348)).
-- Increase core agent socket timeout to reduce reconnections
-  ([PR #247](https://github.com/scoutapp/scout_apm_python/pull/247)).
+  library](https://wrapt.readthedocs.io/en/latest/) which is more transparent.
+  ([PR #324](https://github.com/scoutapp/scout_apm_python/pull/324))
+- Move context tracking to be async-compatble with `asgiref`'s `Local`.
+  ([PR #349](https://github.com/scoutapp/scout_apm_python/pull/349))
+- Add Starlette integration for Python 3.6+.
+  ([PR #349](https://github.com/scoutapp/scout_apm_python/pull/349))
+- Add Huey integration.
+  ([PR #367](https://github.com/scoutapp/scout_apm_python/pull/367))
+
+### Fixed
+
+- Track SQL Alchemy `executemany` calls as multi-queries.
+  ([PR #340](https://github.com/scoutapp/scout_apm_python/pull/340))
+- Track Elasticsearch index name when it's not passed as a keyword argument.
+  ([PR #348](https://github.com/scoutapp/scout_apm_python/pull/348))
+- Increase core agent socket timeout to reduce reconnections.
+  ([PR #247](https://github.com/scoutapp/scout_apm_python/pull/247))
 - Fix the `RemovedInDjango20Warning` for `django.core.urlresolvers` on Django
-  1.11 ([PR #359](https://github.com/scoutapp/scout_apm_python/pull/359)).
-- Correctly disable Bottle if the agent fails to launch
-  ([PR #364](https://github.com/scoutapp/scout_apm_python/pull/364)).
+  1.11. ([PR #359](https://github.com/scoutapp/scout_apm_python/pull/359))
+- Correctly disable Bottle if the agent fails to launch.
+  ([PR #364](https://github.com/scoutapp/scout_apm_python/pull/364))
 
 ## [2.7.0] 2019-11-03
 
 ### Added
 
 - Python 3.8 testing and PyPI trove classifier - no code changes were required
-  so older versions should work too
-  ([PR #263](https://github.com/scoutapp/scout_apm_python/pull/263)).
+  so older versions should work too.
+  ([PR #263](https://github.com/scoutapp/scout_apm_python/pull/263))
 - Capture better operation names for Django Tastypie resources, e.g.
-  `myapp.api.UserResource.get_list`
-  ([PR #332](https://github.com/scoutapp/scout_apm_python/pull/332)).
+  `myapp.api.UserResource.get_list`.
+  ([PR #332](https://github.com/scoutapp/scout_apm_python/pull/332))
 
 ### Fixed
 
 - Change `scout_apm.api.install()` signature to not take `*args, **kwargs` but
-  just `config` as a keyword argument
-  ([PR #304](https://github.com/scoutapp/scout_apm_python/pull/304)).
-- Rewrite background samplers thread to avoid some rare race conditions
-  ([PR #307](https://github.com/scoutapp/scout_apm_python/pull/307)).
+  just `config` as a keyword argument.
+  ([PR #304](https://github.com/scoutapp/scout_apm_python/pull/304))
+- Rewrite background samplers thread to avoid some rare race conditions.
+  ([PR #307](https://github.com/scoutapp/scout_apm_python/pull/307))
 
 ## [2.6.1] 2019-10-23
 
diff --git a/src/scout_apm/core/config.py b/src/scout_apm/core/config.py
index 31f140f..cb92662 100644
--- a/src/scout_apm/core/config.py
+++ b/src/scout_apm/core/config.py
@@ -55,11 +55,14 @@ class ScoutConfig(object):
         return [
             "app_server",
             "application_root",
+            "core_agent_config_file",
             "core_agent_dir",
             "core_agent_download",
             "core_agent_launch",
+            "core_agent_log_file",
             "core_agent_log_level",
             "core_agent_permissions",
+            "core_agent_socket_path",
             "core_agent_version",
             "disabled_instruments",
             "download_url",
@@ -75,7 +78,6 @@ class ScoutConfig(object):
             "scm_subdirectory",
             "shutdown_message_enabled",
             "shutdown_timeout_seconds",
-            "socket_path",
         ]
 
     def core_agent_permissions(self):
@@ -177,7 +179,7 @@ class Derived(object):
         func_name = "derive_" + key
         return getattr(self, func_name, None)
 
-    def derive_socket_path(self):
+    def derive_core_agent_socket_path(self):
         return "{}/{}/scout-agent.sock".format(
             self.config.value("core_agent_dir"),
             self.config.value("core_agent_full_name"),
diff --git a/src/scout_apm/core/core_agent_manager.py b/src/scout_apm/core/core_agent_manager.py
index 56f2c5d..51ef5d2 100644
--- a/src/scout_apm/core/core_agent_manager.py
+++ b/src/scout_apm/core/core_agent_manager.py
@@ -82,7 +82,10 @@ class CoreAgentManager(object):
         return ["--daemonize", "true"]
 
     def socket_path(self):
+        # Old deprecated name "socket_path"
         socket_path = scout_config.value("socket_path")
+        if socket_path is None:
+            socket_path = scout_config.value("core_agent_socket_path")
         return ["--socket", socket_path]
 
     def log_level(self):
@@ -93,14 +96,22 @@ class CoreAgentManager(object):
         return ["--log-level", log_level]
 
     def log_file(self):
+        # Old deprecated name "log_file"
         path = scout_config.value("log_file")
+        if path is None:
+            path = scout_config.value("core_agent_log_file")
+
         if path is not None:
             return ["--log-file", path]
         else:
             return []
 
     def config_file(self):
+        # Old deprecated name "config_file"
         path = scout_config.value("config_file")
+        if path is None:
+            path = scout_config.value("core_agent_config_file")
+
         if path is not None:
             return ["--config-file", path]
         else:
diff --git a/src/scout_apm/core/socket.py b/src/scout_apm/core/socket.py
index 1e4c2a3..0f6a754 100644
--- a/src/scout_apm/core/socket.py
+++ b/src/scout_apm/core/socket.py
@@ -65,7 +65,11 @@ class CoreAgentSocketThread(SingletonThread):
         return queue_empty
 
     def run(self):
-        self.socket_path = scout_config.value("socket_path")
+        # Old deprecated name "socket_path"
+        socket_path = scout_config.value("socket_path")
+        if socket_path is None:
+            socket_path = scout_config.value("core_agent_socket_path")
+        self.socket_path = socket_path
         self.socket = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
 
         try:


Test patch:
diff --git a/tests/conftest.py b/tests/conftest.py
index 3b5f1e7..bb7f2d3 100644
--- a/tests/conftest.py
+++ b/tests/conftest.py
@@ -120,7 +120,7 @@ def core_agent_manager(core_agent_dir):
     #   Error opening listener on socket: Custom { kind: InvalidInput,
     #   error: StringError("path must be shorter than SUN_LEN") }
     socket_path = "{}/test.sock".format(core_agent_dir)
-    scout_config.set(core_agent_dir=core_agent_dir, socket_path=socket_path)
+    scout_config.set(core_agent_dir=core_agent_dir, core_agent_socket_path=socket_path)
     core_agent_manager = CoreAgentManager()
     try:
         yield core_agent_manager
diff --git a/tests/integration/core/test_core_agent_manager.py b/tests/integration/core/test_core_agent_manager.py
index 4f4772d..34b663a 100644
--- a/tests/integration/core/test_core_agent_manager.py
+++ b/tests/integration/core/test_core_agent_manager.py
@@ -100,6 +100,22 @@ def test_launch_error(caplog, core_agent_manager):
     assert caplog.records[0].exc_info[1] is exception
 
 
+def test_socket_path(core_agent_manager):
+    scout_config.set(core_agent_socket_path="foo")
+
+    result = core_agent_manager.socket_path()
+
+    assert result == ["--socket", "foo"]
+
+
+def test_socket_path_old_name_takes_precedence(core_agent_manager):
+    scout_config.set(socket_path="foo", core_agent_socket_path="bar")
+
+    result = core_agent_manager.socket_path()
+
+    assert result == ["--socket", "foo"]
+
+
 def test_log_level(core_agent_manager):
     scout_config.set(core_agent_log_level="foo")
 
@@ -114,3 +130,35 @@ def test_log_level_old_name_takes_precedence(core_agent_manager):
     result = core_agent_manager.log_level()
 
     assert result == ["--log-level", "foo"]
+
+
+def test_log_file(core_agent_manager):
+    scout_config.set(core_agent_log_file="foo")
+
+    result = core_agent_manager.log_file()
+
+    assert result == ["--log-file", "foo"]
+
+
+def test_log_file_old_name_takes_precedence(core_agent_manager):
+    scout_config.set(log_file="foo", core_agent_log_file="bar")
+
+    result = core_agent_manager.log_file()
+
+    assert result == ["--log-file", "foo"]
+
+
+def test_config_file(core_agent_manager):
+    scout_config.set(core_agent_config_file="foo")
+
+    result = core_agent_manager.config_file()
+
+    assert result == ["--config-file", "foo"]
+
+
+def test_config_file_old_name_takes_precedence(core_agent_manager):
+    scout_config.set(config_file="foo", core_agent_config_file="bar")
+
+    result = core_agent_manager.config_file()
+
+    assert result == ["--config-file", "foo"]
diff --git a/tests/unit/core/test_config.py b/tests/unit/core/test_config.py
index 68a59ad..b8a1e0d 100644
--- a/tests/unit/core/test_config.py
+++ b/tests/unit/core/test_config.py
@@ -45,7 +45,7 @@ def test_get_derived_config_value():
             r"-(x86_64|i686|unknown)"
             r"-(unknown-linux-gnu|apple-darwin|unknown)"
             r"/scout-agent\.sock",
-            config.value("socket_path"),
+            config.value("core_agent_socket_path"),
         )
     finally:
         ScoutConfig.reset_all()