Repository: ansible/ansible
Pull request: remote log info
fixes #81710

Includes the value of the `ansible_target_log_info` var in all module invocations (also globally settable on the control node via the `ANSIBLE_TARGET_LOG_INFO` envvar). Python and Powershell Ansible modules will include this value in  syslog/event log messages they write.

An AWX/Controller job template should be able to set this envvar to the current job ID to have it reflected in the target host's log messages.

##### ISSUE TYPE

- Feature Pull Request


Problem statement:
Configurable sampling/transfer of control-side task context metadata to targets
### Summary

We're often asked how to include arbitrary control-side contextual metadata with task invocations, and to include that metadata in target-side task log messages. e.g.: sending an AWX/Controller Job ID to the target hosts on each module invocation that occurred from that job, and logging it in the module-generated syslog/Windows Application Event Log entries for future correlation with the owning job.

I've not seen any consensus on precisely *which* data to include; one person's "critical forensic correlation data" is another's "unacceptable disclosure of sensitive execution detail". Seems like we'd need a generic facility to specify environment vars and/or hostvars to sample on the control host to be included with task invocations (under a reserved dictionary arg), and adjust the module logging APIs to include them.

My initial thought is to define a new core config element (defaulting to none) that allows the user to define a templated expression that would be rendered as part of each task's templating under a host context. The rendered result would be sent to modules as a new reserved internal module var. The module logging APIs would then include this value verbatim, when present. Other module code would also have access to the value, which could be used for anything. The new config would be settable either via ansible.cfg or an envvar, making it easier for AWX/Controller to later provide a mechanism to configure it for jobs using core versions that support it, while older versions would just silently ignore it. 

Maybe something like:

```
ANSIBLE_ADDITIONAL_TASK_CONTEXT='{{awx_job_id}}'
```

When this config is non-empty, the defined template would be rendered for each task/host invocation, and its result included in a new `_ansible_additional_task_context` reserved module var. The resulting value, as with any Ansible template expression, could be of arbitrary complexity (eg, returning a data structure instead of just a scalar).  The module logging APIs would include the serialized value verbatim in log messages when it is present, eg "ansible_additional_task_context=(whatever the value was)".


### Issue Type

Feature Idea

### Component Name

module invocation and logging

### Additional Information

<!--- Paste example playbooks or commands between quotes below -->
```yaml (paste below)

```


### Code of Conduct

- [X] I agree to follow the Ansible Code of Conduct

Hints from issue discussion:
Files identified in the description:

None

If these files are incorrect, please update the `component name` section of the description or use the [component](https://github.com/ansible/ansibotmini#ansibotmini) bot command.

<!--- boilerplate: components_banner --->

~see #55623~ wrong one .. had same with logging tty but it seems it never got to PR

Commit message: fixed due to fb

Patch to review:
diff --git a/changelogs/fragments/log_id.yml b/changelogs/fragments/log_id.yml
new file mode 100644
index 00000000000000..3233bf796c48ee
--- /dev/null
+++ b/changelogs/fragments/log_id.yml
@@ -0,0 +1,2 @@
+minor_changes:
+  - added configuration item ``TARGET_LOG_INFO`` that allows the user/author to add an information string to the log output on targets.
diff --git a/lib/ansible/config/base.yml b/lib/ansible/config/base.yml
index 057f66430cc389..c9543b6649c0c4 100644
--- a/lib/ansible/config/base.yml
+++ b/lib/ansible/config/base.yml
@@ -1957,6 +1957,14 @@ TAGS_SKIP:
   ini:
   - {key: skip, section: tags}
   version_added: "2.5"
+TARGET_LOG_INFO:
+  name: Target log info
+  description: A string to insert into target logging for tracking purposes
+  env: [{name: ANSIBLE_TARGET_LOG_INFO}]
+  ini:
+  - {key: target_log_info, section: defaults}
+  vars:
+  - name: ansible_target_log_info
 TASK_TIMEOUT:
   name: Task Timeout
   default: 0
diff --git a/lib/ansible/module_utils/basic.py b/lib/ansible/module_utils/basic.py
index 9cfe32e56f01c3..2cfd646d385da1 100644
--- a/lib/ansible/module_utils/basic.py
+++ b/lib/ansible/module_utils/basic.py
@@ -1300,10 +1300,16 @@ def log(self, msg, log_args=None):
             # We want journal to always take text type
             # syslog takes bytes on py2, text type on py3
             if isinstance(msg, binary_type):
-                journal_msg = remove_values(msg.decode('utf-8', 'replace'), self.no_log_values)
+                journal_msg = msg.decode('utf-8', 'replace')
             else:
                 # TODO: surrogateescape is a danger here on Py3
-                journal_msg = remove_values(msg, self.no_log_values)
+                journal_msg = msg
+
+            if self._target_log_info:
+                journal_msg = ' '.join([self._target_log_info, journal_msg])
+
+            # ensure we clean up secrets!
+            journal_msg = remove_values(journal_msg, self.no_log_values)
 
             if PY3:
                 syslog_msg = journal_msg
diff --git a/lib/ansible/module_utils/common/parameters.py b/lib/ansible/module_utils/common/parameters.py
index fb61bd38c49b2b..9e74f59f26d501 100644
--- a/lib/ansible/module_utils/common/parameters.py
+++ b/lib/ansible/module_utils/common/parameters.py
@@ -82,6 +82,7 @@
 
 # if adding boolean attribute, also add to PASS_BOOL
 # some of this dupes defaults from controller config
+# keep in sync with copy in lib/ansible/module_utils/csharp/Ansible.Basic.cs
 PASS_VARS = {
     'check_mode': ('check_mode', False),
     'debug': ('_debug', False),
@@ -91,6 +92,7 @@
     'module_name': ('_name', None),
     'no_log': ('no_log', False),
     'remote_tmp': ('_remote_tmp', None),
+    'target_log_info': ('_target_log_info', None),
     'selinux_special_fs': ('_selinux_special_fs', ['fuse', 'nfs', 'vboxsf', 'ramfs', '9p', 'vfat']),
     'shell_executable': ('_shell', '/bin/sh'),
     'socket': ('_socket_path', None),
diff --git a/lib/ansible/module_utils/csharp/Ansible.Basic.cs b/lib/ansible/module_utils/csharp/Ansible.Basic.cs
index 120536331bc762..028c24530caadc 100644
--- a/lib/ansible/module_utils/csharp/Ansible.Basic.cs
+++ b/lib/ansible/module_utils/csharp/Ansible.Basic.cs
@@ -61,6 +61,7 @@ public class AnsibleModule
         private Dictionary<string, string> passVars = new Dictionary<string, string>()
         {
             // null values means no mapping, not used in Ansible.Basic.AnsibleModule
+            // keep in sync with python counterpart in lib/ansible/module_utils/common/parameters.py
             { "check_mode", "CheckMode" },
             { "debug", "DebugMode" },
             { "diff", "DiffMode" },
@@ -74,6 +75,7 @@ public class AnsibleModule
             { "socket", null },
             { "string_conversion_action", null },
             { "syslog_facility", null },
+            { "target_log_info", "TargetLogInfo"},
             { "tmpdir", "tmpdir" },
             { "verbosity", "Verbosity" },
             { "version", "AnsibleVersion" },
@@ -127,6 +129,7 @@ public class AnsibleModule
         public bool KeepRemoteFiles { get; private set; }
         public string ModuleName { get; private set; }
         public bool NoLog { get; private set; }
+        public string TargetLogInfo { get; private set; }
         public int Verbosity { get; private set; }
         public string AnsibleVersion { get; private set; }
 
@@ -259,6 +262,7 @@ public AnsibleModule(string[] args, IDictionary argumentSpec, IDictionary[] frag
             DiffMode = false;
             KeepRemoteFiles = false;
             ModuleName = "undefined win module";
+            TargetLogInfo = "";
             NoLog = (bool)argumentSpec["no_log"];
             Verbosity = 0;
             AppDomain.CurrentDomain.ProcessExit += CleanupFiles;
@@ -375,8 +379,18 @@ public void LogEvent(string message, EventLogEntryType logEntryType = EventLogEn
                 }
             }
             if (sanitise)
+            {
                 message = (string)RemoveNoLogValues(message, noLogValues);
-            message = String.Format("{0} - {1}", ModuleName, message);
+            }
+
+            if (String.IsNullOrWhiteSpace(TargetLogInfo))
+            {
+                message = String.Format("{0} - {1}", ModuleName, message);
+            }
+            else
+            {
+                message = String.Format("{0} {1} - {2}", ModuleName, TargetLogInfo, message);
+            }
 
             using (EventLog eventLog = new EventLog("Application"))
             {
diff --git a/lib/ansible/plugins/action/__init__.py b/lib/ansible/plugins/action/__init__.py
index b557e3b68ccb3c..7894c7f993c3bd 100644
--- a/lib/ansible/plugins/action/__init__.py
+++ b/lib/ansible/plugins/action/__init__.py
@@ -1000,6 +1000,9 @@ def _update_module_args(self, module_name, module_args, task_vars, ignore_unknow
         # tells the module to ignore options that are not in its argspec.
         module_args['_ansible_ignore_unknown_opts'] = ignore_unknown_opts
 
+        # allow user to insert string to add context to remote loggging
+        module_args['_ansible_target_log_info'] = C.config.get_config_value('TARGET_LOG_INFO', variables=task_vars)
+
     def _execute_module(self, module_name=None, module_args=None, tmp=None, task_vars=None, persist_files=False, delete_remote_tmp=None, wrap_async=False,
                         ignore_unknown_opts: bool = False):
         '''
diff --git a/test/integration/targets/module_utils_Ansible.Basic/library/ansible_basic_tests.ps1 b/test/integration/targets/module_utils_Ansible.Basic/library/ansible_basic_tests.ps1
index df8bf8db7944a5..9532d8463c6daa 100644
--- a/test/integration/targets/module_utils_Ansible.Basic/library/ansible_basic_tests.ps1
+++ b/test/integration/targets/module_utils_Ansible.Basic/library/ansible_basic_tests.ps1
@@ -155,6 +155,7 @@ $tests = @{
         "_ansible_shell_executable": "ignored",
         "_ansible_socket": "ignored",
         "_ansible_syslog_facility": "ignored",
+		"_ansible_target_log_info": "ignored",
         "_ansible_tmpdir": "$($m_tmpdir -replace "\\", "\\")",
         "_ansible_verbosity": 3,
         "_ansible_version": "2.8.0"


Code review comments:
diff_hunk: @@ -375,8 +379,18 @@ public void LogEvent(string message, EventLogEntryType logEntryType = EventLogEn
                 }
             }
             if (sanitise)
+            {
                 message = (string)RemoveNoLogValues(message, noLogValues);
-            message = String.Format("{0} - {1}", ModuleName, message);
+            }
+
+            if (String.IsNullOrWhiteSpace(TargetLogInfo))
+            {
+                message = String.Format("{0} - {1}", ModuleName, message);
+            }
+            else
+            {
+                message = String.Format("{0} {1} - {2}", ModuleName, TargetLogInfo, message);
path: lib/ansible/module_utils/csharp/Ansible.Basic.cs
text: @user1:
Missed this last time, but on the Python version, we're masking *after*  the TARGET_LOG_INFO stuff has been spliced in, so for parity, the RemoveNoLogValues check/call should move below this.