Repository: DataDog/dd-agent

Issue:
Agent startup broadcasts 'emergency' message on RPM-based system
Upon install/restart, a message is broadcast to syslog:

``` console
[root@localhost log]# service datadog-agent restart
Stopping Datadog Agent (using killproc on supervisord):    [  OK  ]
Starting Datadog Agent (using supervisord):
Message from syslogd@localhost at Jan 22 19:52:02 ...
 �<30>dd.collector[7539]: INFO (collector.py:387): Hostnames: {'socket-hostname': 'localhost.localdomain', 'hostname': 'mike-box', 'socket-fqdn': 'localhost.localdomain'}, tags: {}
                                                           [  OK  ]
[root@localhost log]# cat /etc/redhat-release
CentOS release 6.4 (Final)
```

Stock rsyslog.conf - nothing fancy, rsyslog 5.8.10-6.el6

datadog-agent version 4.0.1-218
Python 2.6.6 (r266:84292, Feb 22 2013, 00:00:18)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-3)] on linux2

syslog:

``` log
Jan 22 19:52:01 localhost dd.pup[7540]: INFO (pup.py:305): Starting pup
Jan 22 19:52:01 localhost dd.dogstatsd[7537]: INFO (dogstatsd.py:82): Reporting to localhost:17123 every 10s
Jan 22 19:52:01 localhost dd.dogstatsd[7537]: INFO (dogstatsd.py:237): Listening on host & port: ('127.0.0.1', 8125)
Jan 22 19:52:01 localhost dd.forwarder[7538]: INFO (ddagent.py:107): Done with custom emitters
Jan 22 19:52:01 localhost dd.forwarder[7538]: WARNING (ddagent.py:154): You are a Datadog user so we will send data to https://app.datadoghq.com
Jan 22 19:52:01 localhost dd.forwarder[7538]: INFO (ddagent.py:394): Listening on port 17123
Jan 22 19:52:01 localhost dd.collector[7539]: INFO (agent.py:211): Agent version 4.0.1
Jan 22 19:52:01 localhost dd.collector[7539]: INFO (agent.py:232): Running in foreground
Jan 22 19:52:01 localhost dd.collector[7539]: INFO (migration.py:365): Running migration script
Jan 22 19:52:01 localhost dd.collector[7539]: INFO (config.py:757): initialized checks.d checks: ['network']
Jan 22 19:52:01 localhost dd.collector[7539]: INFO (config.py:758): initialization failed checks.d checks: []
Jan 22 19:52:01 localhost dd.collector[7539]: INFO (datadog.py:70): Dogstream parsers: []
Jan 22 19:52:02 localhost �<30>dd.collector[7539]: INFO (collector.py:387): Hostnames: {'socket-hostname': 'localhost.localdomain', 'hostname': 'mike-box', 'socket-fqdn': 'localhost.localdomain'}, tags: {}
Jan 22 19:52:06 localhost dd.collector[7539]: INFO (collector.py:251): Running check network
```

Log is INFO level, why is it being broadcast as emergency?
One customer has mentioned that this message is misleading their users.

Log line is driven from https://github.com/DataDog/dd-agent/blob/e4ce8a8b747772a5371513b64ab834f575c750d3/checks/collector.py#L391

Removing the leading `u` from the line and restarting the agent locally prevents the message from being broadcast, and produces a log line without a leading Unicode character `�<30>` as well as no broadcast message.

Tried on Ubuntu 12.04 with Python 2.7.3 - problem not seen. Nor in Ubuntu 10.04 2.6.5 - which is disconcerting, as I couldn't find anything in the ChangeLogs for 2.6.6 specific to SysLogHandler.

The logging was introduced in a6d44b86ced9e531725edfec7dde0da31144a599 - and triggers on Python 2.6.6.
A bug report on a similar limitation here: http://bugs.python.org/issue7077

I'm guessing that the unicode is to prevent trying to log non-ASCII characters that may be in the tags, but this broadcasts an emergency warning on CentOS/RedHat.



Discussion:

Resolving patch:
diff --git a/checks/collector.py b/checks/collector.py
--- a/checks/collector.py
+++ b/checks/collector.py
@@ -388,7 +388,7 @@ def _build_payload(self, start_event=True):
 
             # Log the metadata on the first run
             if self._is_first_run():
-                log.info(u"Hostnames: %s, tags: %s" % (repr(self.metadata_cache), payload['host-tags']))
+                log.info("Hostnames: %s, tags: %s" % (repr(self.metadata_cache), payload['host-tags']))
 
         return payload
 


Test patch: