Repository: Azure/WALinuxAgent
Pull request #2215 (2021-03-25 17:39:20)

Initialize published_hostname using /var/lib/cloud/data/set-hostname
published_hostname needs to be initialized with the value created at provisioning time, otherwise the agent would not detect changes in the hostname done before the env thread starts monitoring the hostname.

codecov[bot]: # [Codecov](https://codecov.io/gh/Azure/WALinuxAgent/pull/2215?src=pr&el=h1) Report
> Merging [#2215](https://codecov.io/gh/Azure/WALinuxAgent/pull/2215?src=pr&el=desc) (200c304) into [develop](https://codecov.io/gh/Azure/WALinuxAgent/commit/cdcfd0d1e1f12ff4eade18abd3e99ac431678b61?el=desc) (cdcfd0d) will **increase** coverage by `0.09%`.
> The diff coverage is `57.89%`.

[![Impacted file tree graph](https://codecov.io/gh/Azure/WALinuxAgent/pull/2215/graphs/tree.svg?width=650&height=150&src=pr&token=ae90E2KYCw)](https://codecov.io/gh/Azure/WALinuxAgent/pull/2215?src=pr&el=tree)

```diff
@@             Coverage Diff             @@
##           develop    #2215      +/-   ##
===========================================
+ Coverage    70.48%   70.58%   +0.09%     
===========================================
  Files           96       96              
  Lines        13574    13591      +17     
  Branches      1918     1921       +3     
===========================================
+ Hits          9568     9593      +25     
+ Misses        3591     3583       -8     
  Partials       415      415              
```


| [Impacted Files](https://codecov.io/gh/Azure/WALinuxAgent/pull/2215?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [azurelinuxagent/common/conf.py](https://codecov.io/gh/Azure/WALinuxAgent/pull/2215/diff?src=pr&el=tree#diff-YXp1cmVsaW51eGFnZW50L2NvbW1vbi9jb25mLnB5) | `77.03% <ø> (ø)` | |
| [azurelinuxagent/common/osutil/default.py](https://codecov.io/gh/Azure/WALinuxAgent/pull/2215/diff?src=pr&el=tree#diff-YXp1cmVsaW51eGFnZW50L2NvbW1vbi9vc3V0aWwvZGVmYXVsdC5weQ==) | `60.12% <57.89%> (+0.72%)` | :arrow_up: |
| [azurelinuxagent/common/event.py](https://codecov.io/gh/Azure/WALinuxAgent/pull/2215/diff?src=pr&el=tree#diff-YXp1cmVsaW51eGFnZW50L2NvbW1vbi9ldmVudC5weQ==) | `85.97% <0.00%> (+0.67%)` | :arrow_up: |
| [azurelinuxagent/ga/collect\_telemetry\_events.py](https://codecov.io/gh/Azure/WALinuxAgent/pull/2215/diff?src=pr&el=tree#diff-YXp1cmVsaW51eGFnZW50L2dhL2NvbGxlY3RfdGVsZW1ldHJ5X2V2ZW50cy5weQ==) | `91.43% <0.00%> (+1.71%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/Azure/WALinuxAgent/pull/2215?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/Azure/WALinuxAgent/pull/2215?src=pr&el=footer). Last update [cdcfd0d...200c304](https://codecov.io/gh/Azure/WALinuxAgent/pull/2215?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Patch:
diff --git a/azurelinuxagent/common/conf.py b/azurelinuxagent/common/conf.py
index 2b446ad8..5b6b9778 100644
--- a/azurelinuxagent/common/conf.py
+++ b/azurelinuxagent/common/conf.py
@@ -248,6 +248,7 @@ def get_lib_dir(conf=__conf__):
 
 
 def get_published_hostname(conf=__conf__):
+    # Some applications rely on this file; do not remove this setting
     return os.path.join(get_lib_dir(conf), 'published_hostname')
 
 
diff --git a/azurelinuxagent/common/osutil/default.py b/azurelinuxagent/common/osutil/default.py
index aa8c69a5..066e1431 100644
--- a/azurelinuxagent/common/osutil/default.py
+++ b/azurelinuxagent/common/osutil/default.py
@@ -21,6 +21,7 @@ import datetime
 import errno
 import fcntl
 import glob
+import json
 import multiprocessing
 import os
 import platform
@@ -1309,16 +1310,44 @@ class DefaultOSUtil(object):
     def get_hostname_record(self):
         hostname_record = conf.get_published_hostname()
         if not os.path.exists(hostname_record):
-            # this file is created at provisioning time with agents >= 2.2.3
-            hostname = socket.gethostname()
-            logger.info('Hostname record does not exist, '
-                        'creating [{0}] with hostname [{1}]',
-                        hostname_record,
-                        hostname)
+            # older agents (but newer or equal to 2.2.3) create published_hostname during provisioning; when provisioning is done
+            # by cloud-init the hostname is written to set-hostname
+            hostname = self._get_cloud_init_hostname()
+            if hostname is None:
+                logger.info("Retrieving hostname using socket.gethostname()")
+                hostname = socket.gethostname()
+            logger.info('Published hostname record does not exist, creating [{0}] with hostname [{1}]', hostname_record, hostname)
             self.set_hostname_record(hostname)
         record = fileutil.read_file(hostname_record)
         return record
 
+    @staticmethod
+    def _get_cloud_init_hostname():
+        """
+        Retrieves the hostname set by cloud-init; returns None if cloud-init did not set the hostname or if there is an
+        error retrieving it.
+        """
+        hostname_file = '/var/lib/cloud/data/set-hostname'
+        try:
+            if os.path.exists(hostname_file):
+                #
+                # The format is similar to
+                #
+                #     $ cat /var/lib/cloud/data/set-hostname
+                #     {
+                #      "fqdn": "nam-u18",
+                #      "hostname": "nam-u18"
+                #     }
+                #
+                logger.info("Retrieving hostname from {0}", hostname_file)
+                with open(hostname_file, 'r') as file_:
+                    hostname_info = json.load(file_)
+                if "hostname" in hostname_info:
+                    return hostname_info["hostname"]
+        except Exception as exception:
+            logger.warn("Error retrieving hostname: {0}", ustr(exception))
+        return None
+
     def del_account(self, username):
         if self.is_sys_user(username):
             logger.error("{0} is a system user. Will not delete it.", username)


Test patch:
diff --git a/tests/common/osutil/test_default.py b/tests/common/osutil/test_default.py
index aa8adf66..65d7ae0f 100644
--- a/tests/common/osutil/test_default.py
+++ b/tests/common/osutil/test_default.py
@@ -34,7 +34,8 @@ from azurelinuxagent.common.future import ustr
 from azurelinuxagent.common.osutil import get_osutil
 from azurelinuxagent.common.utils import fileutil
 from azurelinuxagent.common.utils.flexible_version import FlexibleVersion
-from tests.tools import AgentTestCase, patch, open_patch, load_data
+from tests.common.mock_environment import MockEnvironment
+from tests.tools import AgentTestCase, patch, open_patch, load_data, data_dir
 
 actual_get_proc_net_route = 'azurelinuxagent.common.osutil.default.DefaultOSUtil._get_proc_net_route'
 
@@ -44,12 +45,6 @@ def fake_is_loopback(_, iface):
 
 
 class TestOSUtil(AgentTestCase):
-    def setUp(self):
-        AgentTestCase.setUp(self)
-
-    def tearDown(self):
-        AgentTestCase.tearDown(self)
-
     def test_restart(self):
         # setup
         retries = 3
@@ -951,6 +946,7 @@ Chain OUTPUT (policy ACCEPT 104 packets, 43628 bytes)
         dev = osutil.DefaultOSUtil().device_for_ide_port(1)
         self.assertIsNone(dev, 'None should be returned if no resource disk found')
 
+
 def osutil_get_dhcp_pid_should_return_a_list_of_pids(test_instance, osutil_instance):
     """
     This is a very basic test for osutil.get_dhcp_pid. It is simply meant to exercise the implementation of that method
@@ -971,5 +967,47 @@ def osutil_get_dhcp_pid_should_return_a_list_of_pids(test_instance, osutil_insta
     test_instance.assertTrue(len(pid) != 0, "get_dhcp_pid did not return a PID")
 
 
+class TestGetPublishedHostname(AgentTestCase):
+    def setUp(self):
+        AgentTestCase.setUp(self)
+        self.__published_hostname = os.path.join(self.tmp_dir, "published_hostname")
+        self.__patcher = patch('azurelinuxagent.common.osutil.default.conf.get_published_hostname', return_value=self.__published_hostname)
+        self.__patcher.start()
+
+    def tearDown(self):
+        self.__patcher.stop()
+        AgentTestCase.tearDown(self)
+
+    def __get_published_hostname_contents(self):
+        with open(self.__published_hostname, "r") as file_:
+            return file_.read()
+
+    def test_get_hostname_record_should_create_published_hostname(self):
+        actual = osutil.DefaultOSUtil().get_hostname_record()
+
+        expected = socket.gethostname()
+        self.assertEqual(expected, actual, "get_hostname_record returned an incorrect hostname")
+        self.assertTrue(os.path.exists(self.__published_hostname), "The published_hostname file was not created")
+        self.assertEqual(expected, self.__get_published_hostname_contents(), "get_hostname_record returned an incorrect hostname")
+
+    def test_get_hostname_record_should_use_existing_published_hostname(self):
+        expected = "a-sample-hostname-used-for-testing"
+        with open(self.__published_hostname, "w") as file_:
+            file_.write(expected)
+
+        actual = osutil.DefaultOSUtil().get_hostname_record()
+
+        self.assertEqual(expected, actual, "get_hostname_record returned an incorrect hostname")
+        self.assertEqual(expected, self.__get_published_hostname_contents(), "get_hostname_record returned an incorrect hostname")
+
+    def test_get_hostname_record_should_initialize_the_host_name_using_cloud_init_info(self):
+        with MockEnvironment(self.tmp_dir, files=[('/var/lib/cloud/data/set-hostname', os.path.join(data_dir, "cloud-init", "set-hostname"))]):
+            actual = osutil.DefaultOSUtil().get_hostname_record()
+
+        expected = "a-sample-set-hostname"
+        self.assertEqual(expected, actual, "get_hostname_record returned an incorrect hostname")
+        self.assertEqual(expected, self.__get_published_hostname_contents(), "get_hostname_record returned an incorrect hostname")
+
+
 if __name__ == '__main__':
     unittest.main()
diff --git a/tests/data/cloud-init/set-hostname b/tests/data/cloud-init/set-hostname
new file mode 100644
index 00000000..3d89ca77
--- /dev/null
+++ b/tests/data/cloud-init/set-hostname
@@ -0,0 +1,4 @@
+{
+ "fqdn": "a-sample-set-hostname.domain.com",
+ "hostname": "a-sample-set-hostname"
+}