Repository: Azure/WALinuxAgent
Pull request #2199 (2021-03-05 15:48:00)

Delete CGroups.Enabled flag; add debug info
I added CGroups.Enabled a few iterations ago, but after the first round of tests I don't see a good motivation for keeping it so I am removing it.

Also, I am adding some small changes to help end-to-end tests. Some of those changes are activated using new "Debug" flags in waagent.conf.

codecov[bot]: # [Codecov](https://codecov.io/gh/Azure/WALinuxAgent/pull/2199?src=pr&el=h1) Report
> Merging [#2199](https://codecov.io/gh/Azure/WALinuxAgent/pull/2199?src=pr&el=desc) (7496a77) into [develop](https://codecov.io/gh/Azure/WALinuxAgent/commit/3069e8ea8a48422cee4f0266641120b672db96bd?el=desc) (3069e8e) will **decrease** coverage by `0.03%`.
> The diff coverage is `100.00%`.

[![Impacted file tree graph](https://codecov.io/gh/Azure/WALinuxAgent/pull/2199/graphs/tree.svg?width=650&height=150&src=pr&token=ae90E2KYCw)](https://codecov.io/gh/Azure/WALinuxAgent/pull/2199?src=pr&el=tree)

```diff
@@             Coverage Diff             @@
##           develop    #2199      +/-   ##
===========================================
- Coverage    70.28%   70.25%   -0.04%     
===========================================
  Files           96       96              
  Lines        13449    13451       +2     
  Branches      1903     1901       -2     
===========================================
- Hits          9453     9450       -3     
- Misses        3585     3588       +3     
- Partials       411      413       +2     
```


| [Impacted Files](https://codecov.io/gh/Azure/WALinuxAgent/pull/2199?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [azurelinuxagent/common/cgroupconfigurator.py](https://codecov.io/gh/Azure/WALinuxAgent/pull/2199/diff?src=pr&el=tree#diff-YXp1cmVsaW51eGFnZW50L2NvbW1vbi9jZ3JvdXBjb25maWd1cmF0b3IucHk=) | `71.01% <100.00%> (-0.25%)` | :arrow_down: |
| [azurelinuxagent/common/conf.py](https://codecov.io/gh/Azure/WALinuxAgent/pull/2199/diff?src=pr&el=tree#diff-YXp1cmVsaW51eGFnZW50L2NvbW1vbi9jb25mLnB5) | `76.58% <100.00%> (-0.04%)` | :arrow_down: |
| [azurelinuxagent/common/version.py](https://codecov.io/gh/Azure/WALinuxAgent/pull/2199/diff?src=pr&el=tree#diff-YXp1cmVsaW51eGFnZW50L2NvbW1vbi92ZXJzaW9uLnB5) | `72.28% <100.00%> (ø)` | |
| [azurelinuxagent/ga/monitor.py](https://codecov.io/gh/Azure/WALinuxAgent/pull/2199/diff?src=pr&el=tree#diff-YXp1cmVsaW51eGFnZW50L2dhL21vbml0b3IucHk=) | `75.58% <100.00%> (+0.14%)` | :arrow_up: |
| [azurelinuxagent/ga/collect\_telemetry\_events.py](https://codecov.io/gh/Azure/WALinuxAgent/pull/2199/diff?src=pr&el=tree#diff-YXp1cmVsaW51eGFnZW50L2dhL2NvbGxlY3RfdGVsZW1ldHJ5X2V2ZW50cy5weQ==) | `89.72% <0.00%> (-1.37%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/Azure/WALinuxAgent/pull/2199?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/Azure/WALinuxAgent/pull/2199?src=pr&el=footer). Last update [3069e8e...1853f7c](https://codecov.io/gh/Azure/WALinuxAgent/pull/2199?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Patch:
diff --git a/README.md b/README.md
index 702fbc88..25fd4a82 100644
--- a/README.md
+++ b/README.md
@@ -219,7 +219,6 @@ OS.SshClientAliveInterval=180
 OS.SshDir=/etc/ssh
 HttpProxy.Host=None
 HttpProxy.Port=None
-CGroups.Enabled=y
 ```
 
 The various configuration options are described in detail below. Configuration
diff --git a/azurelinuxagent/common/cgroupconfigurator.py b/azurelinuxagent/common/cgroupconfigurator.py
index 067fab44..18a5c0bf 100644
--- a/azurelinuxagent/common/cgroupconfigurator.py
+++ b/azurelinuxagent/common/cgroupconfigurator.py
@@ -19,7 +19,6 @@ import os
 import re
 import subprocess
 
-from azurelinuxagent.common import conf
 from azurelinuxagent.common import logger
 from azurelinuxagent.common.cgroup import CpuCgroup, AGENT_NAME_TELEMETRY, MetricsCounter
 from azurelinuxagent.common.cgroupapi import CGroupsApi, SystemdCgroupsApi, SystemdRunError
@@ -75,13 +74,13 @@ _AGENT_THROTTLED_TIME_THRESHOLD = 120  # 2 minutes
 
 def _log_cgroup_info(format_string, *args):
     message = format_string.format(*args)
-    logger.info(message)
+    logger.info("[CGI]" + message)
     add_event(op=WALAEventOperation.CGroupsInfo, message=message)
 
 
 def _log_cgroup_warning(format_string, *args):
     message = format_string.format(*args)
-    logger.info(message)  # log as INFO for now, in the future it should be logged as WARNING
+    logger.info("[CGW]" + message)  # log as INFO for now, in the future it should be logged as WARNING
     add_event(op=WALAEventOperation.CGroupsInfo, message=message, is_success=False, log_event=False)
 
 
@@ -106,9 +105,6 @@ class CGroupConfigurator(object):
                 if self._initialized:
                     return
 
-                if not conf.get_cgroups_enabled():
-                    return
-
                 # check whether cgroup monitoring is supported on the current distro
                 self._cgroups_supported = CGroupsApi.cgroups_supported()
                 if not self._cgroups_supported:
diff --git a/azurelinuxagent/common/conf.py b/azurelinuxagent/common/conf.py
index b83f5ace..9d1ac9e2 100644
--- a/azurelinuxagent/common/conf.py
+++ b/azurelinuxagent/common/conf.py
@@ -111,7 +111,11 @@ __SWITCH_OPTIONS__ = {
     "ResourceDisk.EnableSwapEncryption": False,
     "AutoUpdate.Enabled": True,
     "EnableOverProvisioning": True,
-    "CGroups.Enabled": True,
+    #
+    # "Debug" options are experimental and may be removed in later
+    # versions of the Agent.
+    #
+    "Debug.CgroupLogMetrics": False,
 }
 
 
@@ -150,7 +154,12 @@ __INTEGER_OPTIONS__ = {
     "HttpProxy.Port": None,
     "ResourceDisk.SwapSizeMB": 0,
     "Autoupdate.Frequency": 3600,
-    "Logs.CollectPeriod": 3600
+    "Logs.CollectPeriod": 3600,
+    #
+    # "Debug" options are experimental and may be removed in later
+    # versions of the Agent.
+    #
+    "Debug.CgroupCheckPeriod": 300,
 }
 
 
@@ -458,3 +467,19 @@ def get_cgroups_enabled(conf=__conf__):
 
 def get_monitor_network_configuration_changes(conf=__conf__):
     return conf.get_switch("Monitor.NetworkConfigurationChanges", False)
+
+
+def get_cgroup_check_period(conf=__conf__):
+    """
+    How often to perform checks on cgroups (are the processes in the cgroups as expected,
+    has the agent exceeded its quota, etc)
+    """
+    return conf.get_int("Debug.CgroupCheckPeriod", 300)
+
+
+def get_cgroup_log_metrics(conf=__conf__):
+    """
+    If True, resource usage metrics are written to the local log
+    """
+    return conf.get_switch("Debug.CgroupLogMetrics", False)
+
diff --git a/azurelinuxagent/ga/monitor.py b/azurelinuxagent/ga/monitor.py
index 941737ef..17220b82 100644
--- a/azurelinuxagent/ga/monitor.py
+++ b/azurelinuxagent/ga/monitor.py
@@ -49,13 +49,14 @@ class PollResourceUsage(PeriodicOperation):
     It also checks whether there are processes in the agent's cgroup that should not be there.
     """
     def __init__(self):
-        super(PollResourceUsage, self).__init__(datetime.timedelta(minutes=5))
+        super(PollResourceUsage, self).__init__(conf.get_cgroup_check_period())
+        self.__log_metrics = conf.get_cgroup_log_metrics()
 
     def _operation(self):
         tracked_metrics = CGroupsTelemetry.poll_all_tracked()
 
         for metric in tracked_metrics:
-            report_metric(metric.category, metric.counter, metric.instance, metric.value)
+            report_metric(metric.category, metric.counter, metric.instance, metric.value, log_event=self.__log_metrics)
 
         CGroupConfigurator.get_instance().check_cgroups(tracked_metrics)
 
diff --git a/config/waagent.conf b/config/waagent.conf
index 964532dd..87e201e9 100644
--- a/config/waagent.conf
+++ b/config/waagent.conf
@@ -150,6 +150,3 @@ OS.RemovePersistentNetRulesPeriod=30
 
 # How often (in seconds) to monitor for DHCP client restarts
 OS.MonitorDhcpClientRestartPeriod=30
-
-# Use cgroups to monitor the resource usage of the agent and extensions
-CGroups.Enabled=y


Test patch:
diff --git a/tests/common/test_cgroupconfigurator.py b/tests/common/test_cgroupconfigurator.py
index bbafe824..793c40ff 100644
--- a/tests/common/test_cgroupconfigurator.py
+++ b/tests/common/test_cgroupconfigurator.py
@@ -69,11 +69,6 @@ class CGroupConfiguratorSystemdTestCase(AgentTestCase):
         with self._get_cgroup_configurator() as configurator:
             self.assertTrue(configurator.enabled(), "cgroups were not enabled")
 
-    def test_initialize_should_not_enable_cgroups_when_they_are_disabled_in_the_agent_configuration(self):
-        with patch('azurelinuxagent.common.conf.get_cgroups_enabled', return_value=False):
-            with self._get_cgroup_configurator() as configurator:
-                self.assertFalse(configurator.enabled(), "cgroups should not be enabled")
-
     def test_initialize_should_start_tracking_the_agent_cgroups(self):
         with self._get_cgroup_configurator() as configurator:
             tracked = CGroupsTelemetry._tracked
diff --git a/tests/common/test_conf.py b/tests/common/test_conf.py
index 8db5759f..8f98f3d7 100644
--- a/tests/common/test_conf.py
+++ b/tests/common/test_conf.py
@@ -65,8 +65,7 @@ class TestConf(AgentTestCase):
         "AutoUpdate.GAFamily": "Prod",
         "EnableOverProvisioning": True,
         "OS.AllowHTTP": False,
-        "OS.EnableFirewall": False,
-        "CGroups.Enabled": True,
+        "OS.EnableFirewall": False
     }
 
     def setUp(self):
@@ -133,6 +132,3 @@ class TestConf(AgentTestCase):
 
     def test_get_extensions_enabled(self):
         self.assertTrue(conf.get_extensions_enabled(self.conf))
-
-    def test_get_cgroups_enabled(self):
-        self.assertTrue(conf.get_cgroups_enabled(self.conf))
diff --git a/tests/test_agent.py b/tests/test_agent.py
index 91b2dc69..7a4db941 100644
--- a/tests/test_agent.py
+++ b/tests/test_agent.py
@@ -26,8 +26,9 @@ EXPECTED_CONFIGURATION = \
 """AutoUpdate.Enabled = True
 AutoUpdate.GAFamily = Prod
 Autoupdate.Frequency = 3600
-CGroups.Enabled = True
 DVD.MountPoint = /mnt/cdrom/secure
+Debug.CgroupCheckPeriod = 300
+Debug.CgroupLogMetrics = False
 DetectScvmmEnv = False
 EnableOverProvisioning = True
 Extension.LogDir = /var/log/azure