Repository: green-coding-solutions/green-metrics-tool
Pull request #1193: Phase padding can now be turned on and off
Author: login: ArneTR
Created: 2025-05-17T10:44:18Z | Merged: 2025-05-18T05:48:23Z

TODO: Needs still setting option in the admin and a migration and default entry in structure.sql for user_id = 1

<!-- greptile_comment -->

## Greptile Summary

Adds configurable phase padding functionality to the Green Metrics Tool, allowing users to control padding at phase ends through both UI settings and CLI flags.

- Added phase padding toggle in settings UI with proper database migrations and default configuration in `structure.sql`
- Added `--no-phase-padding` CLI flag in `runner.py` for command-line control
- Modified `ScenarioRunner` to calculate `_phase_padding_ms` based on metric provider resolution
- Added phase padding capability to user permissions system in `lib/user.py`
- Added comprehensive test coverage in `test_frontend.py` for UI interactions and state persistence



<!-- /greptile_comment -->


Diff:
diff --git a/docker/structure.sql b/docker/structure.sql
index a5cb343f9..fa097b13e 100644
--- a/docker/structure.sql
+++ b/docker/structure.sql
@@ -45,7 +45,8 @@ VALUES (
                 "measurement.dev_no_optimizations",
                 "measurement.disabled_metric_providers",
                 "measurement.flow_process_duration",
-                "measurement.total_duration"
+                "measurement.total_duration",
+                "measurement.phase_padding"
             ]
         },
         "api": {
@@ -101,6 +102,7 @@ VALUES (
         },
         "machines": [1],
         "measurement": {
+            "phase_padding": true,
             "quotas": {},
             "dev_no_sleeps": false,
             "dev_no_optimizations": false,
diff --git a/frontend/js/settings.js b/frontend/js/settings.js
index 02babe19e..b1beb5cd3 100644
--- a/frontend/js/settings.js
+++ b/frontend/js/settings.js
@@ -25,6 +25,7 @@ const getSettings = async () => {
 
         if (data?.data?._capabilities?.measurement?.dev_no_optimizations === true) document.querySelector('#measurement-dev-no-optimizations').checked = true;
         if (data?.data?._capabilities?.measurement?.dev_no_sleeps === true) document.querySelector('#measurement-dev-no-sleeps').checked = true;
+        if (data?.data?._capabilities?.measurement?.phase_padding === true) document.querySelector('#measurement-phase-padding').checked = true;
         document.querySelector('#measurement-flow-process-duration').value = data?.data?._capabilities?.measurement?.flow_process_duration;
         document.querySelector('#measurement-total-duration').value = data?.data?._capabilities?.measurement?.total_duration;
         $('#measurement-disabled-metric-providers').dropdown('set exactly', data?.data?._capabilities?.measurement?.disabled_metric_providers);
diff --git a/frontend/settings.html b/frontend/settings.html
index aa11a28be..0bdc244a3 100644
--- a/frontend/settings.html
+++ b/frontend/settings.html
@@ -116,6 +116,11 @@ <h1 class="ui header float left"><a id="menu-toggle" class="opened"><i class="ba
                                     <td><input type="text" placeholder="Input a value" id="measurement-total-duration" name="measurement-total-duration" data-setting="measurement.total_duration" required="" ></td>
                                     <td><button id="save-measurement-total-duration" class="ui positive small button" onclick="updateSetting(this);">Save</button></td>
                                 </tr>
+                                <tr>
+                                    <td>Phase padding</td>
+                                    <td><input type="checkbox" id="measurement-phase-padding" name="measurement-phase-padding" data-setting="measurement.phase_padding"></td>
+                                    <td><button id="save-measurement-phase-padding" class="ui positive small button" onclick="updateSetting(this);">Save</button></td>
+                                </tr>
                                 <tr>
                                     <td>(DEV) No Sleeps during run<br><span><small><i class="question circle icon"></i>You can use this while developing to speed up runs on the cluster. However it will make the measurement invalid as important cooldown times are skipped.</small></span></td>
                                     <td><input type="checkbox" id="measurement-dev-no-sleeps" name="measurement-dev-no-sleeps" data-setting="measurement.dev_no_sleeps"></td>
diff --git a/lib/scenario_runner.py b/lib/scenario_runner.py
index fc4db76d4..447713e61 100644
--- a/lib/scenario_runner.py
+++ b/lib/scenario_runner.py
@@ -69,11 +69,12 @@ def __init__(self,
         dev_no_sleeps=False, dev_cache_build=False, dev_no_metrics=False,
         dev_flow_timetravel=False, dev_no_optimizations=False, docker_prune=False, job_id=None,
         user_id=1, measurement_flow_process_duration=None, measurement_total_duration=None, disabled_metric_providers=None, allowed_run_args=None, dev_no_phase_stats=False, dev_no_save=False,
-        skip_volume_inspect=False, commit_hash_folder=None, usage_scenario_variables=None):
+        skip_volume_inspect=False, commit_hash_folder=None, usage_scenario_variables=None, phase_padding=True):
 
         if skip_unsafe is True and allow_unsafe is True:
             raise RuntimeError('Cannot specify both --skip-unsafe and --allow-unsafe')
 
+        config = GlobalConfig().config
         # variables that should not change if you call run multiple times
         if name:
             self._name = name
@@ -104,7 +105,7 @@ def __init__(self,
         self._architecture = utils.get_architecture()
 
         self._sci = {'R_d': None, 'R': 0}
-        self._sci |= GlobalConfig().config.get('sci', None)  # merge in data from machine config like I, TE etc.
+        self._sci |= config.get('sci', None)  # merge in data from machine config like I, TE etc.
 
         self._job_id = job_id
         self._arguments = locals()
@@ -112,7 +113,7 @@ def __init__(self,
         self._run_id = None
         self._commit_hash = None
         self._commit_timestamp = None
-        self._sampling_interval_padding = 0
+
         self._commit_hash_folder = commit_hash_folder if commit_hash_folder else ''
         self._user_id = user_id
         self._measurement_flow_process_duration = measurement_flow_process_duration
@@ -120,6 +121,11 @@ def __init__(self,
         self._disabled_metric_providers = [] if disabled_metric_providers is None else disabled_metric_providers
         self._allowed_run_args = [] if allowed_run_args is None else allowed_run_args # They are specific to the orchestrator. However currently we only have one. As soon as we support more orchestrators we will sub-class Runner with dedicated child classes (DockerRunner, PodmanRunner etc.)
         self._last_measurement_duration = 0
+        self._phase_padding = phase_padding
+        self._phase_padding_ms = max(
+            utils.get_metric_providers(config, self._disabled_metric_providers).values(),
+            key=lambda x: x.get('resolution', 0) if x else 0
+        ).get('resolution', 0)
 
         del self._arguments['self'] # self is not needed and also cannot be serialzed. We remove it
 
@@ -500,10 +506,8 @@ def initialize_run(self):
         measurement_config['allowed_run_args'] = self._allowed_run_args
         measurement_config['disabled_metric_providers'] = self._disabled_metric_providers
         measurement_config['sci'] = self._sci
-        self._sampling_interval_padding = measurement_config['sampling_interval_padding'] = max(
-            measurement_config['providers'].values(),
-            key=lambda x: x.get('resolution', 0) if x else 0
-        ).get('resolution', 0)
+        measurement_config['phase_padding'] = self._phase_padding_ms
+
 
         # We issue a fetch_one() instead of a query() here, cause we want to get the RUN_ID
         self._run_id = DB().fetch_one("""
@@ -1271,9 +1275,10 @@ def end_phase(self, phase):
 
         phase_time = int(time.time_ns() / 1_000)
 
-        self.__notes_helper.add_note({'note': f"Ending phase {phase} [UNPADDED]", 'detail_name': '[NOTES]', 'timestamp': phase_time})
-        phase_time += self._sampling_interval_padding*1000 # value is in ms and we need to get to us
-        time.sleep(self._sampling_interval_padding/1000) # no custom sleep here as even with dev_no_sleeps we must ensure phases don't overlap
+        if self._phase_padding:
+            self.__notes_helper.add_note({'note': f"Ending phase {phase} [UNPADDED]", 'detail_name': '[NOTES]', 'timestamp': phase_time})
+            phase_time += self._phase_padding_ms*1000 # value is in ms and we need to get to us
+            time.sleep(self._phase_padding_ms/1000) # no custom sleep here as even with dev_no_sleeps we must ensure phases don't overlap
 
         if phase not in self.__phases:
             raise RuntimeError('Calling end_phase before start_phase. This is a developer error!')
@@ -1286,9 +1291,12 @@ def end_phase(self, phase):
 
                 subprocess.run(['docker', 'pause', container_to_pause], check=True, stdout=subprocess.DEVNULL)
 
-
         self.__phases[phase]['end'] = phase_time
-        self.__notes_helper.add_note({'note': f"Ending phase {phase} [PADDED]", 'detail_name': '[NOTES]', 'timestamp': phase_time})
+
+        if self._phase_padding:
+            self.__notes_helper.add_note({'note': f"Ending phase {phase} [PADDED]", 'detail_name': '[NOTES]', 'timestamp': phase_time})
+        else:
+            self.__notes_helper.add_note({'note': f"Ending phase {phase} [UNPADDED]", 'detail_name': '[NOTES]', 'timestamp': phase_time})
 
     def run_flows(self):
         ps_to_kill_tmp = []
diff --git a/lib/user.py b/lib/user.py
index 7b03d302c..ad6dea5dd 100644
--- a/lib/user.py
+++ b/lib/user.py
@@ -65,7 +65,7 @@ def change_setting(self, name, value):
             raise ValueError(f"You cannot change this setting: {name}")
 
         match name:
-            case 'measurement.dev_no_optimizations' | 'measurement.dev_no_sleeps':
+            case 'measurement.dev_no_optimizations' | 'measurement.dev_no_sleeps' | 'measurement.phase_padding':
                 if not isinstance(value, bool):
                     raise ValueError(f'The setting {name} must be boolean')
             case 'measurement.flow_process_duration' | 'measurement.total_duration':
diff --git a/migrations/2025_05_18_phase_padding.sql b/migrations/2025_05_18_phase_padding.sql
new file mode 100644
index 000000000..c4ed833df
--- /dev/null
+++ b/migrations/2025_05_18_phase_padding.sql
@@ -0,0 +1,18 @@
+UPDATE users
+SET capabilities = jsonb_set(
+    capabilities,
+    '{measurement,phase_padding}',
+    'true'::jsonb,
+    true
+); -- for all users
+
+UPDATE users
+SET capabilities = jsonb_set(
+    capabilities,
+    '{user,updateable_settings}',
+    (
+        COALESCE(capabilities->'user'->'updateable_settings', '[]'::jsonb) ||
+        '["measurement.phase_padding"]'::jsonb
+    ),
+    true
+); -- for all users
\ No newline at end of file
diff --git a/runner.py b/runner.py
index 31fdcdccb..8d520f304 100755
--- a/runner.py
+++ b/runner.py
@@ -47,6 +47,7 @@
     parser.add_argument('--full-docker-prune', action='store_true', help='Stop and remove all containers, build caches, volumes and images on the system')
     parser.add_argument('--docker-prune', action='store_true', help='Prune all unassociated build caches, networks volumes and stopped containers on the system')
     parser.add_argument('--skip-volume-inspect', action='store_true', help='Disable docker volume inspection. Can help if you encounter permission issues.')
+    parser.add_argument('--no-phase-padding', action='store_true', help='Do not add paddings to phase end to capture incomplete last sampling interval.')
     parser.add_argument('--dev-flow-timetravel', action='store_true', help='Allows to repeat a failed flow or timetravel to beginning of flows or restart services.')
     parser.add_argument('--dev-no-metrics', action='store_true', help='Skips loading the metric providers. Runs will be faster, but you will have no metric')
     parser.add_argument('--dev-no-sleeps', action='store_true', help='Removes all sleeps. Resulting measurement data will be skewed.')
@@ -118,7 +119,7 @@
                     dev_flow_timetravel=args.dev_flow_timetravel, dev_no_optimizations=args.dev_no_optimizations,
                     docker_prune=args.docker_prune, dev_no_phase_stats=args.dev_no_phase_stats, user_id=args.user_id,
                     skip_volume_inspect=args.skip_volume_inspect, commit_hash_folder=args.commit_hash_folder,
-                    usage_scenario_variables=variables_dict)
+                    usage_scenario_variables=variables_dict, phase_padding=not args.no_phase_padding)
 
     # Using a very broad exception makes sense in this case as we have excepted all the specific ones before
     #pylint: disable=broad-except
diff --git a/tests/frontend/test_frontend.py b/tests/frontend/test_frontend.py
index 39cb37bdd..15243e7ea 100644
--- a/tests/frontend/test_frontend.py
+++ b/tests/frontend/test_frontend.py
@@ -530,29 +530,54 @@ def test_settings_measurement():
     page.wait_for_load_state("load") # ALL JS should be done
 
     user = User(1)
+    # check default values
+    assert user._capabilities['measurement']['disabled_metric_providers'] == []
+    assert user._capabilities['measurement']['flow_process_duration'] == 86400
+    assert user._capabilities['measurement']['total_duration'] == 86400
+    assert user._capabilities['measurement']['phase_padding'] is True
+    assert user._capabilities['measurement']['dev_no_sleeps'] is False
+    assert user._capabilities['measurement']['dev_no_optimizations'] is False
 
-    value = page.locator('#measurement-total-duration').input_value()
-    assert int(value.strip()) == user._capabilities['measurement']['total_duration']
 
+    value = page.locator('#measurement-disabled-metric-providers').input_value()
+    providers = [] if value.strip() == '' else [value.strip()]
+    assert providers == user._capabilities['measurement']['disabled_metric_providers']
 
     value = page.locator('#measurement-flow-process-duration').input_value()
     assert int(value.strip()) == user._capabilities['measurement']['flow_process_duration']
 
-    value = page.locator('#measurement-disabled-metric-providers').input_value()
-    providers = [] if value.strip() == '' else [value.strip()]
-    assert providers == user._capabilities['measurement']['disabled_metric_providers']
+    value = page.locator('#measurement-total-duration').input_value()
+    assert int(value.strip()) == user._capabilities['measurement']['total_duration']
+
+    value = page.locator('#measurement-phase-padding').is_checked()
+    assert value is user._capabilities['measurement']['phase_padding']
+
+    value = page.locator('#measurement-dev-no-sleeps').is_checked()
+    assert value is user._capabilities['measurement']['dev_no_sleeps']
+
+    value = page.locator('#measurement-dev-no-optimizations').is_checked()
+    assert value is user._capabilities['measurement']['dev_no_optimizations']
 
-    page.locator('#measurement-total-duration').fill('123')
-    page.locator('#measurement-flow-process-duration').fill('456')
     page.evaluate('$("#measurement-disabled-metric-providers").dropdown("set exactly", "NetworkConnectionsProxyContainerProvider");')
+    page.locator('#measurement-flow-process-duration').fill('456')
+    page.locator('#measurement-total-duration').fill('123')
+    page.locator('#measurement-phase-padding').click()
+    page.locator('#measurement-dev-no-sleeps').click()
+    page.locator('#measurement-dev-no-optimizations').click()
 
-    page.locator('#save-measurement-total-duration').click()
-    page.locator('#save-measurement-flow-process-duration').click()
     page.locator('#save-measurement-disabled-metric-providers').click()
+    page.locator('#save-measurement-flow-process-duration').click()
+    page.locator('#save-measurement-total-duration').click()
+    page.locator('#save-measurement-phase-padding').click()
+    page.locator('#save-measurement-dev-no-sleeps').click()
+    page.locator('#save-measurement-dev-no-optimizations').click()
 
     page.wait_for_load_state("networkidle") # ALL AJAX should be done
 
     user = User(1)
-    assert user._capabilities['measurement']['total_duration'] == 123
-    assert user._capabilities['measurement']['flow_process_duration'] == 456
     assert user._capabilities['measurement']['disabled_metric_providers'] == ['NetworkConnectionsProxyContainerProvider']
+    assert user._capabilities['measurement']['flow_process_duration'] == 456
+    assert user._capabilities['measurement']['total_duration'] == 123
+    assert user._capabilities['measurement']['phase_padding'] is False
+    assert user._capabilities['measurement']['dev_no_sleeps'] is True
+    assert user._capabilities['measurement']['dev_no_optimizations'] is True


Reviews:
user: login: greptile-apps[bot]
body: LGTM

<sub>2 file(s) reviewed, no comment(s)</sub>
<sub>[Edit PR Review Bot Settings](https://app.greptile.com/review/github) | [Greptile](https://greptile.com?utm_source=greptile_expert&utm_medium=github&utm_campaign=code_reviews&utm_content=green-metrics-tool_1193)</sub>

state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-17T10:45:42Z
user: login: greptile-apps[bot]
body: <sub>8 file(s) reviewed, 2 comment(s)</sub>
<sub>[Edit PR Review Bot Settings](https://app.greptile.com/review/github) | [Greptile](https://greptile.com?utm_source=greptile_expert&utm_medium=github&utm_campaign=code_reviews&utm_content=green-metrics-tool_1193)</sub>
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-18T05:45:41Z

Review comments:
diff_hunk: @@ -104,22 +105,27 @@ def __init__(self,
         self._architecture = utils.get_architecture()
 
         self._sci = {'R_d': None, 'R': 0}
-        self._sci |= GlobalConfig().config.get('sci', None)  # merge in data from machine config like I, TE etc.
+        self._sci |= config.get('sci', None)  # merge in data from machine config like I, TE etc.
 
         self._job_id = job_id
         self._arguments = locals()
         self._repo_folder = f"{self._tmp_folder}/repo" # default if not changed in checkout_repository
         self._run_id = None
         self._commit_hash = None
         self._commit_timestamp = None
-        self._sampling_interval_padding = 0
+
         self._commit_hash_folder = commit_hash_folder if commit_hash_folder else ''
         self._user_id = user_id
         self._measurement_flow_process_duration = measurement_flow_process_duration
         self._measurement_total_duration = measurement_total_duration
         self._disabled_metric_providers = [] if disabled_metric_providers is None else disabled_metric_providers
         self._allowed_run_args = [] if allowed_run_args is None else allowed_run_args # They are specific to the orchestrator. However currently we only have one. As soon as we support more orchestrators we will sub-class Runner with dedicated child classes (DockerRunner, PodmanRunner etc.)
         self._last_measurement_duration = 0
+        self._phase_padding = phase_padding
+        self._phase_padding_ms = max(
+            utils.get_metric_providers(config, self._disabled_metric_providers).values(),
+            key=lambda x: x.get('resolution', 0) if x else 0
+        ).get('resolution', 0)
path: lib/scenario_runner.py
user: login: greptile-apps[bot]
body: logic: Potential error if no metric providers are configured - `max()` will fail on empty sequence. Should provide a default value or handle empty case.

```suggestion
        self._phase_padding = phase_padding
        providers = utils.get_metric_providers(config, self._disabled_metric_providers).values()
        self._phase_padding_ms = max(
            providers,
            key=lambda x: x.get('resolution', 0) if x else 0
        ).get('resolution', 0) if providers else 0
```
created_at: 2025-05-18T05:44:35Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,18 @@
+UPDATE users
+SET capabilities = jsonb_set(
+    capabilities,
+    '{measurement,phase_padding}',
+    'true'::jsonb,
+    true
+); -- for all users
+
+UPDATE users
+SET capabilities = jsonb_set(
+    capabilities,
+    '{user,updateable_settings}',
+    (
+        COALESCE(capabilities->'user'->'updateable_settings', '[]'::jsonb) ||
+        '["measurement.phase_padding"]'::jsonb
path: migrations/2025_05_18_phase_padding.sql
user: login: greptile-apps[bot]
body: logic: Array concatenation with || can lead to duplicate entries if measurement.phase_padding already exists in updateable_settings. Consider using jsonb_set with a jsonb_uniq() to prevent duplicates.

```suggestion
        (SELECT jsonb_agg(DISTINCT element)
        FROM jsonb_array_elements(
            COALESCE(capabilities->'user'->'updateable_settings', '[]'::jsonb) ||
            '["measurement.phase_padding"]'::jsonb
        ) AS element)
```
created_at: 2025-05-18T05:45:01Z
author_association: CONTRIBUTOR

PR comments:
user: login: ArneTR
created_at: 2025-05-18T05:43:38Z
body: @greptileai

author_association: MEMBER
user: login: github-actions[bot]
created_at: 2025-05-18T05:52:11Z
body: Eco CI Output: 
|Label|🖥 avg. CPU utilization [%]|🔋 Total Energy [Joules]|🔌 avg. Power [Watts]|Duration [Seconds]|
|---|---|---|---|---|
|Measurement #1|28.0738|3379.07|4.16|811.92|
|---|---|---|---|---|
|Total Run|28.07|3379.07|4.16|811.92|
|---|---|---|---|---|
|Additional overhead from Eco CI|N/A|9.79|4.03|2.43|

🌳 CO2 Data:
City: <b>Washington</b>, Lat: <b>38.7095</b>, Lon: <b>-78.1539</b>
IP: <b>172.190.111.22</b>
CO₂ from energy is: 1.199569850 g
CO₂ from manufacturing (embodied carbon) is: 0.231651987 g
<a href='https://www.electricitymaps.com/methodology#carbon-intensity-and-emission-factors' target=_blank rel=noopener>Carbon Intensity</a> for this location: <b>355 gCO₂eq/kWh</b>
<a href='https://sci-guide.greensoftware.foundation/'  target=_blank rel=noopener>SCI</a>: <b>1.431222 gCO₂eq / pipeline run</b> emitted
<hr>
Total cost of whole PR so far: <br><br>
<a href='https://metrics.green-coding.io/ci.html?repo=green-coding-solutions%2Fgreen-metrics-tool&branch=1193%2Fmerge&workflow=56356609'><img src='https://api.green-coding.io/v1/ci/badge/get?repo=green-coding-solutions%2Fgreen-metrics-tool&branch=1193%2Fmerge&workflow=56356609&mode=totals&metric=energy#27611'></a>
<a href='https://metrics.green-coding.io/ci.html?repo=green-coding-solutions%2Fgreen-metrics-tool&branch=1193%2Fmerge&workflow=56356609'><img src='https://api.green-coding.io/v1/ci/badge/get?repo=green-coding-solutions%2Fgreen-metrics-tool&branch=1193%2Fmerge&workflow=56356609&mode=totals&metric=carbon#27611'></a>

author_association: NONE