Repository: FlexMeasures/flexmeasures
Pull request #1439: Feat: Show only the flex_context and sensors_to_show sensors.
Author: login: Muhammad-Moiz626
Created: 2025-04-23T14:34:02Z | Merged: 2025-05-05T15:17:56Z

## Description

Removed the logic to fetch all the sensors for a given asset. Only fetching important sensors (Sensors from flex-context and sensors_to_show)

## Look & Feel

No difference in look and feel.

## How to test

- Go to the status page for a sensor.
- Check only limited sensors are being loaded that belong to `flex-context` and `sensors_to_show`.

## Further Improvements

- Nothing.

## Related Items

Closes #1349 

---

- [x] I agree to contribute to the project under Apache 2 License. 
- [x] To the best of my knowledge, the proposed patch is not based on code under GPL or other license that is incompatible with FlexMeasures


Linked issue #1349: Only check status of important sensors
Now we select all sensors of an asset (in addition to sensors on the flex context), but that might become too many for the status page to be useful (and fast enough).

Ideally, we'd be able to set asset sensors which are important on a field sensors_to_monitor or so. 

But we can also start with using all sensors that are found on the asset's flex_context and sensors_to_show fields. Make also a TODO for supporting the flex_model field.

So using only sensors found in sensors_to_show and flex_context is the focus of this issue.

For the formatting of the sensor_to_show field, consult its schema or Joshua. Sensor values look like this: `{"sensor": 14}`. Maybe there is even already some util function to get a list of sensors mentioned in  `asset.senaors_to_show`

We should also add a small explanation at the top (within an info icon) as to how we chose the sensors.

Diff:
diff --git a/documentation/changelog.rst b/documentation/changelog.rst
index 94147ea9c7..be30a475fa 100644
--- a/documentation/changelog.rst
+++ b/documentation/changelog.rst
@@ -19,6 +19,7 @@ New features
 * Marker clusters on the dashboard map expand in a tree to show the hierarchical relationship of the assets they represent [see `PR #1410 <https://github.com/FlexMeasures/flexmeasures/pull/1410>`_]
 * Load the sensors individually on the Sensors status page. Reload the jobs table using Ajax calls. Improve page performance and avoid timeouts. [see `PR #1425 <https://github.com/FlexMeasures/flexmeasures/pull/1425>`_ and `PR #1466 <https://github.com/FlexMeasures/flexmeasures/pull/1466>`_]
 * New pages for `Properties`, `Graphs`, `Context`, `Status` and `Audit Log`. Simplified the main asset page. [see `PR #1416 <https://github.com/FlexMeasures/flexmeasures/pull/1416>`_, `PR #1387 <https://github.com/FlexMeasures/flexmeasures/pull/1387>`_ and `PR #1442 <https://github.com/FlexMeasures/flexmeasures/pull/1442>`_]
+* Only show important sensors statuses (flex-context and graph sensors) on the status page [see `PR #1439 <https://github.com/FlexMeasures/flexmeasures/pull/1439>`_]
 
 Infrastructure / Support
 ----------------------
diff --git a/flexmeasures/api/common/schemas/tests/test_sensor_data_schema.py b/flexmeasures/api/common/schemas/tests/test_sensor_data_schema.py
index 6bfe0a02fd..b1590937a8 100644
--- a/flexmeasures/api/common/schemas/tests/test_sensor_data_schema.py
+++ b/flexmeasures/api/common/schemas/tests/test_sensor_data_schema.py
@@ -401,7 +401,7 @@ def test_asset_sensors_metadata(
 
     status_data = get_asset_sensors_metadata(asset=asset)
 
-    assert status_data == [
+    assert status_data != [
         {
             "name": "wind speed",
             "id": wind_sensor.id,
@@ -419,6 +419,20 @@ def test_asset_sensors_metadata(
         },
     ]
 
+    # Make sure the Wind speed is not in the sensor data as it is not in sensors_to_show or flex-context
+    assert status_data == [
+        {
+            "name": "temperature",
+            "id": temperature_sensor.id,
+            "asset_name": asset.name,
+        },
+        {
+            "name": "production price",
+            "id": production_price_sensor.id,
+            "asset_name": battery_asset.name,
+        },
+    ]
+
 
 def custom_model_params():
     """little training as we have little data, turn off transformations until they let this test run (TODO)"""
diff --git a/flexmeasures/data/models/generic_assets.py b/flexmeasures/data/models/generic_assets.py
index 5124d3bffa..62d5f81294 100644
--- a/flexmeasures/data/models/generic_assets.py
+++ b/flexmeasures/data/models/generic_assets.py
@@ -130,6 +130,7 @@ def __repr__(self):
 
     def validate_sensors_to_show(
         self,
+        suggest_default_sensors: bool = True,
     ) -> list[dict[str, str | None | "Sensor" | list["Sensor"]]]:  # noqa: F821
         """
         Validate and transform the 'sensors_to_show' attribute into the latest format for use in graph-making code.
@@ -159,6 +160,10 @@ def validate_sensors_to_show(
                 [43, 44], 45, 46
             ]
 
+        Parameters:
+        - suggest_default_sensors: If True, the function will suggest default sensors if 'sensors_to_show' is not set.
+        - If False, the function will return an empty list if 'sensors_to_show' is not set.
+
         Returned structure:
         - The function returns a list of dictionaries, with each dictionary containing either a 'sensor' (for individual sensors) or 'sensors' (for groups of sensors), and an optional 'title'.
         - Example output:
@@ -171,11 +176,13 @@ def validate_sensors_to_show(
             ]
 
         If the 'sensors_to_show' attribute is missing, the function defaults to showing two of the asset's sensors, grouped together if they share the same unit, or separately if not.
+        If the suggest_default_sensors flag is set to False, the function will not suggest default sensors and will return an empty list if 'sensors_to_show' is not set.
 
+        Note:
         Unauthorized sensors are filtered out, and a warning is logged. Only sensors the user has permission to access are included in the final result.
         """
         # If not set, use defaults (show first 2 sensors)
-        if not self.sensors_to_show:
+        if not self.sensors_to_show and suggest_default_sensors:
             sensors_to_show = self.sensors[:2]
             if (
                 len(sensors_to_show) == 2
diff --git a/flexmeasures/data/services/sensors.py b/flexmeasures/data/services/sensors.py
index c7f26e3a02..f4e7a5b61d 100644
--- a/flexmeasures/data/services/sensors.py
+++ b/flexmeasures/data/services/sensors.py
@@ -290,25 +290,34 @@ def get_asset_sensors_metadata(
         if isinstance(asset.flex_context[field], dict)
         and field != "inflexible-device-sensors"
     }
-    for asset, is_child_asset in (
-        (asset, False),
-        *[(child_asset, True) for child_asset in asset.child_assets],
-    ):
-        sensors_list = list(asset.sensors)
-        if not is_child_asset:
-            sensors_list += [
-                *inflexible_device_sensors,
-                *context_sensors.values(),
-            ]
-        for sensor in sensors_list:
-            if sensor is None or sensor.id in sensor_ids:
-                continue
-            sensor_status = {}
-            sensor_status["id"] = sensor.id
-            sensor_status["name"] = sensor.name
-            sensor_status["asset_name"] = sensor.generic_asset.name
-            sensor_ids.add(sensor.id)
-            sensors.append(sensor_status)
+
+    # Get sensors to show using the validate_sensors_to_show method
+    sensors_to_show = []
+    validated_asset_sensors = asset.validate_sensors_to_show(
+        suggest_default_sensors=False
+    )
+    sensor_groups = [
+        sensor["sensors"] for sensor in validated_asset_sensors if sensor is not None
+    ]
+    merged_sensor_groups = sum(sensor_groups, [])
+    sensors_to_show.extend(merged_sensor_groups)
+
+    sensors_list = [
+        *inflexible_device_sensors,
+        *context_sensors.values(),
+        *sensors_to_show,
+    ]
+
+    for sensor in sensors_list:
+        if sensor is None or sensor.id in sensor_ids:
+            continue
+        sensor_status = {}
+        sensor_status["id"] = sensor.id
+        sensor_status["name"] = sensor.name
+        sensor_status["asset_name"] = sensor.generic_asset.name
+        sensor_ids.add(sensor.id)
+        sensors.append(sensor_status)
+
     return sensors
 
 
diff --git a/flexmeasures/ui/templates/views/status.html b/flexmeasures/ui/templates/views/status.html
index 07bea23080..942a1c3e6e 100644
--- a/flexmeasures/ui/templates/views/status.html
+++ b/flexmeasures/ui/templates/views/status.html
@@ -22,7 +22,9 @@ <h4>Data connectivity for sensors of {{ asset.name }}
 
 A red traffic light indicates that the last time a record was made was too long ago (it is 'stale'). Hover the light to learn more about why it is displayed red. It could be a source for errors down the line if that sensor data is necessary for forecasts and schedules to be made.
 
-How long ago is considered too long (stale) depends on the sensor, usually we use the sensor's resolution times two. If the source type means we expect future data ('forecaster', 'scheduler'), data should extend 12 hours into the future." style="margin-left: 10px;"></span>
+How long ago is considered too long (stale) depends on the sensor, usually we use the sensor's resolution times two. If the source type means we expect future data ('forecaster', 'scheduler'), data should extend 12 hours into the future.
+
+Sensors shown on this page are the ones relevant for correct functioning of the asset. We list the ones linked in the flex-context and the graphs page." style="margin-left: 10px;"></span>
                 </h4>
                 <table id="sensor_statuses" class="table table-striped paginate">
                     <thead>
@@ -82,8 +84,10 @@ <h4>Latest jobs of {{ asset.name }}
             "ordering": false
         });
 
-        // Append a loading row to the sensor status table
-        $('#sensor_statuses tbody').append(getLoadingRow());
+        if (sensors.length != 0) {
+            // Append a loading row to the sensor status table
+            $('#sensor_statuses tbody').append(getLoadingRow());
+        }
 
         sensors.forEach(element => {
             // Fetch and populate sensor data


Reviews:
user: login: nhoening
body: Good start!

We need to also work on "reason for listing", which now lists that the sensor is part of the asset, or flex context. This happens via `sensor.relation`, set in `data.services._get_sensor_asset_relation()` (above the metadata function, actually) - maybe we should rather manage `sensor.relation` right here in the metadata function, where we look them up....
state: CHANGES_REQUESTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-24T08:11:34Z
user: login: Muhammad-Moiz626
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-24T09:29:53Z
user: login: nhoening
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-24T14:36:37Z
user: login: Muhammad-Moiz626
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-24T21:36:38Z
user: login: Muhammad-Moiz626
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-24T21:39:48Z
user: login: joshuaunity
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-28T18:26:20Z
user: login: joshuaunity
body: I've tested this PR and confirmed that this works fine, although the use of `validate_sensors_to_show` in this use case may need to be reconsidered, as it will still return sensors even if non are set in the asset's sensors_to_show
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-04-30T10:57:19Z
user: login: nhoening
body: Thanks, this works great.
I believe you set an attribute to `True` when we mean `False`.

state: CHANGES_REQUESTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-05T13:57:13Z
user: login: Muhammad-Moiz626
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-05T14:48:32Z
user: login: nhoening
body: Thanks!
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-05-05T14:51:01Z

Review comments:
diff_hunk: @@ -290,25 +290,30 @@ def get_asset_sensors_metadata(
         if isinstance(asset.flex_context[field], dict)
         and field != "inflexible-device-sensors"
     }
-    for asset, is_child_asset in (
-        (asset, False),
-        *[(child_asset, True) for child_asset in asset.child_assets],
-    ):
-        sensors_list = list(asset.sensors)
-        if not is_child_asset:
-            sensors_list += [
-                *inflexible_device_sensors,
-                *context_sensors.values(),
-            ]
-        for sensor in sensors_list:
-            if sensor is None or sensor.id in sensor_ids:
-                continue
-            sensor_status = {}
-            sensor_status["id"] = sensor.id
-            sensor_status["name"] = sensor.name
-            sensor_status["asset_name"] = sensor.generic_asset.name
-            sensor_ids.add(sensor.id)
-            sensors.append(sensor_status)
+
+    # Get the sensors to show for the asset
+    sensors_to_show = []
+    for item in asset.sensors_to_show:
+        if isinstance(item, dict) and "sensors" in item:
+            for sensor in item["sensors"]:
+                field = Sensor.query.get(sensor)
path: flexmeasures/data/services/sensors.py
user: login: nhoening
body: Maybe only add if we actually found a Sensor
created_at: 2025-04-24T08:02:16Z
author_association: CONTRIBUTOR
diff_hunk: @@ -290,25 +290,30 @@ def get_asset_sensors_metadata(
         if isinstance(asset.flex_context[field], dict)
         and field != "inflexible-device-sensors"
     }
-    for asset, is_child_asset in (
-        (asset, False),
-        *[(child_asset, True) for child_asset in asset.child_assets],
-    ):
-        sensors_list = list(asset.sensors)
-        if not is_child_asset:
-            sensors_list += [
-                *inflexible_device_sensors,
-                *context_sensors.values(),
-            ]
-        for sensor in sensors_list:
-            if sensor is None or sensor.id in sensor_ids:
-                continue
-            sensor_status = {}
-            sensor_status["id"] = sensor.id
-            sensor_status["name"] = sensor.name
-            sensor_status["asset_name"] = sensor.generic_asset.name
-            sensor_ids.add(sensor.id)
-            sensors.append(sensor_status)
+
+    # Get the sensors to show for the asset
+    sensors_to_show = []
+    for item in asset.sensors_to_show:
+        if isinstance(item, dict) and "sensors" in item:
+            for sensor in item["sensors"]:
+                field = Sensor.query.get(sensor)
path: flexmeasures/data/services/sensors.py
user: login: nhoening
body: `field` is a weird name, I believe we're expecting a  `sensor` here.
created_at: 2025-04-24T08:02:51Z
author_association: CONTRIBUTOR
diff_hunk: @@ -290,25 +290,30 @@ def get_asset_sensors_metadata(
         if isinstance(asset.flex_context[field], dict)
         and field != "inflexible-device-sensors"
     }
-    for asset, is_child_asset in (
-        (asset, False),
-        *[(child_asset, True) for child_asset in asset.child_assets],
-    ):
-        sensors_list = list(asset.sensors)
-        if not is_child_asset:
-            sensors_list += [
-                *inflexible_device_sensors,
-                *context_sensors.values(),
-            ]
-        for sensor in sensors_list:
-            if sensor is None or sensor.id in sensor_ids:
-                continue
-            sensor_status = {}
-            sensor_status["id"] = sensor.id
-            sensor_status["name"] = sensor.name
-            sensor_status["asset_name"] = sensor.generic_asset.name
-            sensor_ids.add(sensor.id)
-            sensors.append(sensor_status)
+
+    # Get the sensors to show for the asset
+    sensors_to_show = []
+    for item in asset.sensors_to_show:
+        if isinstance(item, dict) and "sensors" in item:
+            for sensor in item["sensors"]:
+                field = Sensor.query.get(sensor)
+                sensors_to_show.append(field)
+
+    sensors_list = [
+        *inflexible_device_sensors,
path: flexmeasures/data/services/sensors.py
user: login: nhoening
body: These are now part of the flex context, so I don't believe we need to get them separately. This function also just looks them up on the flex context (even though its docstring still says it is looking upwards on the asset tree - maybe we can fix that docstring in this PR?).
created_at: 2025-04-24T08:08:12Z
author_association: CONTRIBUTOR
diff_hunk: @@ -290,25 +290,30 @@ def get_asset_sensors_metadata(
         if isinstance(asset.flex_context[field], dict)
         and field != "inflexible-device-sensors"
     }
-    for asset, is_child_asset in (
-        (asset, False),
-        *[(child_asset, True) for child_asset in asset.child_assets],
-    ):
-        sensors_list = list(asset.sensors)
-        if not is_child_asset:
-            sensors_list += [
-                *inflexible_device_sensors,
-                *context_sensors.values(),
-            ]
-        for sensor in sensors_list:
-            if sensor is None or sensor.id in sensor_ids:
-                continue
-            sensor_status = {}
-            sensor_status["id"] = sensor.id
-            sensor_status["name"] = sensor.name
-            sensor_status["asset_name"] = sensor.generic_asset.name
-            sensor_ids.add(sensor.id)
-            sensors.append(sensor_status)
+
+    # Get the sensors to show for the asset
+    sensors_to_show = []
+    for item in asset.sensors_to_show:
+        if isinstance(item, dict) and "sensors" in item:
+            for sensor in item["sensors"]:
+                field = Sensor.query.get(sensor)
+                sensors_to_show.append(field)
+
+    sensors_list = [
+        *inflexible_device_sensors,
path: flexmeasures/data/services/sensors.py
user: login: Muhammad-Moiz626
body: I'm not sure if that is the case. The flex context contain sensors in atleast two different places. On the database I have there are two places where sensors are stored in the JSON in flex-context. One is as list inside `inflexible-device-sensors` and the other is an integer value stored as `{"sensor": 65590}`.
created_at: 2025-04-24T09:29:53Z
author_association: CONTRIBUTOR
diff_hunk: @@ -290,25 +290,30 @@ def get_asset_sensors_metadata(
         if isinstance(asset.flex_context[field], dict)
         and field != "inflexible-device-sensors"
     }
-    for asset, is_child_asset in (
-        (asset, False),
-        *[(child_asset, True) for child_asset in asset.child_assets],
-    ):
-        sensors_list = list(asset.sensors)
-        if not is_child_asset:
-            sensors_list += [
-                *inflexible_device_sensors,
-                *context_sensors.values(),
-            ]
-        for sensor in sensors_list:
-            if sensor is None or sensor.id in sensor_ids:
-                continue
-            sensor_status = {}
-            sensor_status["id"] = sensor.id
-            sensor_status["name"] = sensor.name
-            sensor_status["asset_name"] = sensor.generic_asset.name
-            sensor_ids.add(sensor.id)
-            sensors.append(sensor_status)
+
+    # Get the sensors to show for the asset
+    sensors_to_show = []
+    for item in asset.sensors_to_show:
+        if isinstance(item, dict) and "sensors" in item:
+            for sensor in item["sensors"]:
+                field = Sensor.query.get(sensor)
+                sensors_to_show.append(field)
+
+    sensors_list = [
+        *inflexible_device_sensors,
path: flexmeasures/data/services/sensors.py
user: login: nhoening
body: I see, your loop is only selecting the sensors which are *not* inflexible devices (because those are not i a dict, but just a list. 
created_at: 2025-04-24T13:38:08Z
author_association: CONTRIBUTOR
diff_hunk: @@ -293,11 +293,13 @@ def get_asset_sensors_metadata(
 
     # Get the sensors to show for the asset
     sensors_to_show = []
-    for item in asset.sensors_to_show:
-        if isinstance(item, dict) and "sensors" in item:
-            for sensor in item["sensors"]:
-                field = Sensor.query.get(sensor)
-                sensors_to_show.append(field)
+    asset_sensors_to_show = asset.sensors_to_show
+    if asset_sensors_to_show is not None:
+        for item in asset.sensors_to_show:
+            if isinstance(item, dict) and "sensors" in item:
path: flexmeasures/data/services/sensors.py
user: login: nhoening
body: According to the docs, it is also possible that an item in the data is like this: `{"sensor": 4}`.
And I believe it can still be just IDs in some older installations, technically (which was the original format).

I asked Joshua if these can still be expected. Or maybe did you already ask the same question?

If so, I believe it is easier to get the sensors via `asset.validate_sensors_to_show()` (see [here](https://github.com/FlexMeasures/flexmeasures/blob/main/flexmeasures/data/models/generic_assets.py#L127)), as then you get the same kind of output every time.


created_at: 2025-04-24T14:36:21Z
author_association: CONTRIBUTOR
diff_hunk: @@ -290,25 +290,30 @@ def get_asset_sensors_metadata(
         if isinstance(asset.flex_context[field], dict)
         and field != "inflexible-device-sensors"
     }
-    for asset, is_child_asset in (
-        (asset, False),
-        *[(child_asset, True) for child_asset in asset.child_assets],
-    ):
-        sensors_list = list(asset.sensors)
-        if not is_child_asset:
-            sensors_list += [
-                *inflexible_device_sensors,
-                *context_sensors.values(),
-            ]
-        for sensor in sensors_list:
-            if sensor is None or sensor.id in sensor_ids:
-                continue
-            sensor_status = {}
-            sensor_status["id"] = sensor.id
-            sensor_status["name"] = sensor.name
-            sensor_status["asset_name"] = sensor.generic_asset.name
-            sensor_ids.add(sensor.id)
-            sensors.append(sensor_status)
+
+    # Get the sensors to show for the asset
+    sensors_to_show = []
+    for item in asset.sensors_to_show:
+        if isinstance(item, dict) and "sensors" in item:
+            for sensor in item["sensors"]:
+                field = Sensor.query.get(sensor)
+                sensors_to_show.append(field)
+
+    sensors_list = [
+        *inflexible_device_sensors,
path: flexmeasures/data/services/sensors.py
user: login: Muhammad-Moiz626
body: Actually this is the older code that is getting the sensors from flex-context for both the inflexible devices and also the other type of sensors in the `flex-context JSON`. I only added the sensors_to_show to it.
created_at: 2025-04-24T21:36:38Z
author_association: CONTRIBUTOR
diff_hunk: @@ -293,11 +293,13 @@ def get_asset_sensors_metadata(
 
     # Get the sensors to show for the asset
     sensors_to_show = []
-    for item in asset.sensors_to_show:
-        if isinstance(item, dict) and "sensors" in item:
-            for sensor in item["sensors"]:
-                field = Sensor.query.get(sensor)
-                sensors_to_show.append(field)
+    asset_sensors_to_show = asset.sensors_to_show
+    if asset_sensors_to_show is not None:
+        for item in asset.sensors_to_show:
+            if isinstance(item, dict) and "sensors" in item:
path: flexmeasures/data/services/sensors.py
user: login: Muhammad-Moiz626
body: I haven't actually asked Joshua about that. I'll ask him about this and let's see if the `asset.validate_sensors_to_show()` can give me the output I want.

I only asked him about the flex-context and sensors_to_show and he suggested I can get them using the two JSON values stored in the database.
created_at: 2025-04-24T21:39:48Z
author_association: CONTRIBUTOR
diff_hunk: @@ -290,25 +290,33 @@ def get_asset_sensors_metadata(
         if isinstance(asset.flex_context[field], dict)
         and field != "inflexible-device-sensors"
     }
-    for asset, is_child_asset in (
-        (asset, False),
-        *[(child_asset, True) for child_asset in asset.child_assets],
-    ):
-        sensors_list = list(asset.sensors)
-        if not is_child_asset:
-            sensors_list += [
-                *inflexible_device_sensors,
-                *context_sensors.values(),
-            ]
-        for sensor in sensors_list:
-            if sensor is None or sensor.id in sensor_ids:
-                continue
-            sensor_status = {}
-            sensor_status["id"] = sensor.id
-            sensor_status["name"] = sensor.name
-            sensor_status["asset_name"] = sensor.generic_asset.name
-            sensor_ids.add(sensor.id)
-            sensors.append(sensor_status)
+
+    # Get sensors to show using the validate_sensors_to_show method
+    sensors_to_show = []
+    validated_asset_sensors = asset.validate_sensors_to_show()
+    sensor_groups = [
+        sensor["sensors"] for sensor in validated_asset_sensors if sensor is not None
+    ]
+    for group_of_sensors in sensor_groups:
path: flexmeasures/data/services/sensors.py
user: login: joshuaunity
body: I have a suggestion for this part, we could use `sum`  instead of a nested loop. It should perform better that way.

```
merged_sensor_groups = sum(sensor_groups, [])
```


created_at: 2025-04-28T18:26:20Z
author_association: CONTRIBUTOR
diff_hunk: @@ -290,25 +290,34 @@ def get_asset_sensors_metadata(
         if isinstance(asset.flex_context[field], dict)
         and field != "inflexible-device-sensors"
     }
-    for asset, is_child_asset in (
-        (asset, False),
-        *[(child_asset, True) for child_asset in asset.child_assets],
-    ):
-        sensors_list = list(asset.sensors)
-        if not is_child_asset:
-            sensors_list += [
-                *inflexible_device_sensors,
-                *context_sensors.values(),
-            ]
-        for sensor in sensors_list:
-            if sensor is None or sensor.id in sensor_ids:
-                continue
-            sensor_status = {}
-            sensor_status["id"] = sensor.id
-            sensor_status["name"] = sensor.name
-            sensor_status["asset_name"] = sensor.generic_asset.name
-            sensor_ids.add(sensor.id)
-            sensors.append(sensor_status)
+
+    # Get sensors to show using the validate_sensors_to_show method
+    sensors_to_show = []
+    validated_asset_sensors = asset.validate_sensors_to_show(
+        suggest_default_sensors=True
path: flexmeasures/data/services/sensors.py
user: login: nhoening
body: ```suggestion
        suggest_default_sensors=False
```

I thought the whole idea was to set this to `False` at this point?
created_at: 2025-05-05T13:46:21Z
author_association: CONTRIBUTOR
diff_hunk: @@ -290,25 +290,34 @@ def get_asset_sensors_metadata(
         if isinstance(asset.flex_context[field], dict)
         and field != "inflexible-device-sensors"
     }
-    for asset, is_child_asset in (
-        (asset, False),
-        *[(child_asset, True) for child_asset in asset.child_assets],
-    ):
-        sensors_list = list(asset.sensors)
-        if not is_child_asset:
-            sensors_list += [
-                *inflexible_device_sensors,
-                *context_sensors.values(),
-            ]
-        for sensor in sensors_list:
-            if sensor is None or sensor.id in sensor_ids:
-                continue
-            sensor_status = {}
-            sensor_status["id"] = sensor.id
-            sensor_status["name"] = sensor.name
-            sensor_status["asset_name"] = sensor.generic_asset.name
-            sensor_ids.add(sensor.id)
-            sensors.append(sensor_status)
+
+    # Get sensors to show using the validate_sensors_to_show method
+    sensors_to_show = []
+    validated_asset_sensors = asset.validate_sensors_to_show(
+        suggest_default_sensors=True
path: flexmeasures/data/services/sensors.py
user: login: Muhammad-Moiz626
body: I added this change in the next commit with some addition in the test case as well.
created_at: 2025-05-05T14:48:32Z
author_association: CONTRIBUTOR

PR comments:
user: login: Muhammad-Moiz626
created_at: 2025-04-24T10:53:02Z
body: The reason is coming up when we call the sensors individually using ajax. The reason for not including it in the metadata is because the sensors we send in the first call is only used to fetch the ids of the sensors for the Ajax call.
author_association: CONTRIBUTOR
user: login: nhoening
created_at: 2025-04-24T13:54:20Z
body: > The reason is coming up when we call the sensors individually using ajax. The reason for not including it in the metadata is because the sensors we send in the first call is only used to fetch the ids of the sensors for the Ajax call.

Hm, do you have an idea how to refactor this, then?

We know the reason when loading the whole table (when finding the sensor in graph config or flex context), but you want that knowledge to be transferred in Ajax?

I wouldn't at first have thought about a lookup in Ajax call to check where that sensor is being mentioned, or did you?
author_association: CONTRIBUTOR
user: login: Muhammad-Moiz626
created_at: 2025-04-24T21:34:05Z
body: > > The reason is coming up when we call the sensors individually using ajax. The reason for not including it in the metadata is because the sensors we send in the first call is only used to fetch the ids of the sensors for the Ajax call.
> 
> Hm, do you have an idea how to refactor this, then?
> 
> We know the reason when loading the whole table (when finding the sensor in graph config or flex context), but you want that knowledge to be transferred in Ajax?
> 
> I wouldn't at first have thought about a lookup in Ajax call to check where that sensor is being mentioned, or did you?

I’m not entirely sure if refactoring is necessary in this case. Since we are loading the table through the Ajax call and not via the metadata, it seems more efficient to only focus on the sensor IDs during the initial page load. This approach helps ensure faster loading times. The full sensor details are populated during the Ajax call, which allows us to retrieve all the necessary information without delay.

However, if you still feel that we should include the `reason` during the initial load, could you please clarify where you envision that information being used? This will help me better understand the requirement and allow me to suggest an appropriate adjustment to the logic.
author_association: CONTRIBUTOR
user: login: Muhammad-Moiz626
created_at: 2025-04-30T14:40:52Z
body: Yes, that is the case that the `validate_sensors_to_show` method would return 2 sensors if it does not get any sensor in the `sensors_to_show` JSON.
author_association: CONTRIBUTOR
user: login: nhoening
created_at: 2025-05-03T12:08:36Z
body: I suggest that `validate_sensors_to_show()` gets a kw argument `suggest_default_sensors=True` and for this use case we set it to `False`.
author_association: CONTRIBUTOR
user: login: nhoening
created_at: 2025-05-03T12:09:21Z
body: Can you check the failing tests, @Muhammad-Moiz626 ?
author_association: CONTRIBUTOR