Repository: quintel/etsource
Pull request #3223: Custom curve flhs hydro river, solar thermal & geothermal
Author: login: kaskranenburgQ
Created: 2025-03-05T09:10:49Z | Merged: 2025-05-23T08:50:56Z

This PR aims to close #3119. 

It does so by updating the flh inputs for geothermal, solar thermal and hydro technologies. 
Please note that the produced energy of each of the technologies in the input statements will change with the flh's expect for **buildings_space_heater_solar_thermal** & **households_water_heater_solar_thermal**, since the energy production of these technologies is set via share inputs and not via a combintion of capaicty and flh's. 

Goes with:
- https://github.com/quintel/etengine/pull/1537
- https://github.com/quintel/etmodel/pull/4496
- https://github.com/quintel/etsource/pull/3223

Linked issue #3119: Solar thermal and geothermal custom curve full load hours don't work 
When uploading a custom curve for solar thermal and geothermal full load hours, full load hours of the relevant nodes are not updated. 

For solar thermal, part of the issue is that `energy_heat_solar_thermal` in the `flh_of_solar_thermal` input is outdated. These nodes are now subdivided in `energy_heat_solar_ht_solar_thermal`, `energy_heat_solar_mt_solar_thermal`, `energy_heat_solar_lt_solar_thermal`.

```
- query = UPDATE(V(energy_heat_solar_thermal, buildings_space_heater_solar_thermal, households_water_heater_solar_thermal), full_load_hours, USER_INPUT())
- priority = 3
- unit = hours
- update_period = future
```

Similarly, for `flh_of_geothermal_heat` the node `energy_heat_well_geothermal` is outdated. We currently have the nodes: `energy_heat_well_deep_ht_geothermal`, `energy_heat_well_deep_mt_geothermal`, `energy_heat_well_shallow_lt_geothermal` (and `energy_heat_well_shallow_heatpump_mt_geothermal`?) to be updated here. 

However, when I implement the correct node names, the full load hours still don't update. So there is another problem here which should be divided in further. 

Linked issue comments: user: login: github-actions[bot]
created_at: 2024-11-20T06:36:55Z
body: This issue has had no activity for 60 days and will be closed in 7 days. Removing the "Stale" label or posting a comment will prevent it from being closed automatically. You can also add the "Pinned" label to ensure it isn't marked as stale in the future.
author_association: NONE
user: login: mabijkerk
created_at: 2025-02-27T14:23:38Z
body: These FLH inputs are not listed in the front-end, but they are automatically triggered when a user upload the corresponding custom curve. You can see this in the [user_curves.yml](https://github.com/quintel/etsource/blob/master/config/user_curves.yml).

```yml
geothermal_heat:
  type: capacity_profile
  display_group: heat_production
  reduce:
    as: full_load_hours
    sets:
      - flh_of_geothermal_heat
```

- [ ] Update the geothermal, solar thermal and hydro to use the correct nodes
- [ ] Complete the inputs with min, max etc. and add description that they are used for custom curves
- [ ] Test if it works by uploading a custom curve for each technology
- [ ] Nice-to-have: update locales from "Extreme weather conditions" to "Weather conditions" and fix URLs
author_association: MEMBER

Diff:
diff --git a/config/energy_node_positions.yml b/config/energy_node_positions.yml
index f94563d166..baf3626620 100644
--- a/config/energy_node_positions.yml
+++ b/config/energy_node_positions.yml
@@ -1045,7 +1045,7 @@ energy_heat_well_shallow_heatpump_mt_geothermal:
   "y": 6980
 energy_heat_well_shallow_lt_geothermal:
   x: 6060
-  "y": 6200
+  "y": 6080
 energy_hydrogen_after_distribution:
   x: 5200
   "y": 8920
diff --git a/config/user_curves.yml b/config/user_curves.yml
index c7a4eb3b1d..1349fcda0a 100644
--- a/config/user_curves.yml
+++ b/config/user_curves.yml
@@ -133,7 +133,7 @@ interconnector_1_export_must_run:
   display_group: must_run_interconnectors
   reduce:
     as: full_load_hours
-    sets: 
+    sets:
      - external_coupling_interconnector_1_flh_of_export_user_curve
 
 interconnector_2_price:
@@ -653,15 +653,23 @@ weather/solar_thermal:
   reduce:
     as: full_load_hours
     sets:
-      - flh_of_solar_thermal
+      - flh_of_solar_thermal_user_curve
+
+geothermal_heat_shallow:
+  type: capacity_profile
+  display_group: heat_production
+  reduce:
+    as: full_load_hours
+    sets:
+      - flh_of_geothermal_heat_shallow_user_curve
 
-geothermal_heat:
+geothermal_heat_deep:
   type: capacity_profile
   display_group: heat_production
   reduce:
     as: full_load_hours
     sets:
-      - flh_of_geothermal_heat
+      - flh_of_geothermal_heat_deep_user_curve
 
 river:
   type: capacity_profile
@@ -669,7 +677,7 @@ river:
   reduce:
     as: full_load_hours
     sets:
-      - flh_of_hydro_river
+      - flh_of_hydro_river_user_curve
 
 weather/air_temperature:
   type: temperature
diff --git a/inputs/full_load_hours/flh_of_geothermal_heat.ad b/inputs/full_load_hours/flh_of_geothermal_heat.ad
deleted file mode 100644
index 6ba17155ab..0000000000
--- a/inputs/full_load_hours/flh_of_geothermal_heat.ad
+++ /dev/null
@@ -1,4 +0,0 @@
-- query = UPDATE(V(energy_heat_well_geothermal, industry_heat_well_geothermal, agriculture_geothermal), full_load_hours, USER_INPUT())
-- priority = 3
-- unit = hours
-- update_period = future
diff --git a/inputs/full_load_hours/flh_of_geothermal_heat_deep_user_curve.ad b/inputs/full_load_hours/flh_of_geothermal_heat_deep_user_curve.ad
new file mode 100644
index 0000000000..de6f92f03f
--- /dev/null
+++ b/inputs/full_load_hours/flh_of_geothermal_heat_deep_user_curve.ad
@@ -0,0 +1,36 @@
+# This input updates the FLHs of deep geothermal heat technologies when a custom curve is uploaded,
+# see configuration in user_curves.yml.
+# FLH of agriculture_geothermal are updated in a different manner since users
+# set the share of this technology in heat demand, whereas for the other technologies the
+# output capacity can be set. Therefore, the method for changing the FLHs of agricultural geothermal
+# will change the number of units and generates the same heat output, whereas for the other technologies
+# the number of units stay fixed and the heat output changes as result of changed FLHs.
+
+- query =
+    original_nou_energy_heat_well_deep_ht_geothermal =                      V(energy_heat_well_deep_ht_geothermal,      number_of_units);
+    original_nou_energy_heat_well_deep_mt_geothermal =                      V(energy_heat_well_deep_mt_geothermal,      number_of_units);
+    original_nou_industry_heat_well_geothermal =                            V(industry_heat_well_geothermal,            number_of_units);
+
+    EACH(
+      UPDATE(V(energy_heat_well_deep_ht_geothermal), full_load_hours, USER_INPUT()),
+      UPDATE(V(energy_heat_well_deep_ht_geothermal), number_of_units, original_nou_energy_heat_well_deep_ht_geothermal),
+      UPDATE(V(energy_heat_well_deep_ht_geothermal), preset_demand_by_steam_hot_water_production, V(energy_heat_well_deep_ht_geothermal, steam_hot_water_production_based_on_number_of_units)),
+
+      UPDATE(V(energy_heat_well_deep_mt_geothermal), full_load_hours, USER_INPUT()),
+      UPDATE(V(energy_heat_well_deep_mt_geothermal), number_of_units, original_nou_energy_heat_well_deep_mt_geothermal),
+      UPDATE(V(energy_heat_well_deep_mt_geothermal), preset_demand_by_steam_hot_water_production, V(energy_heat_well_deep_mt_geothermal, steam_hot_water_production_based_on_number_of_units)),
+
+      UPDATE(V(industry_heat_well_geothermal), full_load_hours, USER_INPUT()),
+      UPDATE(V(industry_heat_well_geothermal), number_of_units, original_nou_industry_heat_well_geothermal),
+      UPDATE(V(industry_heat_well_geothermal), preset_demand_by_steam_hot_water_production, V(industry_heat_well_geothermal, steam_hot_water_production_based_on_number_of_units)),
+
+      UPDATE(V(agriculture_geothermal), full_load_hours, USER_INPUT())
+    )
+
+- priority = 1
+- max_value = 8760.0
+- min_value = 0.0
+- start_value_gql = present:V(agriculture_geothermal, full_load_hours)
+- step_value = 1.0
+- unit = hours
+- update_period = future
diff --git a/inputs/full_load_hours/flh_of_geothermal_heat_shallow_user_curve.ad b/inputs/full_load_hours/flh_of_geothermal_heat_shallow_user_curve.ad
new file mode 100644
index 0000000000..0621d1dc3b
--- /dev/null
+++ b/inputs/full_load_hours/flh_of_geothermal_heat_shallow_user_curve.ad
@@ -0,0 +1,25 @@
+# This input updates the FLHs of shallow geothermal heat technologies when a custom curve is uploaded,
+# see configuration in user_curves.yml. When a custom curve is uploaded, the heat output of these
+# technologies changes whereas the number of units stay fixed.
+
+- query =
+    original_nou_energy_heat_well_shallow_lt_geothermal =                   V(energy_heat_well_shallow_lt_geothermal,             number_of_units);
+    original_nou_energy_heat_well_shallow_heatpump_mt_geothermal =          V(energy_heat_well_shallow_heatpump_mt_geothermal,    number_of_units);
+
+    EACH(
+      UPDATE(V(energy_heat_well_shallow_lt_geothermal), full_load_hours, USER_INPUT()),
+      UPDATE(V(energy_heat_well_shallow_lt_geothermal), number_of_units, original_nou_energy_heat_well_shallow_lt_geothermal),
+      UPDATE(V(energy_heat_well_shallow_lt_geothermal), preset_demand_by_steam_hot_water_production, V(energy_heat_well_shallow_lt_geothermal, steam_hot_water_production_based_on_number_of_units)),
+
+      UPDATE(V(energy_heat_well_shallow_heatpump_mt_geothermal), full_load_hours, USER_INPUT()),
+      UPDATE(V(energy_heat_well_shallow_heatpump_mt_geothermal), number_of_units, original_nou_energy_heat_well_shallow_heatpump_mt_geothermal),
+      UPDATE(V(energy_heat_well_shallow_heatpump_mt_geothermal), preset_demand_by_steam_hot_water_production, V(energy_heat_well_shallow_heatpump_mt_geothermal, steam_hot_water_production_based_on_number_of_units)),
+    )
+
+- priority = 1
+- max_value = 8760.0
+- min_value = 0.0
+- start_value_gql = present:V(agriculture_geothermal, full_load_hours)
+- step_value = 1.0
+- unit = hours
+- update_period = future
diff --git a/inputs/full_load_hours/flh_of_hydro_river.ad b/inputs/full_load_hours/flh_of_hydro_river.ad
deleted file mode 100644
index cf7140cecb..0000000000
--- a/inputs/full_load_hours/flh_of_hydro_river.ad
+++ /dev/null
@@ -1,4 +0,0 @@
-- query = UPDATE(V(energy_power_hydro_river), full_load_hours, USER_INPUT())
-- priority = 3
-- unit = hours
-- update_period = future
diff --git a/inputs/full_load_hours/flh_of_hydro_river_user_curve.ad b/inputs/full_load_hours/flh_of_hydro_river_user_curve.ad
new file mode 100644
index 0000000000..682594f15f
--- /dev/null
+++ b/inputs/full_load_hours/flh_of_hydro_river_user_curve.ad
@@ -0,0 +1,19 @@
+# This input updates the FLHs of hydro river
+# These FLH inputs are not listed in the front-end, but is automatically triggered when a user uploads the corresponding custom curve
+# You can see this in the user_curves.yml
+
+- query =
+    original_nou = V(energy_power_hydro_river, number_of_units);
+
+    EACH(
+      UPDATE(V(energy_power_hydro_river), full_load_hours, USER_INPUT()),
+      UPDATE(V(energy_power_hydro_river), number_of_units, original_nou),
+      UPDATE(V(energy_power_hydro_river), preset_demand_by_electricity_production, V(energy_power_hydro_river, production_based_on_number_of_units)),
+    )
+- priority = 1
+- max_value = 8760.0
+- min_value = 0.0
+- start_value_gql = present:V(energy_power_hydro_river, full_load_hours)
+- step_value = 1.0
+- unit = hours
+- update_period = future
diff --git a/inputs/full_load_hours/flh_of_solar_thermal.ad b/inputs/full_load_hours/flh_of_solar_thermal.ad
deleted file mode 100644
index 0143cb0f04..0000000000
--- a/inputs/full_load_hours/flh_of_solar_thermal.ad
+++ /dev/null
@@ -1,4 +0,0 @@
-- query = UPDATE(V(energy_heat_solar_thermal, buildings_space_heater_solar_thermal, households_water_heater_solar_thermal), full_load_hours, USER_INPUT())
-- priority = 3
-- unit = hours
-- update_period = future
diff --git a/inputs/full_load_hours/flh_of_solar_thermal_user_curve.ad b/inputs/full_load_hours/flh_of_solar_thermal_user_curve.ad
new file mode 100644
index 0000000000..ae38b8942a
--- /dev/null
+++ b/inputs/full_load_hours/flh_of_solar_thermal_user_curve.ad
@@ -0,0 +1,38 @@
+# This input updates the FLHs of solar thermal technologies when a custom curve is uploaded,
+# see configuration in user_curves.yml.
+# FLH of solar thermal in buildings and households are updated in a different manner since users
+# set shares of this technology, whereas for the other technologies the
+# output capacity can be set. Therefore, the method for changing the FLHs of solar thermal in
+# buildings and households will change the number of units and generates the same heat output,
+# whereas for the other technologies the number of units stay fixed and the heat output changes
+# as result of changed FLHs.
+
+- query =
+    original_nou_energy_heat_solar_ht_solar_thermal =             V(energy_heat_solar_ht_solar_thermal,      number_of_units);
+    original_nou_energy_heat_solar_lt_solar_thermal =             V(energy_heat_solar_lt_solar_thermal,      number_of_units);
+    original_nou_energy_heat_solar_mt_solar_thermal =             V(energy_heat_solar_mt_solar_thermal,      number_of_units);
+
+    EACH(
+      UPDATE(V(energy_heat_solar_ht_solar_thermal), full_load_hours, USER_INPUT()),
+      UPDATE(V(energy_heat_solar_ht_solar_thermal), number_of_units, original_nou_energy_heat_solar_ht_solar_thermal),
+      UPDATE(V(energy_heat_solar_ht_solar_thermal), preset_demand_by_steam_hot_water_production, V(energy_heat_solar_ht_solar_thermal, steam_hot_water_production_based_on_number_of_units)),
+
+      UPDATE(V(energy_heat_solar_lt_solar_thermal), full_load_hours, USER_INPUT()),
+      UPDATE(V(energy_heat_solar_lt_solar_thermal), number_of_units, original_nou_energy_heat_solar_lt_solar_thermal),
+      UPDATE(V(energy_heat_solar_lt_solar_thermal), preset_demand_by_steam_hot_water_production, V(energy_heat_solar_lt_solar_thermal, steam_hot_water_production_based_on_number_of_units)),
+
+      UPDATE(V(energy_heat_solar_mt_solar_thermal), full_load_hours, USER_INPUT()),
+      UPDATE(V(energy_heat_solar_mt_solar_thermal), number_of_units, original_nou_energy_heat_solar_mt_solar_thermal),
+      UPDATE(V(energy_heat_solar_mt_solar_thermal), preset_demand_by_steam_hot_water_production, V(energy_heat_solar_mt_solar_thermal, steam_hot_water_production_based_on_number_of_units)),
+
+      UPDATE(V(buildings_space_heater_solar_thermal), full_load_hours, USER_INPUT()),
+      UPDATE(V(households_water_heater_solar_thermal), full_load_hours, USER_INPUT()),
+    )
+
+- priority = 1
+- max_value = 8760.0
+- min_value = 0.0
+- start_value_gql = present:V(households_water_heater_solar_thermal, full_load_hours)
+- step_value = 1.0
+- unit = hours
+- update_period = future


Reviews:
user: login: kndehaan
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-09T14:05:39Z
user: login: kndehaan
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-09T14:07:25Z
user: login: kndehaan
body: Nice work! 
state: APPROVED
author_association: MEMBER
submitted_at: 2025-05-19T08:43:07Z

Review comments:
diff_hunk: @@ -0,0 +1,42 @@
+# This input updates the FLHs of geothermal heat
+# These FLH inputs are not listed in the front-end, but is automatically triggered when a user uploads the corresponding custom curve
+# You can see this in the user_curves.yml
+
+- query =
+    original_nou_energy_heat_well_shallow_lt_geothermal =                   V(energy_heat_well_shallow_lt_geothermal,      number_of_units);
+    original_nou_energy_heat_well_deep_ht_geothermal =                      V(energy_heat_well_deep_ht_geothermal,      number_of_units);
+    original_nou_energy_heat_well_deep_mt_geothermal =                      V(energy_heat_well_deep_mt_geothermal,      number_of_units);
+    original_nou_energy_heat_well_shallow_heatpump_mt_geothermal =          V(energy_heat_well_shallow_heatpump_mt_geothermal,    number_of_units);
+    original_nou_industry_heat_well_geothermal =                            V(industry_heat_well_geothermal,   number_of_units);
+
+    EACH(
+      UPDATE(V(energy_heat_well_shallow_lt_geothermal), full_load_hours, USER_INPUT()),
+      UPDATE(V(energy_heat_well_shallow_lt_geothermal), number_of_units, original_nou_energy_heat_well_shallow_lt_geothermal),
+      UPDATE(V(energy_heat_well_shallow_lt_geothermal), preset_demand_by_steam_hot_water_production, V(energy_heat_well_shallow_lt_geothermal, steam_hot_water_production_based_on_number_of_units)),
+
+      UPDATE(V(energy_heat_well_deep_ht_geothermal), full_load_hours, USER_INPUT()),
+      UPDATE(V(energy_heat_well_deep_ht_geothermal), number_of_units, original_nou_energy_heat_well_deep_ht_geothermal),
+      UPDATE(V(energy_heat_well_deep_ht_geothermal), preset_demand_by_steam_hot_water_production, V(energy_heat_well_deep_ht_geothermal, steam_hot_water_production_based_on_number_of_units)),
+
+      UPDATE(V(energy_heat_well_deep_mt_geothermal), full_load_hours, USER_INPUT()),
+      UPDATE(V(energy_heat_well_deep_mt_geothermal), number_of_units, original_nou_energy_heat_well_deep_mt_geothermal),
+      UPDATE(V(energy_heat_well_deep_mt_geothermal), preset_demand_by_steam_hot_water_production, V(energy_heat_well_deep_mt_geothermal, steam_hot_water_production_based_on_number_of_units)),
+
+      UPDATE(V(energy_heat_well_shallow_heatpump_mt_geothermal), full_load_hours, USER_INPUT()),
+      UPDATE(V(energy_heat_well_shallow_heatpump_mt_geothermal), number_of_units, original_nou_energy_heat_well_shallow_heatpump_mt_geothermal),
+      UPDATE(V(energy_heat_well_shallow_heatpump_mt_geothermal), preset_demand_by_steam_hot_water_production, V(energy_heat_well_shallow_heatpump_mt_geothermal, steam_hot_water_production_based_on_number_of_units)),
+
+      UPDATE(V(industry_heat_well_geothermal), full_load_hours, USER_INPUT()),
+      UPDATE(V(industry_heat_well_geothermal), number_of_units, original_nou_industry_heat_well_geothermal),
+      UPDATE(V(industry_heat_well_geothermal), preset_demand_by_steam_hot_water_production, V(industry_heat_well_geothermal, steam_hot_water_production_based_on_number_of_units)),
+
+      UPDATE(V(agriculture_geothermal), full_load_hours, USER_INPUT())
path: inputs/full_load_hours/flh_of_geothermal_heat_user_curve.ad
user: login: kndehaan
body: Was deliberately chosen to not apply the same method of setting the FLH for `agriculture_geothermal`?
created_at: 2025-05-09T14:05:38Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,33 @@
+# This input updates the FLHs of solar thermal
+# These FLH inputs are not listed in the front-end, but is automatically triggered when a user uploads the corresponding custom curve
+# You can see this in the user_curves.yml
+
+- query =
+    original_nou_energy_heat_solar_ht_solar_thermal =             V(energy_heat_solar_ht_solar_thermal,      number_of_units);
+    original_nou_energy_heat_solar_lt_solar_thermal =             V(energy_heat_solar_lt_solar_thermal,      number_of_units);
+    original_nou_energy_heat_solar_mt_solar_thermal =             V(energy_heat_solar_mt_solar_thermal,      number_of_units);
+    
+    EACH(
+      UPDATE(V(energy_heat_solar_ht_solar_thermal), full_load_hours, USER_INPUT()),
+      UPDATE(V(energy_heat_solar_ht_solar_thermal), number_of_units, original_nou_energy_heat_solar_ht_solar_thermal),
+      UPDATE(V(energy_heat_solar_ht_solar_thermal), preset_demand_by_steam_hot_water_production, V(energy_heat_solar_ht_solar_thermal, steam_hot_water_production_based_on_number_of_units)),
+
+      UPDATE(V(energy_heat_solar_lt_solar_thermal), full_load_hours, USER_INPUT()),
+      UPDATE(V(energy_heat_solar_lt_solar_thermal), number_of_units, original_nou_energy_heat_solar_lt_solar_thermal),
+      UPDATE(V(energy_heat_solar_lt_solar_thermal), preset_demand_by_steam_hot_water_production, V(energy_heat_solar_lt_solar_thermal, steam_hot_water_production_based_on_number_of_units)),
+
+      UPDATE(V(energy_heat_solar_mt_solar_thermal), full_load_hours, USER_INPUT()),
+      UPDATE(V(energy_heat_solar_mt_solar_thermal), number_of_units, original_nou_energy_heat_solar_mt_solar_thermal),
+      UPDATE(V(energy_heat_solar_mt_solar_thermal), preset_demand_by_steam_hot_water_production, V(energy_heat_solar_mt_solar_thermal, steam_hot_water_production_based_on_number_of_units)),
+
+      UPDATE(V(buildings_space_heater_solar_thermal), full_load_hours, USER_INPUT()),
+      UPDATE(V(households_water_heater_solar_thermal), full_load_hours, USER_INPUT()),
path: inputs/full_load_hours/flh_of_solar_thermal_user_curve.ad
user: login: kndehaan
body: Same question for `buildings_space_heater_solar_thermal` and `households_water_heater_solar_thermal`, not necessary here to apply the same method?
created_at: 2025-05-09T14:07:25Z
author_association: MEMBER

PR comments:
user: login: mabijkerk
created_at: 2025-03-07T13:28:51Z
body: @kaskranenburgQ will this change model results? If so, we cannot merge it without discussing it with our clients.
author_association: MEMBER
user: login: kndehaan
created_at: 2025-03-07T15:35:26Z
body: Just did a quick scan of the changes. In addition to Mathijs' question and remark, I think we also need to write a migration, since the inputs that will be deleted with this PR can be set in scenarios... Let's discuss in the sprint.
author_association: MEMBER
user: login: kaskranenburgQ
created_at: 2025-03-11T10:54:44Z
body: In the modeler sprint we discussed whether a scenario migration was necessary for the changes that were made in these inputs. We concluded that an investigation about the current working of the input statements was necessary: Do these inputs work when you set them in the backend?  And do they work when we upload a curve?

I found that the input statement does work. If  set:
`flh_of_geothermal_heat: 1000` in a blank nl2019 scenario on master, I see that the full load hours of the node **agriculture_geothermal** will be changed to 1000.0:
![Screenshot 2025-03-11 at 11 40 57](https://github.com/user-attachments/assets/8a85530f-88b7-4452-a2b0-3599142ca3c7)


However, when I then upload a capacity profile via the front-end:

![Screenshot 2025-03-11 at 11 45 25](https://github.com/user-attachments/assets/234c9834-63f5-4117-a0f8-ea791b4a684e)

The flh setting is removed from the inputs:
![Screenshot 2025-03-11 at 11 46 02](https://github.com/user-attachments/assets/160db3e9-0ded-4c07-9b82-de286e64fe94)

And the full load hours of agriculture_geothermal is reverted back to it's orginal value:
![Screenshot 2025-03-11 at 11 46 22](https://github.com/user-attachments/assets/cd257a18-3ef4-411d-bf94-3ba846dc0e28)


To conclude: 
- The flh inputs work as a back-end input
- Uploading a curve is not functional and resets the input when it is set. 
- Therefore there is a great chance that users who 'accidentally' have set the input have overwritten the input with a curve. 
- We should check if this is the case.

author_association: CONTRIBUTOR
user: login: louispt1
created_at: 2025-03-12T15:58:58Z
body: I had a look at the scenarios through the server. I couldn't always parse the user_inputs for older scenarios for this hash serialisation issue for some of the values, especially in older scenarios:
`Tried to load unspecified class: Rational (Psych::DisallowedClass)`

But from the scenarios which I could check, and definitely for all the scenarios in the etengine pro backup, none of these inputs are set:
- flh_of_solar_thermal
- flh_of_geothermal_heat
- flh_of_hydro_river

author_association: MEMBER
user: login: kndehaan
created_at: 2025-03-14T11:18:28Z
body: The user curve for hydro river works as it should! 

There's an issue in the geothermal and solar thermal user curve in scenarios where there's already installed capacity of related technologies in the start year. For example, take the a blank EU27 scenario and upload custom curves for solar thermal and geothermal. This correctly changes the full load hours, but incorrectly updates the number of units and incorrectly doesn't affect the production output, see example below for two technologies. 

<img width="975" alt="image" src="https://github.com/user-attachments/assets/f14dbf25-c41f-4861-a2bc-d50d513aa5e5" />

I think this is because the flh_..._user_curve input should include a correction where the original NoU is set and preset demand is updated based `production_based_on_number_of_units`, just like the other flh inputs. In the electricity production technologies, this is done with `preset_demand_by_electricity_production`. For steam hot water production, this method doesn't exist. 

Is there another way to tackle this with the methods we already have? Or do we need to create this method for steam hot water as well? @kaskranenburgQ @mabijkerk 

author_association: MEMBER
user: login: kaskranenburgQ
created_at: 2025-03-19T17:03:19Z
body: Great catch! 
Let's discuss this with @mabijkerk.
author_association: CONTRIBUTOR
user: login: kaskranenburgQ
created_at: 2025-04-10T10:13:04Z
body: Based on the PR of @louispt1 (https://github.com/quintel/etengine/pull/1537), I have added the preset_demand_by_steam_hot_water_production update in the user curve inputs. While testing I found the following points:

1. I have implemented the input statement based on 'production_based_on_number_of_units'. This does not work since this function in turn is based on the typical_electricity_production_per_unit. For hydrogen producers (i.e. for energy_hydrogen_electrolysis_solar_electricity in the input flh_of_solar_pv_solar_radiation) we use hydrogen_production_based_on_number_of_units. In order to be able to update the preset_demand_by_steam_hot_water_production correctly, I propose we also add a function 'steam_hot_water_production_based_on_number_of_untis. 
2. While uploading a curve for solar thermal a huge flow from energy to heat occurs, see picture below:
![Energy flow overview 1289457 (1)](https://github.com/user-attachments/assets/382ac10f-352d-4cba-8d46-3e0902f6751d), it could be that this occurs due to the issue described in point 1. We should check whether this still occurs when the fix for point is implemented.

@louispt1 Could you add this in your PR? Then I will change the inputs to accomodate this change.

author_association: CONTRIBUTOR
user: login: louispt1
created_at: 2025-04-10T13:50:44Z
body: @kaskranenburgQ I've added the method, untested except for the pre-existing spec: https://github.com/quintel/etengine/pull/1537/commits/b62b0377a45fc399a45e4a0805c67d6e366ca55c 
Let me know if there are any issues!
author_association: MEMBER
user: login: kaskranenburgQ
created_at: 2025-05-08T07:36:46Z
body: After a rebase of the engine branch the issue has been solved. @kndehaan Could you do one final review?
author_association: CONTRIBUTOR
user: login: kndehaan
created_at: 2025-05-09T14:17:01Z
body: An uploaded curve for `flh_of_geothermal_heat_user_curve` will set the full load hours of all geothermal nodes to the same full load hours. However, the default full load hours vary per technology (not dataset-dependent):

```
energy_heat_well_shallow_lt_geothermal: 4750
energy_heat_well_deep_mt_geothermal: 6000
energy_heat_well_shallow_heatpump_mt_geothermal: 4750
energy_heat_well_deep_ht_geothermal: 6000
industry_heat_well_geothermal: 7000
agriculture_geothermal: 6000
```

So, wouldn't it be undesirable that with one curve, all technologies' full load hours are updated to the same value? 
author_association: MEMBER
user: login: kndehaan
created_at: 2025-05-12T14:27:23Z
body: The custom curve upload works well now for all technologies that have steam_hot_water output, thus the new methods `preset_demand_by_steam_hot_water_production` and `steam_hot_water_production_based_on_number_of_units` work as expected. 

However, the same problem (not updating production but incorrectly updating nr of units as mentioned [here](https://github.com/quintel/etsource/pull/3223#issuecomment-2724373131)) still exists for all technologies with useable_heat output. More specifically, for:

```
agriculture_geothermal
buildings_space_heater_solar_thermal
households_water_heater_solar_thermal
```

I tried to apply the same method here as is done for the other technologies (including copying the method in ETEngine as done in https://github.com/quintel/etengine/pull/1537 for useable_heat). For it to work, it requires to add `preset_demand` to the node groups. However, I'm not so certain whether these nodes represent preset demand. 

So question is, are these nodes preset demand? If so, we could add `preset_demand` to the node groups and create and apply the same method for useable_heat as well.
author_association: MEMBER
user: login: kaskranenburgQ
created_at: 2025-05-13T15:00:47Z
body: The following to-do's are still open before we can merge this:

- [ ] Split flh_of_geothermal_user_curve into inputs for shallow & deep geothermal. 
- [x] Write migration to remove outdated inputs from scenario's --> With the comment of @louispt1 in mind: https://github.com/quintel/etsource/pull/3223#issuecomment-2718367704, this might not be explicitly necessary.
- [ ] Change documentation in files flh_of_solar_thermal_user_curve & flh_of_geothermal_user_curve, so that it's clear why agriculture_geothermal, buildings_space_heater_solar_thermal, and households_water_heater_solar_thermal have a different update structure. 
- [x] Check whether number of units of agriculture_geothermal, buildings_space_heater_solar_thermal, and households_water_heater_solar_thermal are updated correctly in the current structure

These changes go together with these PR's:

ETModel: https://github.com/quintel/etmodel/pull/4496
ETEngien: https://github.com/quintel/etengine/pull/1537

author_association: CONTRIBUTOR
user: login: louispt1
created_at: 2025-05-14T08:40:46Z
body: I think it would be best practice to still include a migration to remove the outdated inputs - I only checked on a local pro db
author_association: MEMBER