Repository: FlexMeasures/flexmeasures (python)

Issue:
Allow to specify `soc-maxima` and `soc-minima` as a range
> Side-question from #885: Wouldn't it be nice to allow to specify ``soc-maxima`` (and minima of course) as a range, so one would not have to send all consecutive points?
> 
 > ```
> "soc-maxima": [
>     {
>         "value": 51,
>         "start": "2024-02-05T12:00:00+01:00",
>         "end": "2024-02-05T13:30:00+01:00"
>     }
> ]
> ```
> 
> One problem I see is overlaps but that should cause an error.

_Originally posted by @nhoening in https://github.com/FlexMeasures/flexmeasures/pull/885#discussion_r1375131453_
            

Pull request:
Allow to specify soc-maxima and soc-minima as a range
## Description

Non-breaking API change to support less verbose ways of setting the same constraint for a given time window.

## Look & Feel

```
{
    "flex-model": {
        "soc-maxima": [
            {
                "value": 50,
                "datetime": "2024-02-05T11:00:00+01:00",
            },
            {
                "value": 51,
                "start": "2024-02-05T12:00:00+01:00",
                "end": "2024-02-05T13:30:00+01:00"
            },
            {
                "value": 52,
                "start": "2024-02-05T15:00:00+01:00",
                "duration": "PT1H"
            },
            {
                "value": 53,
                "duration": "PT15M",
                "end": "2024-02-05T18:30:00+01:00"
            }
        ]
    }
}
```

## How to test

- [x] Still need to check the flexmeasures-client for compatibility.

## Further Improvements

- [x] Update V2G documentation page
- [x] Update API notation page [not updated, as it points to the V2G documentation page for examples.]
- [x] Test whether a `P1D` duration over a DST transition takes into account the sensor's IANA timezone [with [4db562d](https://github.com/FlexMeasures/flexmeasures/pull/899/commits/4db562d988968dc101a31709da273696007f66be) we now ground the nominal durations to the start or end, and I've added tests for `P1M` durations. However, I commented out the `P1D` and `P1W` test cases, as they require a fundamental fix in isodate.]

## Related Items

Closes #898.

---

- [x] I agree to contribute to the project under Apache 2 License. 
- [x] To the best of my knowledge, the proposed patch is not based on code under GPL or other license that is incompatible with FlexMeasures


Patch:
diff --git a/documentation/api/change_log.rst b/documentation/api/change_log.rst
index b23f66f2..4f3692f9 100644
--- a/documentation/api/change_log.rst
+++ b/documentation/api/change_log.rst
@@ -5,6 +5,11 @@ API change log
 
 .. note:: The FlexMeasures API follows its own versioning scheme. This is also reflected in the URL (e.g. `/api/v3_0`), allowing developers to upgrade at their own pace.
 
+v3.0-15 | 2024-01-11
+""""""""""""""""""""
+
+- Support setting SoC constraints in the flex model for a given time period rather than a single datetime, using the new ``start``, ``end`` and/or ``duration`` fields of ``soc-maxima``, ``soc-minima`` and ``soc-targets``.
+
 v3.0-14 | 2023-12-07
 """"""""""""""""""""
 
diff --git a/documentation/api/v2g.rst b/documentation/api/v2g.rst
index e111d04e..ee8363d1 100644
--- a/documentation/api/v2g.rst
+++ b/documentation/api/v2g.rst
@@ -52,20 +52,8 @@ To enable a temporary target SoC of more than 85% (for car reservations, see the
             "soc-maxima": [
                 {
                     "value": 51,
-                    "datetime": "2024-02-04T10:35:00+01:00"
-                },
-                {
-                    "value": 51,
-                    "datetime": "2024-02-04T10:40:00+01:00"
-                },
-                ...
-                {
-                    "value": 51,
-                    "datetime": "2024-02-05T04:20:00+01:00"
-                },
-                {
-                    "value": 51,
-                    "datetime": "2024-02-05T04:25:00+01:00"
+                    "start": "2024-02-04T10:35:00+01:00",
+                    "end": "2024-02-05T04:25:00+01:00"
                 }
             ],
             "soc-unit": "kWh"
@@ -106,19 +94,8 @@ To make sure the car remains at 95% SoC for some time, additional soc-minima con
             "soc-minima": [
                 {
                     "value": 57,
-                    "datetime": "2024-02-05T08:00:00+01:00"
-                },
-                {
-                    "value": 57,
-                    "datetime": "2024-02-05T08:05:00+01:00"
-                },
-                {
-                    "value": 57,
-                    "datetime": "2024-02-05T08:10:00+01:00"
-                },
-                {
-                    "value": 57,
-                    "datetime": "2024-02-05T08:15:00+01:00"
+                    "start": "2024-02-05T08:00:00+01:00",
+                    "end": "2024-02-05T08:15:00+01:00"
                 }
             ]
         }
diff --git a/documentation/changelog.rst b/documentation/changelog.rst
index bb521308..0fd02749 100644
--- a/documentation/changelog.rst
+++ b/documentation/changelog.rst
@@ -12,6 +12,8 @@ v0.19.0 | February xx, 2024
 New features
 -------------
 
+* Support a less verbose way of setting the same :abbr:`SoC (state of charge)` constraint for a given time window [see `PR #899 <https://github.com/FlexMeasures/flexmeasures/pull/899>`_]
+
 Infrastructure / Support
 ----------------------
 
diff --git a/flexmeasures/api/v3_0/sensors.py b/flexmeasures/api/v3_0/sensors.py
index 41de3d5a..c975cd6d 100644
--- a/flexmeasures/api/v3_0/sensors.py
+++ b/flexmeasures/api/v3_0/sensors.py
@@ -320,11 +320,8 @@ class SensorAPI(FlaskView):
                     "soc-minima": [
                         {
                             "value": 15,
-                            "datetime": "2015-06-02T14:00:00+00:00"
-                        },
-                        {
-                            "value": 20,
-                            "datetime": "2015-06-02T15:00:00+00:00"
+                            "start": "2015-06-02T14:00:00+00:00",
+                            "end": "2015-06-02T15:00:00+00:00"
                         }
                     ],
                     "soc-min": 10,
diff --git a/flexmeasures/data/models/planning/storage.py b/flexmeasures/data/models/planning/storage.py
index 0e75b3b3..c5f32575 100644
--- a/flexmeasures/data/models/planning/storage.py
+++ b/flexmeasures/data/models/planning/storage.py
@@ -490,9 +490,7 @@ class MetaStorageScheduler(Scheduler):
         """
         soc_targets = self.flex_model.get("soc_targets")
         if soc_targets:
-            max_target_datetime = max(
-                [soc_target["datetime"] for soc_target in soc_targets]
-            )
+            max_target_datetime = max([soc_target["end"] for soc_target in soc_targets])
             if max_target_datetime > self.end:
                 max_server_horizon = get_max_planning_horizon(self.resolution)
                 if max_server_horizon:
@@ -705,7 +703,6 @@ def build_device_soc_values(
     if isinstance(soc_values, pd.Series):  # some tests prepare it this way
         device_values = soc_values
     else:
-        disregarded_datetimes = []
         device_values = initialize_series(
             np.nan,
             start=start_of_schedule,
@@ -714,28 +711,41 @@ def build_device_soc_values(
             inclusive="right",  # note that target values are indexed by their due date (i.e. inclusive="right")
         )
 
+        max_server_horizon = get_max_planning_horizon(resolution)
+        disregarded_periods: list[tuple[datetime, datetime]] = []
         for soc_value in soc_values:
             soc = soc_value["value"]
-            soc_datetime = soc_value["datetime"].astimezone(
+            # convert timezone, otherwise DST would be problematic
+            soc_constraint_start = soc_value["start"].astimezone(
                 device_values.index.tzinfo
-            )  # otherwise DST would be problematic
-            if soc_datetime > end_of_schedule:
+            )
+            soc_constraint_end = soc_value["end"].astimezone(device_values.index.tzinfo)
+            if soc_constraint_end > end_of_schedule:
                 # Skip too-far-into-the-future target
-                disregarded_datetimes += [soc_datetime]
-                max_server_horizon = get_max_planning_horizon(resolution)
+                disregarded_periods += [(soc_constraint_start, soc_constraint_end)]
+                if soc_constraint_start <= end_of_schedule:
+                    device_values.loc[soc_constraint_start:end_of_schedule] = soc
                 continue
 
-            device_values.loc[soc_datetime] = soc
+            device_values.loc[soc_constraint_start:soc_constraint_end] = soc
 
-        if disregarded_datetimes:
-            if len(disregarded_datetimes) == 1:
+        if not disregarded_periods:
+            pass
+        elif len(disregarded_periods) == 1:
+            soc_constraint_start, soc_constraint_end = disregarded_periods[0]
+            if soc_constraint_start == soc_constraint_end:
                 current_app.logger.warning(
-                    f"Disregarding 1 target datetime {disregarded_datetimes[0]}, because it exceeds {end_of_schedule}. Maximum scheduling horizon is {max_server_horizon}."
+                    f"Disregarding target datetime {soc_constraint_end}, because it exceeds {end_of_schedule}. Maximum scheduling horizon is {max_server_horizon}."
                 )
             else:
                 current_app.logger.warning(
-                    f"Disregarding {len(disregarded_datetimes)} target datetimes from {min(disregarded_datetimes)} until {max(disregarded_datetimes)}, because they exceed {end_of_schedule}. Maximum scheduling horizon is {max_server_horizon}."
+                    f"Disregarding target datetimes that exceed {end_of_schedule} (within the window {soc_constraint_start} until {soc_constraint_end}). Maximum scheduling horizon is {max_server_horizon}."
                 )
+        else:
+            soc_constraint_starts, soc_constraint_ends = zip(*disregarded_periods)
+            current_app.logger.warning(
+                f"Disregarding target datetimes that exceed {end_of_schedule} (within the window {min(soc_constraint_starts)} until {max(soc_constraint_ends)} spanning {len(disregarded_periods)} targets). Maximum scheduling horizon is {max_server_horizon}."
+            )
 
         # soc_values are at the end of each time slot, while prices are indexed by the start of each time slot
         device_values = device_values[start_of_schedule + resolution : end_of_schedule]
diff --git a/flexmeasures/data/schemas/scheduling/storage.py b/flexmeasures/data/schemas/scheduling/storage.py
index da38650f..f8632990 100644
--- a/flexmeasures/data/schemas/scheduling/storage.py
+++ b/flexmeasures/data/schemas/scheduling/storage.py
@@ -2,6 +2,7 @@ from __future__ import annotations
 
 from datetime import datetime
 
+import isodate
 from flask import current_app
 from marshmallow import (
     Schema,
@@ -11,10 +12,11 @@ from marshmallow import (
     fields,
     validates,
 )
-from marshmallow.validate import OneOf, ValidationError
+from marshmallow.validate import OneOf, ValidationError, Validator
+import pandas as pd
 
 from flexmeasures.data.models.time_series import Sensor
-from flexmeasures.data.schemas.times import AwareDateTimeField
+from flexmeasures.data.schemas.times import AwareDateTimeField, DurationField
 from flexmeasures.data.schemas.units import QuantityField
 from flexmeasures.data.schemas.sensors import QuantityOrSensor
 
@@ -49,15 +51,25 @@ class EfficiencyField(QuantityField):
 
 
 class SOCValueSchema(Schema):
-    """
-    A point in time with a target value.
-    """
-
     value = fields.Float(required=True)
-    datetime = AwareDateTimeField(required=True)
-
-    def __init__(self, *args, **kwargs):
-        self.value_validator = kwargs.pop("value_validator", None)
+    datetime = AwareDateTimeField(required=False)
+    start = AwareDateTimeField(required=False)
+    end = AwareDateTimeField(required=False)
+    duration = DurationField(required=False)
+
+    def __init__(
+        self,
+        timezone: str | None = None,
+        value_validator: Validator | None = None,
+        *args,
+        **kwargs,
+    ):
+        """A time period (or single point) with a target value.
+
+        :param timezone:  Optionally, set a timezone to be able to interpret nominal durations.
+        """
+        self.timezone = timezone
+        self.value_validator = value_validator
         super().__init__(*args, **kwargs)
 
     @validates("value")
@@ -65,6 +77,56 @@ class SOCValueSchema(Schema):
         if self.value_validator is not None:
             self.value_validator(_value)
 
+    @validates_schema
+    def check_time_window(self, data: dict, **kwargs):
+        """Checks whether a complete time interval can be derived from the timing fields.
+
+        The data is updated in-place, guaranteeing that the 'start' and 'end' fields are filled out.
+        """
+        dt = data.get("datetime")
+        start = data.get("start")
+        end = data.get("end")
+        duration = data.get("duration")
+
+        if dt is not None:
+            if any([p is not None for p in (start, end, duration)]):
+                raise ValidationError(
+                    "If using the 'datetime' field, no 'start', 'end' or 'duration' is expected."
+                )
+            data["start"] = dt
+            data["end"] = dt
+        elif duration is not None:
+            if self.timezone is None and isinstance(duration, isodate.Duration):
+                raise ValidationError(
+                    "Cannot interpret nominal duration used in the 'duration' field without a known timezone."
+                )
+            elif all([p is None for p in (start, end)]) or all(
+                [p is not None for p in (start, end)]
+            ):
+                raise ValidationError(
+                    "If using the 'duration' field, either 'start' or 'end' is expected."
+                )
+            if start is not None:
+                grounded = DurationField.ground_from(
+                    duration, pd.Timestamp(start).tz_convert(self.timezone)
+                )
+                data["start"] = start
+                data["end"] = start + grounded
+            else:
+                grounded = DurationField.ground_from(
+                    -duration, pd.Timestamp(end).tz_convert(self.timezone)
+                )
+                data["start"] = end + grounded
+                data["end"] = end
+        else:
+            if any([p is None for p in (start, end)]):
+                raise ValidationError(
+                    "Missing field(s) to describe timing: use the 'datetime' field, "
+                    "or a combination of 2 fields of 'start', 'end' and 'duration'."
+                )
+            data["start"] = start
+            data["end"] = end
+
 
 class StorageFlexModelSchema(Schema):
     """
@@ -89,11 +151,23 @@ class StorageFlexModelSchema(Schema):
         "MW", data_key="production-capacity", required=False
     )
 
-    soc_maxima = fields.List(fields.Nested(SOCValueSchema()), data_key="soc-maxima")
+    # Timezone placeholder is overridden in __init__
+    soc_maxima = fields.List(
+        fields.Nested(SOCValueSchema(timezone="placeholder")),
+        data_key="soc-maxima",
+    )
     soc_minima = fields.List(
-        fields.Nested(SOCValueSchema(value_validator=validate.Range(min=0))),
+        fields.Nested(
+            SOCValueSchema(
+                timezone="placeholder", value_validator=validate.Range(min=0)
+            )
+        ),
         data_key="soc-minima",
     )
+    soc_targets = fields.List(
+        fields.Nested(SOCValueSchema(timezone="placeholder")),
+        data_key="soc-targets",
+    )
 
     soc_unit = fields.Str(
         validate=OneOf(
@@ -104,7 +178,6 @@ class StorageFlexModelSchema(Schema):
         ),
         data_key="soc-unit",
     )  # todo: allow unit to be set per field, using QuantityField("%", validate=validate.Range(min=0, max=1))
-    soc_targets = fields.List(fields.Nested(SOCValueSchema()), data_key="soc-targets")
 
     charging_efficiency = QuantityOrSensor(
         "%", data_key="charging-efficiency", required=False
@@ -129,6 +202,22 @@ class StorageFlexModelSchema(Schema):
         """Pass the schedule's start, so we can use it to validate soc-target datetimes."""
         self.start = start
         self.sensor = sensor
+        self.soc_maxima = fields.List(
+            fields.Nested(SOCValueSchema(timezone=sensor.timezone)),
+            data_key="soc-maxima",
+        )
+        self.soc_minima = fields.List(
+            fields.Nested(
+                SOCValueSchema(
+                    timezone=sensor.timezone, value_validator=validate.Range(min=0)
+                )
+            ),
+            data_key="soc-minima",
+        )
+        self.soc_targets = fields.List(
+            fields.Nested(SOCValueSchema(timezone=sensor.timezone)),
+            data_key="soc-targets",
+        )
         super().__init__(*args, **kwargs)
 
     @validates_schema
@@ -139,7 +228,7 @@ class StorageFlexModelSchema(Schema):
         max_server_horizon = current_app.config.get("FLEXMEASURES_MAX_PLANNING_HORIZON")
         if isinstance(max_server_horizon, int):
             max_server_horizon *= self.sensor.event_resolution
-        max_target_datetime = max([target["datetime"] for target in soc_targets])
+        max_target_datetime = max([target["end"] for target in soc_targets])
         max_server_datetime = self.start + max_server_horizon
         if max_target_datetime > max_server_datetime:
             current_app.logger.warning(
diff --git a/flexmeasures/data/services/scheduling.py b/flexmeasures/data/services/scheduling.py
index 07956b07..19a1d8d9 100644
--- a/flexmeasures/data/services/scheduling.py
+++ b/flexmeasures/data/services/scheduling.py
@@ -310,7 +310,7 @@ def make_schedule(
 
     consumption_schedule: SchedulerOutputType = scheduler.compute()
 
-    # in case we are getting a custom Scheduler that hasn't implement the multiple output return
+    # in case we are getting a custom Scheduler that hasn't implemented the multiple output return
     # this should only be called whenever the Scheduler applies to the Sensor.
     if isinstance(consumption_schedule, pd.Series):
         assert isinstance(asset_or_sensor, Sensor), ""
diff --git a/flexmeasures/utils/time_utils.py b/flexmeasures/utils/time_utils.py
index 2c3107d9..ac958812 100644
--- a/flexmeasures/utils/time_utils.py
+++ b/flexmeasures/utils/time_utils.py
@@ -97,7 +97,7 @@ def naturalized_datetime_str(dt: datetime | None, now: datetime | None = None) -
     The dt parameter (as well as the now parameter if you use it)
     can be either naive or tz-aware. We assume UTC in the naive case.
 
-    We use the the humanize library to generate a human-friendly string.
+    We use the `humanize` library to generate a human-friendly string.
     If dt is not longer ago than 24 hours, we use humanize.naturaltime (e.g. "3 hours ago"),
     otherwise humanize.naturaldate (e.g. "one week ago")
 


Test patch:
diff --git a/flexmeasures/api/v3_0/tests/test_sensor_schedules.py b/flexmeasures/api/v3_0/tests/test_sensor_schedules.py
index 0654298e..186a2559 100644
--- a/flexmeasures/api/v3_0/tests/test_sensor_schedules.py
+++ b/flexmeasures/api/v3_0/tests/test_sensor_schedules.py
@@ -1,22 +1,16 @@
 from flask import url_for
 import pytest
-from isodate import parse_datetime, parse_duration
+from isodate import parse_datetime
 
-import pandas as pd
 from rq.job import Job
 
 from flexmeasures.api.common.responses import unknown_schedule, unrecognized_event
 from flexmeasures.api.tests.utils import check_deprecation
 from flexmeasures.api.v3_0.tests.utils import message_for_trigger_schedule
 from flexmeasures.data.models.data_sources import DataSource
-from flexmeasures.data.models.generic_assets import GenericAsset
-from flexmeasures.data.models.time_series import Sensor, TimedBelief
+from flexmeasures.data.models.time_series import Sensor
 from flexmeasures.data.tests.utils import work_on_rq
-from flexmeasures.data.services.scheduling import (
-    handle_scheduling_exception,
-    get_data_source_for_job,
-)
-from flexmeasures.utils.calculations import integrate_time_series
+from flexmeasures.data.services.scheduling import handle_scheduling_exception
 
 
 @pytest.mark.parametrize(
@@ -163,174 +157,6 @@ def test_trigger_and_get_schedule_with_unknown_prices(
     assert "prices unknown" in get_schedule_response.json["message"].lower()
 
 
-@pytest.mark.parametrize(
-    "message, asset_name",
-    [
-        (message_for_trigger_schedule(), "Test battery"),
-        (message_for_trigger_schedule(with_targets=True), "Test charging station"),
-    ],
-)
-@pytest.mark.parametrize(
-    "requesting_user", ["test_prosumer_user@seita.nl"], indirect=True
-)
-def test_trigger_and_get_schedule(
-    app,
-    add_market_prices,
-    add_battery_assets,
-    battery_soc_sensor,
-    add_charging_station_assets,
-    keep_scheduling_queue_empty,
-    message,
-    asset_name,
-    requesting_user,
-):  # noqa: C901
-    # Include the price sensor and site-power-capacity in the flex-context explicitly, to test deserialization
-    price_sensor_id = add_market_prices["epex_da"].id
-    message["flex-context"] = {
-        "consumption-price-sensor": price_sensor_id,
-        "production-price-sensor": price_sensor_id,
-        "site-power-capacity": "1 TW",  # should be big enough to avoid any infeasibilities
-    }
-
-    # trigger a schedule through the /sensors/<id>/schedules/trigger [POST] api endpoint
-    assert len(app.queues["scheduling"]) == 0
-
-    sensor = (
-        Sensor.query.filter(Sensor.name == "power")
-        .join(GenericAsset)
-        .filter(GenericAsset.id == Sensor.generic_asset_id)
-        .filter(GenericAsset.name == asset_name)
-        .one_or_none()
-    )
-    with app.test_client() as client:
-        trigger_schedule_response = client.post(
-            url_for("SensorAPI:trigger_schedule", id=sensor.id),
-            json=message,
-        )
-        print("Server responded with:\n%s" % trigger_schedule_response.json)
-        assert trigger_schedule_response.status_code == 200
-        job_id = trigger_schedule_response.json["schedule"]
-
-    # look for scheduling jobs in queue
-    assert (
-        len(app.queues["scheduling"]) == 1
-    )  # only 1 schedule should be made for 1 asset
-    job = app.queues["scheduling"].jobs[0]
-    assert job.kwargs["asset_or_sensor"]["id"] == sensor.id
-    assert job.kwargs["start"] == parse_datetime(message["start"])
-    assert job.id == job_id
-
-    # process the scheduling queue
-    work_on_rq(app.queues["scheduling"], exc_handler=handle_scheduling_exception)
-    assert (
-        Job.fetch(job_id, connection=app.queues["scheduling"].connection).is_finished
-        is True
-    )
-
-    # Derive some expectations from the POSTed message
-    if "flex-model" not in message:
-        start_soc = message["soc-at-start"] / 1000  # in MWh
-        roundtrip_efficiency = (
-            float(message["roundtrip-efficiency"].replace("%", "")) / 100.0
-        )
-        storage_efficiency = (
-            float(message["storage-efficiency"].replace("%", "")) / 100.0
-        )
-        soc_targets = message.get("soc-targets")
-    else:
-        start_soc = message["flex-model"]["soc-at-start"] / 1000  # in MWh
-        roundtrip_efficiency = (
-            float(message["flex-model"]["roundtrip-efficiency"].replace("%", ""))
-            / 100.0
-        )
-        storage_efficiency = (
-            float(message["flex-model"]["storage-efficiency"].replace("%", "")) / 100.0
-        )
-        soc_targets = message["flex-model"].get("soc-targets")
-    resolution = sensor.event_resolution
-    if soc_targets:
-        # Schedule length may be extended to accommodate targets that lie beyond the schedule's end
-        max_target_datetime = max(
-            [parse_datetime(soc_target["datetime"]) for soc_target in soc_targets]
-        )
-        expected_length_of_schedule = (
-            max(
-                parse_duration(message["duration"]),
-                max_target_datetime - parse_datetime(message["start"]),
-            )
-            / resolution
-        )
-    else:
-        expected_length_of_schedule = parse_duration(message["duration"]) / resolution
-
-    # check results are in the database
-
-    # First, make sure the scheduler data source is now there
-    job.refresh()  # catch meta info that was added on this very instance
-    scheduler_source = get_data_source_for_job(job)
-    assert scheduler_source is not None
-
-    # Then, check if the data was created
-    power_values = (
-        TimedBelief.query.filter(TimedBelief.sensor_id == sensor.id)
-        .filter(TimedBelief.source_id == scheduler_source.id)
-        .all()
-    )
-    consumption_schedule = pd.Series(
-        [-v.event_value for v in power_values],
-        index=pd.DatetimeIndex([v.event_start for v in power_values], freq=resolution),
-    )  # For consumption schedules, positive values denote consumption. For the db, consumption is negative
-    assert len(consumption_schedule) == expected_length_of_schedule
-
-    # check targets, if applicable
-    if soc_targets:
-        soc_schedule = integrate_time_series(
-            consumption_schedule,
-            start_soc,
-            up_efficiency=roundtrip_efficiency**0.5,
-            down_efficiency=roundtrip_efficiency**0.5,
-            storage_efficiency=storage_efficiency,
-            decimal_precision=6,
-        )
-        print(consumption_schedule)
-        print(soc_schedule)
-        for target in soc_targets:
-            assert soc_schedule[target["datetime"]] == target["value"] / 1000
-
-    # try to retrieve the schedule through the /sensors/<id>/schedules/<job_id> [GET] api endpoint
-    get_schedule_response = client.get(
-        url_for("SensorAPI:get_schedule", id=sensor.id, uuid=job_id),
-        query_string={"duration": "PT48H"},
-    )
-    print("Server responded with:\n%s" % get_schedule_response.json)
-    assert get_schedule_response.status_code == 200
-    # assert get_schedule_response.json["type"] == "GetDeviceMessageResponse"
-    assert len(get_schedule_response.json["values"]) == expected_length_of_schedule
-
-    # Test that a shorter planning horizon yields the same result for the shorter planning horizon
-    get_schedule_response_short = client.get(
-        url_for("SensorAPI:get_schedule", id=sensor.id, uuid=job_id),
-        query_string={"duration": "PT6H"},
-    )
-    assert (
-        get_schedule_response_short.json["values"]
-        == get_schedule_response.json["values"][0:24]
-    )
-
-    # Test that a much longer planning horizon yields the same result (when there are only 2 days of prices)
-    get_schedule_response_long = client.get(
-        url_for("SensorAPI:get_schedule", id=sensor.id, uuid=job_id),
-        query_string={"duration": "PT1000H"},
-    )
-    assert (
-        get_schedule_response_long.json["values"][0:192]
-        == get_schedule_response.json["values"]
-    )
-
-    # Check whether the soc-at-start was persisted as an asset attribute
-    assert sensor.generic_asset.get_attribute("soc_in_mwh") == start_soc
-
-
 @pytest.mark.parametrize(
     "requesting_user", ["test_prosumer_user@seita.nl"], indirect=True
 )
diff --git a/flexmeasures/api/v3_0/tests/test_sensor_schedules_fresh_db.py b/flexmeasures/api/v3_0/tests/test_sensor_schedules_fresh_db.py
new file mode 100644
index 00000000..77ebf15b
--- /dev/null
+++ b/flexmeasures/api/v3_0/tests/test_sensor_schedules_fresh_db.py
@@ -0,0 +1,195 @@
+from flask import url_for
+import pytest
+from isodate import parse_datetime, parse_duration
+
+import pandas as pd
+from rq.job import Job
+
+from flexmeasures.api.v3_0.tests.utils import message_for_trigger_schedule
+from flexmeasures.data.models.generic_assets import GenericAsset
+from flexmeasures.data.models.time_series import Sensor, TimedBelief
+from flexmeasures.data.tests.utils import work_on_rq
+from flexmeasures.data.services.scheduling import (
+    handle_scheduling_exception,
+    get_data_source_for_job,
+)
+from flexmeasures.utils.calculations import integrate_time_series
+
+
+@pytest.mark.parametrize(
+    "message, asset_name",
+    [
+        (message_for_trigger_schedule(), "Test battery"),
+        (message_for_trigger_schedule(with_targets=True), "Test charging station"),
+        (
+            message_for_trigger_schedule(with_targets=True, use_time_window=True),
+            "Test charging station",
+        ),
+    ],
+)
+@pytest.mark.parametrize(
+    "requesting_user", ["test_prosumer_user@seita.nl"], indirect=True
+)
+def test_trigger_and_get_schedule(
+    app,
+    add_market_prices_fresh_db,
+    add_battery_assets_fresh_db,
+    battery_soc_sensor_fresh_db,
+    add_charging_station_assets_fresh_db,
+    keep_scheduling_queue_empty,
+    message,
+    asset_name,
+    requesting_user,
+):  # noqa: C901
+    # Include the price sensor and site-power-capacity in the flex-context explicitly, to test deserialization
+    price_sensor_id = add_market_prices_fresh_db["epex_da"].id
+    message["flex-context"] = {
+        "consumption-price-sensor": price_sensor_id,
+        "production-price-sensor": price_sensor_id,
+        "site-power-capacity": "1 TW",  # should be big enough to avoid any infeasibilities
+    }
+
+    # trigger a schedule through the /sensors/<id>/schedules/trigger [POST] api endpoint
+    assert len(app.queues["scheduling"]) == 0
+
+    sensor = (
+        Sensor.query.filter(Sensor.name == "power")
+        .join(GenericAsset)
+        .filter(GenericAsset.id == Sensor.generic_asset_id)
+        .filter(GenericAsset.name == asset_name)
+        .one_or_none()
+    )
+    with app.test_client() as client:
+        trigger_schedule_response = client.post(
+            url_for("SensorAPI:trigger_schedule", id=sensor.id),
+            json=message,
+        )
+        print("Server responded with:\n%s" % trigger_schedule_response.json)
+        assert trigger_schedule_response.status_code == 200
+        job_id = trigger_schedule_response.json["schedule"]
+
+    # look for scheduling jobs in queue
+    assert (
+        len(app.queues["scheduling"]) == 1
+    )  # only 1 schedule should be made for 1 asset
+    job = app.queues["scheduling"].jobs[0]
+    print(job.kwargs)
+    assert job.kwargs["asset_or_sensor"]["id"] == sensor.id
+    assert job.kwargs["start"] == parse_datetime(message["start"])
+    assert job.id == job_id
+
+    # process the scheduling queue
+    work_on_rq(app.queues["scheduling"], exc_handler=handle_scheduling_exception)
+    assert (
+        Job.fetch(job_id, connection=app.queues["scheduling"].connection).is_finished
+        is True
+    )
+
+    # Derive some expectations from the POSTed message
+    if "flex-model" not in message:
+        start_soc = message["soc-at-start"] / 1000  # in MWh
+        roundtrip_efficiency = (
+            float(message["roundtrip-efficiency"].replace("%", "")) / 100.0
+        )
+        storage_efficiency = (
+            float(message["storage-efficiency"].replace("%", "")) / 100.0
+        )
+        soc_targets = message.get("soc-targets")
+    else:
+        start_soc = message["flex-model"]["soc-at-start"] / 1000  # in MWh
+        roundtrip_efficiency = (
+            float(message["flex-model"]["roundtrip-efficiency"].replace("%", ""))
+            / 100.0
+        )
+        storage_efficiency = (
+            float(message["flex-model"]["storage-efficiency"].replace("%", "")) / 100.0
+        )
+        soc_targets = message["flex-model"].get("soc-targets")
+    resolution = sensor.event_resolution
+    if soc_targets:
+        # Schedule length may be extended to accommodate targets that lie beyond the schedule's end
+        max_target_datetime = max(
+            [
+                parse_datetime(soc_target.get("datetime", soc_target.get("end")))
+                for soc_target in soc_targets
+            ]
+        )
+        expected_length_of_schedule = (
+            max(
+                parse_duration(message["duration"]),
+                max_target_datetime - parse_datetime(message["start"]),
+            )
+            / resolution
+        )
+    else:
+        expected_length_of_schedule = parse_duration(message["duration"]) / resolution
+
+    # check results are in the database
+
+    # First, make sure the scheduler data source is now there
+    job.refresh()  # catch meta info that was added on this very instance
+    scheduler_source = get_data_source_for_job(job)
+    assert scheduler_source is not None
+
+    # Then, check if the data was created
+    power_values = (
+        TimedBelief.query.filter(TimedBelief.sensor_id == sensor.id)
+        .filter(TimedBelief.source_id == scheduler_source.id)
+        .all()
+    )
+    consumption_schedule = pd.Series(
+        [-v.event_value for v in power_values],
+        index=pd.DatetimeIndex([v.event_start for v in power_values], freq=resolution),
+    )  # For consumption schedules, positive values denote consumption. For the db, consumption is negative
+    assert len(consumption_schedule) == expected_length_of_schedule
+
+    # check targets, if applicable
+    if soc_targets:
+        soc_schedule = integrate_time_series(
+            consumption_schedule,
+            start_soc,
+            up_efficiency=roundtrip_efficiency**0.5,
+            down_efficiency=roundtrip_efficiency**0.5,
+            storage_efficiency=storage_efficiency,
+            decimal_precision=6,
+        )
+        print(consumption_schedule)
+        print(soc_schedule)
+        for target in soc_targets:
+            assert (
+                soc_schedule[target.get("datetime", target.get("end"))]
+                == target["value"] / 1000
+            )
+
+    # try to retrieve the schedule through the /sensors/<id>/schedules/<job_id> [GET] api endpoint
+    get_schedule_response = client.get(
+        url_for("SensorAPI:get_schedule", id=sensor.id, uuid=job_id),
+        query_string={"duration": "PT48H"},
+    )
+    print("Server responded with:\n%s" % get_schedule_response.json)
+    assert get_schedule_response.status_code == 200
+    # assert get_schedule_response.json["type"] == "GetDeviceMessageResponse"
+    assert len(get_schedule_response.json["values"]) == expected_length_of_schedule
+
+    # Test that a shorter planning horizon yields the same result for the shorter planning horizon
+    get_schedule_response_short = client.get(
+        url_for("SensorAPI:get_schedule", id=sensor.id, uuid=job_id),
+        query_string={"duration": "PT6H"},
+    )
+    assert (
+        get_schedule_response_short.json["values"]
+        == get_schedule_response.json["values"][0:24]
+    )
+
+    # Test that a much longer planning horizon yields the same result (when there are only 2 days of prices)
+    get_schedule_response_long = client.get(
+        url_for("SensorAPI:get_schedule", id=sensor.id, uuid=job_id),
+        query_string={"duration": "PT1000H"},
+    )
+    assert (
+        get_schedule_response_long.json["values"][0:192]
+        == get_schedule_response.json["values"]
+    )
+
+    # Check whether the soc-at-start was persisted as an asset attribute
+    assert sensor.generic_asset.get_attribute("soc_in_mwh") == start_soc
diff --git a/flexmeasures/api/v3_0/tests/utils.py b/flexmeasures/api/v3_0/tests/utils.py
index 56dcd549..9e0ecd1a 100644
--- a/flexmeasures/api/v3_0/tests/utils.py
+++ b/flexmeasures/api/v3_0/tests/utils.py
@@ -56,6 +56,7 @@ def message_for_trigger_schedule(
     with_targets: bool = False,
     realistic_targets: bool = True,
     too_far_into_the_future_targets: bool = False,
+    use_time_window: bool = False,
 ) -> dict:
     message = {
         "start": "2015-01-01T00:00:00+01:00",
@@ -87,14 +88,21 @@ def message_for_trigger_schedule(
             target_datetime = "2015-02-02T23:00:00+01:00"
         else:
             target_datetime = "2015-01-02T23:00:00+01:00"
+        if use_time_window:
+            target_time_window = {
+                "start": "2015-01-02T22:45:00+01:00",
+                "end": target_datetime,
+            }
+        else:
+            target_time_window = {"datetime": target_datetime}
         message["flex-model"]["soc-targets"] = [
-            {"value": target_value, "datetime": target_datetime}
+            {"value": target_value, **target_time_window}
         ]
         # Also create some minima and maxima constraints to test correct deserialization using the soc-unit
         message["flex-model"]["soc-minima"] = [
-            {"value": target_value - 1, "datetime": target_datetime}
+            {"value": target_value - 1, **target_time_window}
         ]
         message["flex-model"]["soc-maxima"] = [
-            {"value": target_value + 1, "datetime": target_datetime}
+            {"value": target_value + 1, **target_time_window}
         ]
     return message
diff --git a/flexmeasures/conftest.py b/flexmeasures/conftest.py
index 1ba59bfd..1beaccf6 100644
--- a/flexmeasures/conftest.py
+++ b/flexmeasures/conftest.py
@@ -1048,6 +1048,17 @@ def add_sensors(db: SQLAlchemy, setup_generic_assets):
 
 @pytest.fixture(scope="module")
 def battery_soc_sensor(db: SQLAlchemy, setup_generic_assets):
+    """Add a battery SOC sensor to the db."""
+    return create_battery_soc_sensor(db, setup_generic_assets)
+
+
+@pytest.fixture(scope="function")
+def battery_soc_sensor_fresh_db(fresh_db: SQLAlchemy, setup_generic_assets_fresh_db):
+    """Add a battery SOC sensor to the fresh db."""
+    return create_battery_soc_sensor(fresh_db, setup_generic_assets_fresh_db)
+
+
+def create_battery_soc_sensor(db: SQLAlchemy, setup_generic_assets):
     """Add a battery SOC sensor."""
     soc_sensor = Sensor(
         name="state of charge",
diff --git a/flexmeasures/data/models/planning/tests/test_solver.py b/flexmeasures/data/models/planning/tests/test_solver.py
index f7cab6bd..33665fcf 100644
--- a/flexmeasures/data/models/planning/tests/test_solver.py
+++ b/flexmeasures/data/models/planning/tests/test_solver.py
@@ -19,6 +19,7 @@ from flexmeasures.data.models.planning.storage import (
 from flexmeasures.data.models.planning.linear_optimization import device_scheduler
 from flexmeasures.data.models.planning.tests.utils import check_constraints
 from flexmeasures.data.models.planning.utils import initialize_series, initialize_df
+from flexmeasures.data.schemas.scheduling.storage import SOCValueSchema
 from flexmeasures.utils.calculations import (
     apply_stock_changes_and_losses,
     integrate_time_series,
@@ -1261,33 +1262,51 @@ def test_capacity(
 
 
 @pytest.mark.parametrize(
-    ["soc_values", "log_message"],
+    ["soc_values", "log_message", "expected_num_targets"],
     [
+        (
+            [
+                {
+                    "start": datetime(2023, 5, 19, tzinfo=pytz.utc),
+                    "end": datetime(2023, 5, 23, tzinfo=pytz.utc),
+                    "value": 1.0,
+                },
+            ],
+            "Disregarding target datetimes that exceed 2023-05-20 00:00:00+00:00 (within the window 2023-05-19 00:00:00+00:00 until 2023-05-23 00:00:00+00:00).",
+            1 * 24 * 60 / 5
+            + 1,  # every 5-minute mark on the 19th including both midnights
+        ),
         (
             [
                 {"datetime": datetime(2023, 5, 19, tzinfo=pytz.utc), "value": 1.0},
-                {"datetime": datetime(2023, 5, 22, tzinfo=pytz.utc), "value": 1.0},
                 {"datetime": datetime(2023, 5, 23, tzinfo=pytz.utc), "value": 1.0},
-                {"datetime": datetime(2023, 5, 21, tzinfo=pytz.utc), "value": 1.0},
             ],
-            "Disregarding 3 target datetimes from 2023-05-21 00:00:00+00:00 until 2023-05-23 00:00:00+00:00, because they exceed 2023-05-20 00:00:00+00:00",
+            "Disregarding target datetime 2023-05-23 00:00:00+00:00, because it exceeds 2023-05-20 00:00:00+00:00.",
+            1,  # only the 19th
         ),
         (
             [
                 {"datetime": datetime(2023, 5, 19, tzinfo=pytz.utc), "value": 1.0},
+                {"datetime": datetime(2023, 5, 22, tzinfo=pytz.utc), "value": 1.0},
                 {"datetime": datetime(2023, 5, 23, tzinfo=pytz.utc), "value": 1.0},
+                {"datetime": datetime(2023, 5, 21, tzinfo=pytz.utc), "value": 1.0},
             ],
-            "Disregarding 1 target datetime 2023-05-23 00:00:00+00:00, because it exceeds 2023-05-20 00:00:00+00:00",
+            "Disregarding target datetimes that exceed 2023-05-20 00:00:00+00:00 (within the window 2023-05-21 00:00:00+00:00 until 2023-05-23 00:00:00+00:00 spanning 3 targets).",
+            1,  # only the 19th
         ),
     ],
 )
-def test_build_device_soc_values(caplog, soc_values, log_message):
+def test_build_device_soc_values(caplog, soc_values, log_message, expected_num_targets):
     caplog.set_level(logging.WARNING)
     soc_at_start = 3.0
     start_of_schedule = datetime(2023, 5, 18, tzinfo=pytz.utc)
     end_of_schedule = datetime(2023, 5, 20, tzinfo=pytz.utc)
     resolution = timedelta(minutes=5)
 
+    # Convert SoC datetimes to periods with a start and end.
+    for soc in soc_values:
+        SOCValueSchema().check_time_window(soc)
+
     with caplog.at_level(logging.WARNING):
         device_values = build_device_soc_values(
             soc_values=soc_values,
@@ -1299,6 +1318,15 @@ def test_build_device_soc_values(caplog, soc_values, log_message):
     print(device_values)
     assert log_message in caplog.text
 
+    # Check test assumption
+    for soc in soc_values:
+        assert soc["value"] == 1
+    soc_delta = 1 - soc_at_start
+    soc_delta_per_resolution = soc_delta * timedelta(hours=1) / resolution
+
+    assert soc_delta_per_resolution in device_values
+    assert np.count_nonzero(~np.isnan(device_values)) == expected_num_targets
+
 
 @pytest.mark.parametrize(
     "battery_name, production_sensor, consumption_sensor, production_quantity, consumption_quantity, expected_production, expected_consumption",
@@ -1406,7 +1434,6 @@ def test_battery_power_capacity_as_sensor(
     expected_production,
     expected_consumption,
 ):
-
     epex_da, battery = get_sensors_from_db(
         add_battery_assets, battery_name=battery_name
     )
@@ -1453,7 +1480,6 @@ def test_battery_power_capacity_as_sensor(
 def get_efficiency_problem_device_constraints(
     extra_flex_model, efficiency_sensors, add_battery_assets
 ) -> pd.DataFrame:
-
     _, battery = get_sensors_from_db(add_battery_assets)
     tz = pytz.timezone("Europe/Amsterdam")
     start = tz.localize(datetime(2015, 1, 1))
diff --git a/flexmeasures/data/schemas/tests/test_scheduling.py b/flexmeasures/data/schemas/tests/test_scheduling.py
index 7bbd180c..9095b355 100644
--- a/flexmeasures/data/schemas/tests/test_scheduling.py
+++ b/flexmeasures/data/schemas/tests/test_scheduling.py
@@ -1,17 +1,100 @@
+from datetime import datetime
+import pytz
 import pytest
 
+from marshmallow.validate import ValidationError
+import pandas as pd
+
 from flexmeasures.data.schemas.scheduling.process import (
     ProcessSchedulerFlexModelSchema,
     ProcessType,
 )
-from flexmeasures.data.schemas.scheduling.storage import StorageFlexModelSchema
-from marshmallow.validate import ValidationError
-from datetime import datetime
-import pytz
+from flexmeasures.data.schemas.scheduling.storage import (
+    SOCValueSchema,
+    StorageFlexModelSchema,
+)
 
 
-def test_process_scheduler_flex_model_load(db, app, setup_dummy_sensors):
+@pytest.mark.parametrize(
+    ["timing_input", "expected_start", "expected_end"],
+    [
+        (
+            {"datetime": "2023-03-27T00:00:00+02:00"},
+            "2023-03-27T00:00:00+02:00",
+            "2023-03-27T00:00:00+02:00",
+        ),
+        (
+            {"start": "2023-03-26T00:00:00+01:00", "end": "2023-03-27T00:00:00+02:00"},
+            "2023-03-26T00:00:00+01:00",
+            "2023-03-27T00:00:00+02:00",
+        ),
+        (
+            {"start": "2023-03-26T00:00:00+01:00", "duration": "PT24H"},
+            "2023-03-26T00:00:00+01:00",
+            "2023-03-27T01:00:00+02:00",
+        ),
+        # https://github.com/gweis/isodate/issues/74
+        # (
+        #     {"start": "2023-03-26T00:00:00+01:00", "duration": "P1D"},
+        #     "2023-03-26T00:00:00+01:00",
+        #     "2023-03-27T00:00:00+02:00",
+        # ),
+        # (
+        #     {"start": "2023-03-26T00:00:00+01:00", "duration": "P1W"},
+        #     "2023-03-26T00:00:00+01:00",
+        #     "2023-04-02T00:00:00+02:00",
+        # ),
+        (
+            {"start": "2023-03-26T00:00:00+01:00", "duration": "P1M"},
+            "2023-03-26T00:00:00+01:00",
+            "2023-04-26T00:00:00+02:00",
+        ),
+        (
+            {"end": "2023-03-27T00:00:00+02:00", "duration": "PT24H"},
+            "2023-03-25T23:00:00+01:00",
+            "2023-03-27T00:00:00+02:00",
+        ),
+        (
+            {"start": "2023-10-29T00:00:00+02:00", "duration": "PT24H"},
+            "2023-10-29T00:00:00+02:00",
+            "2023-10-29T23:00:00+01:00",
+        ),
+        # https://github.com/gweis/isodate/issues/74
+        # (
+        #     {"start": "2023-10-29T00:00:00+02:00", "duration": "P1D"},
+        #     "2023-10-29T00:00:00+02:00",
+        #     "2023-10-30T00:00:00+01:00",
+        # ),
+        # (
+        #     {"start": "2023-10-29T00:00:00+02:00", "duration": "P1W"},
+        #     "2023-10-29T00:00:00+02:00",
+        #     "2023-11-05T00:00:00+01:00",
+        # ),
+        (
+            {"start": "2023-10-29T00:00:00+02:00", "duration": "P1M"},
+            "2023-10-29T00:00:00+02:00",
+            "2023-11-29T00:00:00+01:00",
+        ),
+        (
+            {"end": "2023-11-29T00:00:00+01:00", "duration": "P1M"},
+            "2023-10-29T00:00:00+02:00",
+            "2023-11-29T00:00:00+01:00",
+        ),
+    ],
+)
+def test_soc_value_field(timing_input, expected_start, expected_end):
+    data = SOCValueSchema(timezone="Europe/Amsterdam").load(
+        {
+            "value": 3,
+            **timing_input,
+        }
+    )
+    print(data)
+    assert data["start"] == pd.Timestamp(expected_start)
+    assert data["end"] == pd.Timestamp(expected_end)
+
 
+def test_process_scheduler_flex_model_load(db, app, setup_dummy_sensors):
     sensor1, _ = setup_dummy_sensors
 
     schema = ProcessSchedulerFlexModelSchema(
@@ -34,7 +117,6 @@ def test_process_scheduler_flex_model_load(db, app, setup_dummy_sensors):
 
 
 def test_process_scheduler_flex_model_process_type(db, app, setup_dummy_sensors):
-
     sensor1, _ = setup_dummy_sensors
 
     # checking default
diff --git a/flexmeasures/data/schemas/tests/test_times.py b/flexmeasures/data/schemas/tests/test_times.py
index b33dbbeb..8f3415a5 100644
--- a/flexmeasures/data/schemas/tests/test_times.py
+++ b/flexmeasures/data/schemas/tests/test_times.py
@@ -13,7 +13,7 @@ from flexmeasures.data.schemas.times import DurationField, DurationValidationErr
         ("PT1H", timedelta(hours=1)),
         ("PT6M", timedelta(minutes=6)),
         ("PT6H", timedelta(hours=6)),
-        ("P2DT1H", timedelta(hours=49)),
+        ("P2DT1H", timedelta(hours=49)),  # https://github.com/gweis/isodate/issues/74
     ],
 )
 def test_duration_field_straightforward(duration_input, exp_deserialization):