Repository: NHSDigital/immunisation-fhir-api
Pull request #548: VED-329: Delta-Record-Fix
Author: login: Akol125
Created: 2025-05-29T15:41:32Z | Merged: 2025-05-30T01:36:30Z

## Summary
* Routine Change
* :exclamation: Breaking Change
* :robot: Operational or Infrastructure Change
* :sparkles: New Feature
* :warning: Potential issues that might be caused by this change

Add any other relevant notes or explanations here. **Remove this line if you have nothing to add.**


## Reviews Required
* [x] Dev
* [ ] Test
* [ ] Tech Author
* [ ] Product Owner


## Review Checklist
:information_source: This section is to be filled in by the **reviewer**.

* [ ] I have reviewed the changes in this PR and they fill all or part of the acceptance criteria of the ticket, and the code is in a mergeable state.
* [ ] If there were infrastructure, operational, or build changes, I have made sure there is sufficient evidence that the changes will work.
* [ ] I have ensured the changelog has been updated by the submitter, if necessary.



Diff:
diff --git a/backend/src/models/fhir_immunization_pre_validators.py b/backend/src/models/fhir_immunization_pre_validators.py
index 66b70a2002..46c9279d04 100644
--- a/backend/src/models/fhir_immunization_pre_validators.py
+++ b/backend/src/models/fhir_immunization_pre_validators.py
@@ -519,7 +519,7 @@ def pre_validate_recorded(self, values: dict) -> dict:
         """
         try:
             recorded = values["recorded"]
-            PreValidation.for_date_time(recorded, "recorded")
+            PreValidation.for_date_time(recorded, "recorded",strict_timezone=False)
         except KeyError:
             pass
 
diff --git a/backend/src/models/utils/pre_validator_utils.py b/backend/src/models/utils/pre_validator_utils.py
index 56f1d1bca4..7f39643d26 100644
--- a/backend/src/models/utils/pre_validator_utils.py
+++ b/backend/src/models/utils/pre_validator_utils.py
@@ -98,7 +98,7 @@ def for_date(field_value: str, field_location: str):
             ) from value_error
 
     @staticmethod
-    def for_date_time(field_value: str, field_location: str):
+    def for_date_time(field_value: str, field_location: str, strict_timezone: bool = True):
         """
         Apply pre-validation to a datetime field to ensure that it is a string (JSON dates must be written as strings)
         containing a valid datetime. Note that partial dates are valid for FHIR, but are not allowed for this API.
@@ -120,9 +120,11 @@ def for_date_time(field_value: str, field_location: str):
             "- 'YYYY-MM-DDThh:mm:ss.f' — Full date and time with milliseconds (any level of precision)"
             "- 'YYYY-MM-DDThh:mm:ss%z' — Full date and time with timezone (e.g. +00:00 or +01:00)"
             "- 'YYYY-MM-DDThh:mm:ss.f%z' — Full date and time with milliseconds and timezone"
-            "Only '+00:00' and '+01:00' are accepted as valid timezone offsets."
-            f"Note that partial dates are not allowed for {field_location} in this service."
         )
+         
+        if strict_timezone:
+            error_message += "Only '+00:00' and '+01:00' are accepted as valid timezone offsets.\n"
+            error_message += f"Note that partial dates are not allowed for {field_location} in this service."
 
         allowed_suffixes = {"+00:00", "+01:00", "+0000", "+0100",}
 
@@ -136,7 +138,7 @@ def for_date_time(field_value: str, field_location: str):
             try:
                 fhir_date = datetime.strptime(field_value, fmt)
                 
-                if fhir_date.tzinfo is not None:
+                if strict_timezone and fhir_date.tzinfo is not None:
                    if not any(field_value.endswith(suffix) for suffix in allowed_suffixes):
                        raise ValueError(error_message)
                 return fhir_date.isoformat()
diff --git a/backend/tests/test_immunization_pre_validator.py b/backend/tests/test_immunization_pre_validator.py
index 2dcd400aae..6862db9a80 100644
--- a/backend/tests/test_immunization_pre_validator.py
+++ b/backend/tests/test_immunization_pre_validator.py
@@ -645,7 +645,7 @@ def test_pre_validate_practitioner_name_family(self):
 
     def test_pre_validate_recorded(self):
         """Test pre_validate_recorded accepts valid values and rejects invalid values"""
-        ValidatorModelTests.test_date_time_value(self, field_location="recorded")
+        ValidatorModelTests.test_date_time_value(self, field_location="recorded", is_occurrence_date_time=False)
 
     def test_pre_validate_primary_source(self):
         """Test pre_validate_primary_source accepts valid values and rejects invalid values"""
diff --git a/backend/tests/utils/pre_validation_test_utils.py b/backend/tests/utils/pre_validation_test_utils.py
index 5328be6eab..0f4e5ff2f5 100644
--- a/backend/tests/utils/pre_validation_test_utils.py
+++ b/backend/tests/utils/pre_validation_test_utils.py
@@ -328,11 +328,29 @@ def test_date_time_value(
         * Invalid date time string formats
         * Invalid date-times
         """
+        expected_error_message = (
+            f"{field_location} must be a valid datetime in one of the following formats:"
+            "- 'YYYY-MM-DD' — Full date only"
+            "- 'YYYY-MM-DDThh:mm:ss' — Full date and time without milliseconds"
+            "- 'YYYY-MM-DDThh:mm:ss.f' — Full date and time with milliseconds (any level of precision)"
+            "- 'YYYY-MM-DDThh:mm:ss%z' — Full date and time with timezone (e.g. +00:00 or +01:00)"
+            "- 'YYYY-MM-DDThh:mm:ss.f%z' — Full date and time with milliseconds and timezone"
+        )
+
+        if is_occurrence_date_time:
+             expected_error_message += "Only '+00:00' and '+01:00' are accepted as valid timezone offsets.\n"
+             expected_error_message += f"Note that partial dates are not allowed for {field_location} in this service."
+             valid_datetime_formats = ValidValues.for_date_times_strict_timezones
+             invalid_datetime_formats = InvalidValues.for_date_time_string_formats_for_strict_timezone
+        else:
+            # For recorded, skip values that are valid ISO with non-restricted timezone
+            valid_datetime_formats = ValidValues.for_date_times_relaxed_timezones
+            invalid_datetime_formats = InvalidValues.for_date_time_string_formats_for_relaxed_timezone
 
         valid_json_data = deepcopy(test_instance.json_data)
 
         # Test that valid data is accepted
-        test_valid_values_accepted(test_instance, valid_json_data, field_location, ValidValues.for_date_times)
+        test_valid_values_accepted(test_instance, valid_json_data, field_location, valid_datetime_formats)
 
         # Set list of invalid data types to test
         invalid_data_types_for_strings = InvalidDataTypes.for_strings
@@ -349,19 +367,8 @@ def test_date_time_value(
                 expected_error_message=f"{field_location} must be a string",
             )
 
-        expected_error_message = (
-            f"{field_location} must be a valid datetime in one of the following formats:"
-            "- 'YYYY-MM-DD' — Full date only"
-            "- 'YYYY-MM-DDThh:mm:ss' — Full date and time without milliseconds"
-            "- 'YYYY-MM-DDThh:mm:ss.f' — Full date and time with milliseconds (any level of precision)"
-            "- 'YYYY-MM-DDThh:mm:ss%z' — Full date and time with timezone (e.g. +00:00 or +01:00)"
-            "- 'YYYY-MM-DDThh:mm:ss.f%z' — Full date and time with milliseconds and timezone"
-            "Only '+00:00' and '+01:00' are accepted as valid timezone offsets."
-            f"Note that partial dates are not allowed for {field_location} in this service."
-        )
-
         # Test invalid date time string formats
-        for invalid_occurrence_date_time in InvalidValues.for_date_time_string_formats:
+        for invalid_occurrence_date_time in invalid_datetime_formats:
             test_invalid_values_rejected(
                 test_instance,
                 valid_json_data,
diff --git a/backend/tests/utils/values_for_tests.py b/backend/tests/utils/values_for_tests.py
index 102327a18d..cfbcfcb8bd 100644
--- a/backend/tests/utils/values_for_tests.py
+++ b/backend/tests/utils/values_for_tests.py
@@ -31,7 +31,7 @@ class ValidValues:
 
     nhs_number = "9990548609"
 
-    for_date_times = [
+    for_date_times_strict_timezones = [
         "2000-01-01",  # Full date only
         "2000-01-01T00:00:00+00:00",  # Time and offset all zeroes
         "2000-01-01T10:34:27",        # Date with Time only
@@ -44,6 +44,12 @@ class ValidValues:
         "1933-12-31T11:11:11.111111+00:00",  # DateTime with milliseconds to 6 decimal places
     ]
 
+    for_date_times_relaxed_timezones = for_date_times_strict_timezones + [
+        "2000-01-01T00:00:00+05:00",  # Time and offset all zeroes
+        "1933-12-31T11:11:11-01:00",  # Negative offset (with hours and minutes not 0)
+        "1933-12-31T11:11:11.1-05:00",  # DateTime with milliseconds to 1 decimal place
+    ]
+
     for_strings_with_any_length_chars = (
         "This is a really long string with more than 100 characters to test whether the validator is working well!! "
     )
@@ -281,7 +287,7 @@ class InvalidValues:
     ]
 
     # Strings which are not in acceptable date time format
-    for_date_time_string_formats = [
+    for_date_time_string_formats_for_relaxed_timezone = [
         "",  # Empty string
         "invalid",  # Invalid format
         "20000101",  # Date digits only (i.e. without hypens)
@@ -290,10 +296,6 @@ class InvalidValues:
         "2000",  # Year only
         "2000-01",  # Year and month only
         "2000-01-01T00:00:00+00",  # Date and time with GMT timezone offset only in hours
-        "2000-01-01T00:00:00-00:00",  # Date and time with negative GMT timezone offset
-        "2000-01-01T00:00:00-01:00",  # Date and time with negative GMT timezone offset
-        "2000-01-01T00:00:00-05:00",  # Date and time with negative offset asides from GMT and BST
-        "2000-01-01T00:00:00+05:00",  # Date and time with offset asides from GMT and BST
         "2000-01-01T00:00:00+01",  # Date and time with BST timezone offset only in hours
         "12000-01-01T00:00:00+00:00",  # Extra character at start of string
         "2000-01-01T00:00:00+00:001",  # Extra character at end of string
@@ -302,7 +304,6 @@ class InvalidValues:
         "2000-01-0122:22:22+00:00.000",  # Missing T (with milliseconds)
         "2000-01-01T222222+00:00",  # Missing time colons
         "2000-01-01T22:22:2200:00",  # Missing timezone indicator
-        "2000-01-01T22:22:22-0100",  # Missing timezone colon
         "2000-01-01T22:22:22-01",  # Timezone hours only
         "99-01-01T00:00:00+00:00",  # Missing century (i.e. only 2 digits for year)
         "01-01-2000T00:00:00+00:00",  # Date in wrong order (DD-MM-YYYY)
@@ -323,6 +324,14 @@ class InvalidValues:
         "2000-01-01T00:00:00+00:60",  # Timezone minute 60
     ]
 
+    for_date_time_string_formats_for_strict_timezone = for_date_time_string_formats_for_relaxed_timezone + [
+        "2000-01-01T22:22:22-0100",  # Missing timezone colon
+        "2000-01-01T00:00:00-01:00",  # Date and time with negative GMT timezone offset
+        "2000-01-01T00:00:00-05:00",  # Date and time with negative offset asides from GMT and BST
+        "2000-01-01T00:00:00+05:00",  # Date and time with offset asides from GMT and BST
+        "2000-01-01T00:00:00-00:00",  # Date and time with negative GMT timezone offset
+    ]
+
     for_lists_of_strings_of_length_1 = [[1], [False], [["Test1"]]]
 
     for_lists_of_dicts_of_length_1 = [[1], [False], [["Invalid"]], ["Invalid"]]


Reviews:
user: login: mfjarvis
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-29T16:15:41Z
user: login: mfjarvis
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-29T16:17:25Z
user: login: Akol125
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-29T17:13:10Z
user: login: Akol125
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-29T18:21:33Z
user: login: mfjarvis
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-05-30T00:35:46Z

Review comments:
diff_hunk: @@ -323,6 +323,40 @@ class InvalidValues:
         "2000-01-01T00:00:00+00:60",  # Timezone minute 60
     ]
 
+    for_date_time_string_formats_for_recorded = [
path: backend/tests/utils/values_for_tests.py
user: login: mfjarvis
body: We could declare the invalid formats for the stricter validation in terms of the ones for the more relaxed validation, e.g.:
```
invalid_date_time_formats_for_relaxed_timezone = [
"",
"invalid",
"20000101",
...
]

invalid_date_time_formats_for_strict_timezone = invalid_date_time_formats_for_relaxed_timezone + [
"2000-01-01T00:00:00-01:00",
...
]
```
created_at: 2025-05-29T16:15:41Z
author_association: CONTRIBUTOR
diff_hunk: @@ -96,9 +96,18 @@ def for_date(field_value: str, field_location: str):
             raise ValueError(
                 f'{field_location} must be a valid date string in the format "YYYY-MM-DD"'
             ) from value_error
+    
+    @staticmethod
+    def for_recorded_date_time(field_value: str, field_location: str):
path: backend/src/models/utils/pre_validator_utils.py
user: login: mfjarvis
body: I'd prefer not to have the field name in the name of the validation method. This could potentially be used for other fields where we want to have the same relaxed validation.

Do we even need this method? Why not just call `for_date_time` but pass in `strict_timezone=False`? Then it's really clear what the difference is
created_at: 2025-05-29T16:17:25Z
author_association: CONTRIBUTOR
diff_hunk: @@ -323,6 +323,40 @@ class InvalidValues:
         "2000-01-01T00:00:00+00:60",  # Timezone minute 60
     ]
 
+    for_date_time_string_formats_for_recorded = [
path: backend/tests/utils/values_for_tests.py
user: login: Akol125
body: Makes total sense, it does reduce redundancy
created_at: 2025-05-29T17:13:09Z
author_association: CONTRIBUTOR
diff_hunk: @@ -96,9 +96,18 @@ def for_date(field_value: str, field_location: str):
             raise ValueError(
                 f'{field_location} must be a valid date string in the format "YYYY-MM-DD"'
             ) from value_error
+    
+    @staticmethod
+    def for_recorded_date_time(field_value: str, field_location: str):
path: backend/src/models/utils/pre_validator_utils.py
user: login: Akol125
body: Method Removed, now calls for_date_time directly with strict_timezone=False passed into it 
created_at: 2025-05-29T18:21:33Z
author_association: CONTRIBUTOR

PR comments:
user: login: sonarqubecloud[bot]
created_at: 2025-05-29T17:48:45Z
body: ## [![Quality Gate Passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-passed-20px.png 'Quality Gate Passed')](https://sonarcloud.io/dashboard?id=NHSDigital_immunisation-fhir-api&pullRequest=548) **Quality Gate passed**  
Issues  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0 New issues](https://sonarcloud.io/project/issues?id=NHSDigital_immunisation-fhir-api&pullRequest=548&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/accepted-16px.png '') [0 Accepted issues](https://sonarcloud.io/project/issues?id=NHSDigital_immunisation-fhir-api&pullRequest=548&issueStatuses=ACCEPTED)

Measures  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=NHSDigital_immunisation-fhir-api&pullRequest=548&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [100.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=NHSDigital_immunisation-fhir-api&pullRequest=548&metric=new_coverage&view=list)  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=NHSDigital_immunisation-fhir-api&pullRequest=548&metric=new_duplicated_lines_density&view=list)  
  
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=NHSDigital_immunisation-fhir-api&pullRequest=548)


author_association: NONE