Repository: interagent/committee
Pull request #444: Add option to allow empty date and date-time strings
Author: login: zzak
Created: 2025-04-05T03:34:51Z | Merged: 2025-04-07T13:51:36Z

Depends on ota42y/openapi_parser#182.

I wasn't sure how to version guard this, but the tests pass using my branch. But this won't work without a version of openapi_parser which includes that option.

I chose to test the validation error message, but there weren't other examples like this in the code base so maybe unnecessary (testing the parser, not the tool). More just me making sure things were lined up properly.


Diff:
diff --git a/lib/committee/request_unpacker.rb b/lib/committee/request_unpacker.rb
index fa9fe67c..41d99038 100644
--- a/lib/committee/request_unpacker.rb
+++ b/lib/committee/request_unpacker.rb
@@ -21,11 +21,12 @@ def indifferent_params(object)
     end
 
     def initialize(options = {})
-      @allow_form_params          = options[:allow_form_params]
-      @allow_get_body             = options[:allow_get_body]
-      @allow_query_params         = options[:allow_query_params]
-      @allow_non_get_query_params = options[:allow_non_get_query_params]
-      @optimistic_json            = options[:optimistic_json]
+      @allow_empty_date_and_datetime = options[:allow_empty_date_and_datetime]
+      @allow_form_params             = options[:allow_form_params]
+      @allow_get_body                = options[:allow_get_body]
+      @allow_query_params            = options[:allow_query_params]
+      @allow_non_get_query_params    = options[:allow_non_get_query_params]
+      @optimistic_json               = options[:optimistic_json]
     end
 
     # return params and is_form_params
diff --git a/lib/committee/schema_validator/open_api_3.rb b/lib/committee/schema_validator/open_api_3.rb
index f3a31462..ddc1ecaf 100644
--- a/lib/committee/schema_validator/open_api_3.rb
+++ b/lib/committee/schema_validator/open_api_3.rb
@@ -84,6 +84,7 @@ def header(request)
 
       def request_unpack(request)
         unpacker = Committee::RequestUnpacker.new(
+          allow_empty_date_and_datetime: validator_option.allow_empty_date_and_datetime,
           allow_form_params:  validator_option.allow_form_params,
           allow_get_body:     validator_option.allow_get_body,
           allow_query_params: validator_option.allow_query_params,
diff --git a/lib/committee/schema_validator/open_api_3/operation_wrapper.rb b/lib/committee/schema_validator/open_api_3/operation_wrapper.rb
index 376c207a..bc4c093a 100644
--- a/lib/committee/schema_validator/open_api_3/operation_wrapper.rb
+++ b/lib/committee/schema_validator/open_api_3/operation_wrapper.rb
@@ -31,10 +31,12 @@ def coerce_path_parameter(validator_option)
         end
 
         # @param [Boolean] strict when not content_type or status code definition, raise error
-        def validate_response_params(status_code, headers, response_data, strict, check_header)
+        def validate_response_params(status_code, headers, response_data, strict, check_header, validator_options: {})
           response_body = OpenAPIParser::RequestOperation::ValidatableResponseBody.new(status_code, response_data, headers)
 
-          return request_operation.validate_response_body(response_body, response_validate_options(strict, check_header))
+          return request_operation.validate_response_body(
+            response_body,
+            response_validate_options(strict, check_header, validator_options: validator_options))
         rescue OpenAPIParser::OpenAPIError => e
           raise Committee::InvalidResponse.new(e.message, original_error: e)
         end
@@ -89,11 +91,15 @@ def build_openapi_parser_path_option(validator_option)
 
         # @return [OpenAPIParser::SchemaValidator::Options]
         def build_openapi_parser_option(validator_option, coerce_value)
-          datetime_coerce_class = validator_option.coerce_date_times ? DateTime : nil
-          validate_header = validator_option.check_header
-          OpenAPIParser::SchemaValidator::Options.new(coerce_value: coerce_value,
-                                                      datetime_coerce_class: datetime_coerce_class,
-                                                      validate_header: validate_header)
+          parser_options = {
+            coerce_value: coerce_value,
+            datetime_coerce_class: validator_option.coerce_date_times ? DateTime : nil,
+            validate_header: validator_option.check_header,
+          }
+          if OpenAPIParser::SchemaValidator::Options.method_defined?(:allow_empty_date_and_datetime)
+            parser_options[:allow_empty_date_and_datetime] = validator_option.allow_empty_date_and_datetime
+          end
+          OpenAPIParser::SchemaValidator::Options.new(**parser_options)
         end
 
         def validate_get_request_params(path_params, query_params, headers, validator_option)
@@ -130,8 +136,14 @@ def validate_path_and_query_params(path_params, query_params, headers, validator
           end
         end
 
-        def response_validate_options(strict, check_header)
-          ::OpenAPIParser::SchemaValidator::ResponseValidateOptions.new(strict: strict, validate_header: check_header)
+        def response_validate_options(strict, check_header, validator_options: {})
+          options = { strict: strict, validate_header: check_header }
+
+          if OpenAPIParser::SchemaValidator::ResponseValidateOptions.method_defined?(:validator_options)
+            ::OpenAPIParser::SchemaValidator::ResponseValidateOptions.new(**options, **validator_options)
+          else
+            ::OpenAPIParser::SchemaValidator::ResponseValidateOptions.new(**options)
+          end
         end
       end
     end
diff --git a/lib/committee/schema_validator/open_api_3/response_validator.rb b/lib/committee/schema_validator/open_api_3/response_validator.rb
index 692a94b7..d5659601 100644
--- a/lib/committee/schema_validator/open_api_3/response_validator.rb
+++ b/lib/committee/schema_validator/open_api_3/response_validator.rb
@@ -12,12 +12,15 @@ def initialize(operation_wrapper, validator_option)
           @operation_wrapper = operation_wrapper
           @validate_success_only = validator_option.validate_success_only
           @check_header = validator_option.check_header
+          @allow_empty_date_and_datetime = validator_option.allow_empty_date_and_datetime
         end
 
         def call(status, headers, response_data, strict)
           return unless Committee::Middleware::ResponseValidation.validate?(status, validate_success_only)
 
-          operation_wrapper.validate_response_params(status, headers, response_data, strict, check_header)
+          validator_options = { allow_empty_date_and_datetime: @allow_empty_date_and_datetime }
+
+          operation_wrapper.validate_response_params(status, headers, response_data, strict, check_header, validator_options: validator_options)
         end
 
         private
diff --git a/lib/committee/schema_validator/option.rb b/lib/committee/schema_validator/option.rb
index 73dcdd75..a70be297 100644
--- a/lib/committee/schema_validator/option.rb
+++ b/lib/committee/schema_validator/option.rb
@@ -5,6 +5,7 @@ module SchemaValidator
     class Option
       # Boolean Options
       attr_reader :allow_blank_structures,
+                  :allow_empty_date_and_datetime,
                   :allow_form_params,
                   :allow_get_body,
                   :allow_query_params,
@@ -36,6 +37,7 @@ def initialize(options, schema, schema_type)
 
         # Boolean options and have a common value by default
         @allow_blank_structures           = options.fetch(:allow_blank_structures, false)
+        @allow_empty_date_and_datetime    = options.fetch(:allow_empty_date_and_datetime, false)
         @allow_form_params                = options.fetch(:allow_form_params, true)
         @allow_query_params               = options.fetch(:allow_query_params, true)
         @allow_non_get_query_params       = options.fetch(:allow_non_get_query_params, false)
diff --git a/test/data/openapi3/normal.yaml b/test/data/openapi3/normal.yaml
index fd535293..57e6034c 100644
--- a/test/data/openapi3/normal.yaml
+++ b/test/data/openapi3/normal.yaml
@@ -205,6 +205,12 @@ paths:
         schema:
           type: string
           format: date-time
+      - name: date_string
+        in: query
+        required: false
+        schema:
+          type: string
+          format: date
       - name: normal_array
         in: query
         description: number data
@@ -683,7 +689,7 @@ paths:
         in: query
         required: false
         schema:
-          type: integer    
+          type: integer
       requestBody:
         content:
           application/json:
@@ -722,6 +728,26 @@ paths:
         '204':
           description: sample of remote schema reference
 
+  /date_time:
+    get:
+      responses:
+        '200':
+          description: success
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  date:
+                    type: string
+                    format: date
+                  date-time:
+                    type: string
+                    format: date-time
+              example:
+                date: 2020-05-12
+                date-time: 2020-05-12T00:00:00.00Z
+
 components:
   schemas:
     nested_array:
diff --git a/test/middleware/request_validation_open_api_3_test.rb b/test/middleware/request_validation_open_api_3_test.rb
index f434a7f7..e5be3066 100644
--- a/test/middleware/request_validation_open_api_3_test.rb
+++ b/test/middleware/request_validation_open_api_3_test.rb
@@ -146,7 +146,7 @@ def app
     assert_equal 200, last_response.status
   end
 
-  it "passes given an invalid datetime string with coerce_date_times enabled" do
+  it "errors given an invalid datetime string with coerce_date_times enabled" do
     @app = new_rack_app(schema: open_api_3_schema, coerce_date_times: true)
     params = { "datetime_string" => "invalid_datetime_format" }
     get "/string_params_coercer", params
@@ -244,6 +244,50 @@ def app
     assert_equal 200, last_response.status
   end
 
+  it "errors given an empty datetime string with allow_empty_date_and_datetime disabled" do
+    @app = new_rack_app(schema: open_api_3_schema)
+    params = { "datetime_string" => "" }
+    get "/string_params_coercer", params
+
+    assert_equal 400, last_response.status
+    assert_match(/\\"\\" is not conformant with date-time format/i, last_response.body)
+  end
+
+  it "passes given an empty datetime string with allow_empty_date_and_datetime enabled" do
+    @app = new_rack_app(schema: open_api_3_schema, allow_empty_date_and_datetime: true)
+    params = { "datetime_string" => "" }
+    get "/string_params_coercer", params
+
+    if OpenAPIParser::VERSION >= "2.2.5"
+      assert_equal 200, last_response.status
+    else
+      assert_equal 400, last_response.status
+      assert_match(/\\"\\" is not conformant with date-time format/i, last_response.body)
+    end
+  end
+
+  it "errors given an empty date string with allow_empty_date_and_datetime disabled" do
+    @app = new_rack_app(schema: open_api_3_schema)
+    params = { "date_string" => "" }
+    get "/string_params_coercer", params
+
+    assert_equal 400, last_response.status
+    assert_match(/\\"\\" is not conformant with date format/i, last_response.body)
+  end
+
+  it "passes given an empty date string with allow_empty_date_and_datetime enabled" do
+    @app = new_rack_app(schema: open_api_3_schema, allow_empty_date_and_datetime: true)
+    params = { "date_string" => "" }
+    get "/string_params_coercer", params
+
+    if OpenAPIParser::VERSION >= "2.2.5"
+      assert_equal 200, last_response.status
+    else
+      assert_equal 400, last_response.status
+      assert_match(/\\"\\" is not conformant with date format/i, last_response.body)
+    end
+  end
+
   it "OpenAPI3 detects an invalid request" do
     @app = new_rack_app(schema: open_api_3_schema, strict: true)
     header "Content-Type", "application/json"
diff --git a/test/schema_validator/open_api_3/response_validator_test.rb b/test/schema_validator/open_api_3/response_validator_test.rb
index 5ee6bde0..77ccc91d 100644
--- a/test/schema_validator/open_api_3/response_validator_test.rb
+++ b/test/schema_validator/open_api_3/response_validator_test.rb
@@ -75,6 +75,76 @@
     call_response_validator
   end
 
+  describe 'allow_empty_date_and_datetime' do
+    it "errors given an empty date string with allow_empty_date_and_datetime disabled" do
+      @path = '/date_time'
+      @method = 'get'
+      @data = { 'date' => '' }
+
+      e = assert_raises(Committee::InvalidResponse) {
+        call_response_validator
+      }
+      assert_kind_of(OpenAPIParser::OpenAPIError, e.original_error)
+      assert_match(/\"\" is not conformant with date format/i, e.message)
+    end
+
+    it "errors given an empty date-time string with allow_empty_date_and_datetime disabled" do
+      @path = '/date_time'
+      @method = 'get'
+      @data = { 'date-time' => '' }
+
+      e = assert_raises(Committee::InvalidResponse) {
+        call_response_validator
+      }
+      assert_kind_of(OpenAPIParser::OpenAPIError, e.original_error)
+      assert_match(/\"\" is not conformant with date-time format/i, e.message)
+    end
+
+    it "passes given an empty date string with allow_empty_date_and_datetime enabled" do
+      @validator_option = Committee::SchemaValidator::Option.new(
+        { allow_empty_date_and_datetime: true },
+        open_api_3_schema,
+        :open_api_3)
+
+      @path = '/date_time'
+      @method = 'get'
+      @data = { 'date' => '' }
+
+      if OpenAPIParser::VERSION >= "2.2.6"
+        response = call_response_validator
+        assert_equal({ 'date' => '' }, response)
+      else
+        e = assert_raises(Committee::InvalidResponse) {
+          call_response_validator
+        }
+        assert_kind_of(OpenAPIParser::OpenAPIError, e.original_error)
+        assert_match(/\"\" is not conformant with date format/i, e.message)
+      end
+    end
+
+    it "passes given an empty date-time string with allow_empty_date_and_datetime enabled" do
+      @validator_option = Committee::SchemaValidator::Option.new(
+        { allow_empty_date_and_datetime: true },
+        open_api_3_schema,
+        :open_api_3)
+
+      @path = '/date_time'
+      @method = 'get'
+      @data = { 'date-time' => '' }
+
+      if OpenAPIParser::VERSION >= "2.2.6"
+        response = call_response_validator
+        assert_equal({ 'date-time' => '' }, response)
+      else
+        e = assert_raises(Committee::InvalidResponse) {
+          call_response_validator
+        }
+        assert_kind_of(OpenAPIParser::OpenAPIError, e.original_error)
+        assert_match(/\"\" is not conformant with date-time format/i, e.message)
+      end
+    end
+  end
+
   private
 
   def call_response_validator(strict = false)


Reviews:
user: login: geemus
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-05T14:19:21Z
user: login: geemus
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-05T14:19:34Z
user: login: geemus
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-05T14:21:13Z
user: login: geemus
body: Looks good overall, suggested a couple of fixes for the test names and a possible way to guard for backwards compatibility, let me know what you think. Thanks!
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-05T14:21:51Z
user: login: zzak
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-06T01:54:48Z
user: login: geemus
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-07T13:43:57Z
user: login: geemus
body: lgtm, thanks!
state: APPROVED
author_association: MEMBER
submitted_at: 2025-04-07T13:51:30Z

Review comments:
diff_hunk: @@ -244,6 +244,40 @@ def app
     assert_equal 200, last_response.status
   end
 
+  it "errors given an empty datetime string with allow_empty_date_and_datetime disabled" do
+    @app = new_rack_app(schema: open_api_3_schema)
+    params = { "datetime_string" => "" }
+    get "/string_params_coercer", params
+
+    assert_equal 400, last_response.status
+    assert_match(/\\"\\" is not conformant with date-time format/i, last_response.body)
+  end
+
+  it "passes given an empty datetime string with allow_empty_date_and_datetime disabled" do
path: test/middleware/request_validation_open_api_3_test.rb
user: login: geemus
body: ```suggestion
  it "passes given an empty datetime string with allow_empty_date_and_datetime enabled" do
```
created_at: 2025-04-05T14:19:20Z
author_association: MEMBER
diff_hunk: @@ -244,6 +244,40 @@ def app
     assert_equal 200, last_response.status
   end
 
+  it "errors given an empty datetime string with allow_empty_date_and_datetime disabled" do
+    @app = new_rack_app(schema: open_api_3_schema)
+    params = { "datetime_string" => "" }
+    get "/string_params_coercer", params
+
+    assert_equal 400, last_response.status
+    assert_match(/\\"\\" is not conformant with date-time format/i, last_response.body)
+  end
+
+  it "passes given an empty datetime string with allow_empty_date_and_datetime disabled" do
+    @app = new_rack_app(schema: open_api_3_schema, allow_empty_date_and_datetime: true)
+    params = { "datetime_string" => "" }
+    get "/string_params_coercer", params
+
+    assert_equal 200, last_response.status
+  end
+
+  it "errors given an empty date string with allow_empty_date_and_datetime disabled" do
+    @app = new_rack_app(schema: open_api_3_schema)
+    params = { "date_string" => "" }
+    get "/string_params_coercer", params
+
+    assert_equal 400, last_response.status
+    assert_match(/\\"\\" is not conformant with date format/i, last_response.body)
+  end
+
+  it "passes given an empty date string with allow_empty_date_and_datetime disabled" do
path: test/middleware/request_validation_open_api_3_test.rb
user: login: geemus
body: ```suggestion
  it "passes given an empty date string with allow_empty_date_and_datetime enabled" do
```
created_at: 2025-04-05T14:19:33Z
author_association: MEMBER
diff_hunk: @@ -89,9 +89,11 @@ def build_openapi_parser_path_option(validator_option)
 
         # @return [OpenAPIParser::SchemaValidator::Options]
         def build_openapi_parser_option(validator_option, coerce_value)
+          allow_empty_date_and_datetime = validator_option.allow_empty_date_and_datetime
           datetime_coerce_class = validator_option.coerce_date_times ? DateTime : nil
           validate_header = validator_option.check_header
-          OpenAPIParser::SchemaValidator::Options.new(coerce_value: coerce_value,
+          OpenAPIParser::SchemaValidator::Options.new(allow_empty_date_and_datetime: allow_empty_date_and_datetime,
path: lib/committee/schema_validator/open_api_3/operation_wrapper.rb
user: login: geemus
body: I think a good backwards compatible guard for this might be something like:
```
if OpenAPIParser::Config.instance_methods.include?(:allow_empty_date_and_datetime)
```

So if that is true, you know it recognizes the option and you pass it in. And if it isn't defined it would probably be good to output a warning that tells you it won't do anything and if you need/want it to that you would need to update versions or something.

What do you think?
created_at: 2025-04-05T14:21:13Z
author_association: MEMBER
diff_hunk: @@ -89,9 +89,11 @@ def build_openapi_parser_path_option(validator_option)
 
         # @return [OpenAPIParser::SchemaValidator::Options]
         def build_openapi_parser_option(validator_option, coerce_value)
+          allow_empty_date_and_datetime = validator_option.allow_empty_date_and_datetime
           datetime_coerce_class = validator_option.coerce_date_times ? DateTime : nil
           validate_header = validator_option.check_header
-          OpenAPIParser::SchemaValidator::Options.new(coerce_value: coerce_value,
+          OpenAPIParser::SchemaValidator::Options.new(allow_empty_date_and_datetime: allow_empty_date_and_datetime,
path: lib/committee/schema_validator/open_api_3/operation_wrapper.rb
user: login: zzak
body: Good idea, I went with this:

```
if OpenAPIParser::SchemaValidator::Options.method_defined?(:allow_empty_date_and_datetime)
```

And then splat the keyword arguments.

I'm not sure if maybe using `method_defined?` is better because it allocates one less array (I didn't benchmark) but for us only using this in CI it doesn't matter. Happy to change it though if you have opinions.
created_at: 2025-04-06T01:54:48Z
author_association: CONTRIBUTOR
diff_hunk: @@ -89,9 +89,11 @@ def build_openapi_parser_path_option(validator_option)
 
         # @return [OpenAPIParser::SchemaValidator::Options]
         def build_openapi_parser_option(validator_option, coerce_value)
+          allow_empty_date_and_datetime = validator_option.allow_empty_date_and_datetime
           datetime_coerce_class = validator_option.coerce_date_times ? DateTime : nil
           validate_header = validator_option.check_header
-          OpenAPIParser::SchemaValidator::Options.new(coerce_value: coerce_value,
+          OpenAPIParser::SchemaValidator::Options.new(allow_empty_date_and_datetime: allow_empty_date_and_datetime,
path: lib/committee/schema_validator/open_api_3/operation_wrapper.rb
user: login: geemus
body: Yeah, `method_defined?` seems better. I was looking for that and didn't find it at the time, so grabbed something that would work (and hopefully get the point across). Thanks for finding the cleaner way.
created_at: 2025-04-07T13:43:57Z
author_association: MEMBER

PR comments:
user: login: zzak
created_at: 2025-04-06T04:17:53Z
body: Hang on, I think we also need this in the response validator too.

```
Committee::InvalidResponse:
   #/components/schemas/Component/properties/originated_at Value: "" is not conformant with date-time format
 # lib/committee/schema_validator/open_api_3/operation_wrapper.rb:39:in `rescue in validate_response_params'
 # lib/committee/schema_validator/open_api_3/operation_wrapper.rb:34:in `validate_response_params'
 # lib/committee/schema_validator/open_api_3/response_validator.rb:20:in `call'
 # lib/committee/schema_validator/open_api_3.rb:46:in `response_validate'
 # lib/committee/test/methods.rb:40:in `assert_response_schema_confirm'
 # lib/committee/test/methods.rb:8:in `assert_schema_conform'
```

I'm looking into it, but I think we may need to patch `openapi_parser` again :sob: 
author_association: CONTRIBUTOR
user: login: zzak
created_at: 2025-04-06T09:45:15Z
body: Ok, I've tested this with ota42y/openapi_parser#183 and everything is working now.
author_association: CONTRIBUTOR
user: login: geemus
created_at: 2025-04-07T13:52:32Z
body: @zzak thanks again for working through that. Is that everything you needed to change? Do we just need a committee release now?
author_association: MEMBER
user: login: zzak
created_at: 2025-04-07T13:55:32Z
body: Thank you!! :bow:
Our tests are all passing with branches so I assume release shouldn't break anything. I will update it in my day tomorrow and let you know if anything goes wrong.
author_association: CONTRIBUTOR
user: login: geemus
created_at: 2025-04-07T13:59:07Z
body: Sounds good.

Released v5.5.3

Just let me know if anything comes up.
author_association: MEMBER
user: login: zzak
created_at: 2025-04-07T23:30:49Z
body: :tada: Everything is working, thanks again for all your help!
author_association: CONTRIBUTOR
user: login: geemus
created_at: 2025-04-07T23:42:44Z
body: Awesome, glad to hear it!
author_association: MEMBER