Repository: pydantic/pydantic
Pull request: Do not compute JSON Schema default when plain serializers are used with `when_used` set to `'json-unless-none'` and the default value is `None`
Fixes #10118

<!-- Thank you for your contribution! -->
<!-- Unless your change is trivial, please create an issue to discuss the change before creating a PR -->

## Change Summary

<!-- Please give a short summary of the changes. -->

## Related issue number

<!-- WARNING: please use "fix #123" style references so the issue is closed when this PR is merged. -->

## Checklist

* [ ] The pull request title is a good summary of the changes - it will be used in the changelog
* [ ] Unit tests for the changes exist
* [ ] Tests pass on CI
* [ ] Documentation reflects the changes where applicable
* [ ] My PR is ready to review, **please add a comment including the phrase "please review" to assign reviewers**


Problem statement:
`PlainSerializer` not respecting `when_used` field during generation of json schema
### Initial Checks

- [X] I confirm that I'm using Pydantic V2

### Description

We've a `PlainSerializer` e.g.:

```python
Decimal = Annotated[
    _Decimal, PlainSerializer(lambda x: float(x), return_type=float, when_used='json-unless-none')
]

class MyModel(BaseModel):
  my_decimal: Decimal = Field(default=None)

my_model = MyModel(my_decimal=Decimal(1.0))
logger.info("schema {schema}", schema=my_model.model_json_schema(mode='serialization'))
```
which is not None-safe, however the lambda will be called during the schema generation

The schema generattion fails, cause the serialisation lambda will be called either way in this snippet in pydantic (json_schema.py:1065):

```python
if self.mode == 'serialization':
            if (
                (ser_schema := schema['schema'].get('serialization', {}))
                and (ser_func := ser_schema.get('function'))
                and ser_schema.get('type') == 'function-plain'  # type: ignore
                and ser_schema.get('info_arg') is False  # type: ignore
            ):
                default = ser_func(default)  # type: ignore
```

this started after upgrading these packages:

```
  - Updating pydantic-core (2.18.4 -> 2.20.1)
  - Updating pydantic (2.7.4 -> 2.8.2)
```



### Example Code

```Python
Decimal = Annotated[
    _Decimal, PlainSerializer(lambda x: float(x), return_type=float, when_used='json-unless-none')
]

class MyModel(BaseModel):
  my_decimal: Decimal = Field(default=None)

my_model = MyModel(my_decimal=Decimal(1.0))
logger.info("schema {schema}", schema=my_model.model_json_schema(mode='serialization'))
```
```


### Python, Pydantic & OS Version

```Text
python -c "import pydantic.version; print(pydantic.version.version_info())"                                                    ✔  3.11.0   11:35:45
             pydantic version: 2.8.2
        pydantic-core version: 2.20.1
          pydantic-core build: profile=release pgo=true
                 install path: /Users/.../.venv/lib/python3.11/site-packages/pydantic
               python version: 3.11.0 (main, Nov 18 2022, 12:41:05) [Clang 14.0.0 (clang-1400.0.29.102)]
                     platform: macOS-14.6.1-arm64-arm-64bit
             related packages: mypy-0.931 fastapi-0.101.0 typing_extensions-4.12.2 pydantic-settings-2.4.0
                       commit: unknown
```

Hints from issue discussion:

Commit message: Do not compute JSON Schema default when plain serializers are used with `when_used` set to `'json-unless-none'` and the default value is `None`

Patch to review:
diff --git a/pydantic/json_schema.py b/pydantic/json_schema.py
index 30cd16cdc73..fbd2a3d3ad0 100644
--- a/pydantic/json_schema.py
+++ b/pydantic/json_schema.py
@@ -1043,17 +1043,17 @@ def default_schema(self, schema: core_schema.WithDefaultSchema) -> JsonSchemaVal
         #     return json_schema
 
         # we reflect the application of custom plain, no-info serializers to defaults for
-        # json schemas viewed in serialization mode
+        # JSON Schemas viewed in serialization mode:
         # TODO: improvements along with https://github.com/pydantic/pydantic/issues/8208
-        # TODO: improve type safety here
-        if self.mode == 'serialization':
-            if (
-                (ser_schema := schema['schema'].get('serialization', {}))
-                and (ser_func := ser_schema.get('function'))
-                and ser_schema.get('type') == 'function-plain'  # type: ignore
-                and ser_schema.get('info_arg') is False  # type: ignore
-            ):
-                default = ser_func(default)  # type: ignore
+        if (
+            self.mode == 'serialization'
+            and (ser_schema := schema['schema'].get('serialization'))
+            and (ser_func := ser_schema.get('function'))
+            and ser_schema.get('type') == 'function-plain'
+            and not ser_schema.get('info_arg')
+            and not (default is None and ser_schema.get('when_used') == 'json-unless-none')
+        ):
+            default = ser_func(default)  # type: ignore
 
         try:
             encoded_default = self.encode_default(default)
diff --git a/tests/test_json_schema.py b/tests/test_json_schema.py
index fe36b17ec24..0b2f0aef01b 100644
--- a/tests/test_json_schema.py
+++ b/tests/test_json_schema.py
@@ -6257,6 +6257,22 @@ class Model(BaseModel):
     }
 
 
+def test_plain_serializer_does_not_apply_json_unless_none() -> None:
+    """Test plain serializers aren't used to compute the JSON Schema default if mode is 'json-unless-none'
+    and default value is `None`."""
+
+    class Model(BaseModel):
+        custom_decimal: Annotated[
+            Optional[Decimal], PlainSerializer(lambda x: float(x), when_used='json-unless-none', return_type=float)
+        ] = None
+
+    assert Model.model_json_schema(mode='serialization') == {
+        'properties': {'custom_decimal': {'default': None, 'title': 'Custom Decimal', 'type': 'number'}},
+        'title': 'Model',
+        'type': 'object',
+    }
+
+
 def test_merge_json_schema_extra_from_field_infos() -> None:
     class Model(BaseModel):
         f: Annotated[str, Field(json_schema_extra={'a': 1, 'b': 2})]


Code review comments:
diff_hunk: @@ -1043,17 +1043,30 @@ def default_schema(self, schema: core_schema.WithDefaultSchema) -> JsonSchemaVal
         #     return json_schema
 
         # we reflect the application of custom plain, no-info serializers to defaults for
-        # json schemas viewed in serialization mode
+        # JSON Schemas viewed in serialization mode:
         # TODO: improvements along with https://github.com/pydantic/pydantic/issues/8208
-        # TODO: improve type safety here
-        if self.mode == 'serialization':
-            if (
-                (ser_schema := schema['schema'].get('serialization', {}))
-                and (ser_func := ser_schema.get('function'))
-                and ser_schema.get('type') == 'function-plain'  # type: ignore
-                and ser_schema.get('info_arg') is False  # type: ignore
-            ):
+        if (
+            self.mode == 'serialization'
+            and (ser_schema := schema['schema'].get('serialization'))
+            and (ser_func := ser_schema.get('function'))
+            and ser_schema.get('type') == 'function-plain'
+            and not ser_schema.get('info_arg')
+            and not (default is None and ser_schema.get('when_used') == 'json-unless-none')
+        ):
+            try:
                 default = ser_func(default)  # type: ignore
+            except Exception:
+                # It might be that the provided default needs to be validated first
+                # (assuming `validate_default` is enabled). However, we can't perform
+                # such validation during JSON Schema generation so we don't support
+                # this pattern for now.
+                # (One example is when using `foo: ByteSize = '1MB'`, which validates and
+                # serializes as an int. In this case, `ser_func` is `int` and `int('1MB')` fails).
+                self.emit_warning(
+                    'non-serializable-default',
+                    f'Unable to serialize value {default!r} with the plain serializer; excluding default from JSON schema',
+                )
+                return json_schema
path: pydantic/json_schema.py
text: @user1:
Could we do something like this?

https://github.com/pydantic/pydantic/blob/0c97049bd40ac95cce7bbebad8369b9426af4b09/pydantic/json_schema.py#L2017-L2044

I feel like we shouldn't be skipping that logic here, we could maybe call that function?

@author:
I don't think it makes sense to do so, or we may end up with a wrong default value in the JSON Schema (in the related test fixed, `'1MB'` was set as the default value in the JSON Schema, but this isn't true as the default is `1000000`). In my opinion, it's better to have no value than a wrong one.

However, we emit a warning here, so I'm ok to call `encode_default`.