Repository: langchain
Issue: ChatOpenAI with structured output, outputs always all field of my pydantic object.
Labels: 🤖:bug,investigate,Ɑ:  core

### Checked other resources

- [X] I added a very descriptive title to this issue.
- [X] I searched the LangChain documentation with the integrated search.
- [X] I used the GitHub search to find a similar question and didn't find it.
- [X] I am sure that this is a bug in LangChain rather than my code.
- [X] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

### Example Code



```python
input_messages = [
SystemMessage(content='You are a helpful assistant please output valid json.'),
HumanMessage(content='What is the weather in Tokyo?'),
]

structured_llm = llm.with_structured_output(
output_class, include_raw=True
)
response: dict[str, Any] = await structured_llm.ainvoke(input_messages)  # type: ignore

``` 


Our output_class pydantic object decoded with 
```python
output_class.model_json_schema()
``` 
Looks like

```json
"$defs": {"ActionModel": {"properties": {"search_google": {"anyOf": [
                    {"$ref": "#/$defs/SearchGoogleAction"
                    },
                    {"type": "null"
                    }
                ], "default": None
            }, "go_to_url": {"anyOf": [
                    {"$ref": "#/$defs/GoToUrlAction"
                    },
                    {"type": "null"
                    }
                ], "default": None
            }, "go_back": {"anyOf": [
                    {"$ref": "#/$defs/go_backParams"
                    },
                    {"type": "null"
                    }
                ], "default": None
            }, "click_element": {"anyOf": [
                    {"$ref": "#/$defs/ClickElementAction"
                    },
                    {"type": "null"
                    }
                ], "default": None
            }, "input_text": {"anyOf": [
                    {"$ref": "#/$defs/InputTextAction"
                    },
                    {"type": "null"
                    }
                ], "default": None
            }, "switch_tab": {"anyOf": [
                    {"$ref": "#/$defs/SwitchTabAction"
                    },
                    {"type": "null"
                    }
                ], "default": None
            }, "open_tab": {"anyOf": [
                    {"$ref": "#/$defs/OpenTabAction"
                    },
                    {"type": "null"
                    }
                ], "default": None
            }, "extract_content": {"anyOf": [
                    {"$ref": "#/$defs/ExtractPageContentAction"
                    },
                    {"type": "null"
                    }
                ], "default": None
            }, "done": {"anyOf": [
                    {"$ref": "#/$defs/DoneAction"
                    },
                    {"type": "null"
                    }
                ], "default": None
            }, "scroll_down": {"anyOf": [
                    {"$ref": "#/$defs/ScrollAction"
                    },
                    {"type": "null"
                    }
                ], "default": None
            }, "scroll_up": {"anyOf": [
                    {"$ref": "#/$defs/ScrollAction"
                    },
                    {"type": "null"
                    }
                ], "default": None
            }, "send_keys": {"anyOf": [
                    {"$ref": "#/$defs/SendKeysAction"
                    },
                    {"type": "null"
                    }
                ], "default": None
            }, "scroll_to_text": {"anyOf": [
                    {"$ref": "#/$defs/scroll_to_textParams"
                    },
                    {"type": "null"
                    }
                ], "default": None
            }, "get_dropdown_options": {"anyOf": [
                    {"$ref": "#/$defs/get_dropdown_optionsParams"
                    },
                    {"type": "null"
                    }
                ], "default": None
            }, "select_dropdown_option": {"anyOf": [
                    {"$ref": "#/$defs/select_dropdown_optionParams"
                    },
                    {"type": "null"
                    }
                ], "default": None
            }
        }, "title": "ActionModel", "type": "object"
    }, "AgentBrain": {"description": "Current state of the agent", "properties": {"evaluation_previous_goal": {"title": "Evaluation Previous Goal", "type": "string"
            }, "memory": {"title": "Memory", "type": "string"
            }, "next_goal": {"title": "Next Goal", "type": "string"
            }
        }, "required": ["evaluation_previous_goal", "memory", "next_goal"
        ], "title": "AgentBrain", "type": "object"
    }, "ClickElementAction": {"properties": {"index": {"title": "Index", "type": "integer"
            }, "xpath": {"anyOf": [
                    {"type": "string"
                    },
                    {"type": "null"
                    }
                ], "default": None, "title": "Xpath"
            }
        }, "required": ["index"
        ], "title": "ClickElementAction", "type": "object"
    }, "DoneAction": {"properties": {"text": {"title": "Text", "type": "string"
            }
        }, "required": ["text"
        ], "title": "DoneAction", "type": "object"
    }, "ExtractPageContentAction": {"properties": {"include_links": {"title": "Include Links", "type": "boolean"
            }
        }, "required": ["include_links"
        ], "title": "ExtractPageContentAction", "type": "object"
    }, "GoToUrlAction": {"properties": {"url": {"title": "Url", "type": "string"
            }
        }, "required": ["url"
        ], "title": "GoToUrlAction", "type": "object"
    }, "InputTextAction": {"properties": {"index": {"title": "Index", "type": "integer"
            }, "text": {"title": "Text", "type": "string"
            }, "xpath": {"anyOf": [
                    {"type": "string"
                    },
                    {"type": "null"
                    }
                ], "default": None, "title": "Xpath"
            }
        }, "required": ["index", "text"
        ], "title": "InputTextAction", "type": "object"
    }, "OpenTabAction": {"properties": {"url": {"title": "Url", "type": "string"
            }
        }, "required": ["url"
        ], "title": "OpenTabAction", "type": "object"
    }, "ScrollAction": {"properties": {"amount": {"anyOf": [
                    {"type": "integer"
                    },
                    {"type": "null"
                    }
                ], "default": None, "title": "Amount"
            }
        }, "title": "ScrollAction", "type": "object"
    }, "SearchGoogleAction": {"properties": {"query": {"title": "Query", "type": "string"
            }
        }, "required": ["query"
        ], "title": "SearchGoogleAction", "type": "object"
    }, "SendKeysAction": {"properties": {"keys": {"title": "Keys", "type": "string"
            }
        }, "required": ["keys"
        ], "title": "SendKeysAction", "type": "object"
    }, "SwitchTabAction": {"properties": {"page_id": {"title": "Page Id", "type": "integer"
            }
        }, "required": ["page_id"
        ], "title": "SwitchTabAction", "type": "object"
    }, "get_dropdown_optionsParams": {"properties": {"index": {"title": "Index", "type": "integer"
            }
        }, "required": ["index"
        ], "title": "get_dropdown_optionsParams", "type": "object"
    }, "go_backParams": {"properties": {}, "title": "go_backParams", "type": "object"
    }, "scroll_to_textParams": {"properties": {"text": {"title": "Text", "type": "string"
            }
        }, "required": ["text"
        ], "title": "scroll_to_textParams", "type": "object"
    }, "select_dropdown_optionParams": {"properties": {"index": {"title": "Index", "type": "integer"
            }, "text": {"title": "Text", "type": "string"
            }
        }, "required": ["index", "text"
        ], "title": "select_dropdown_optionParams", "type": "object"
    }
}, "properties": {"current_state": {"$ref": "#/$defs/AgentBrain"
    }, "action": {"items": {"$ref": "#/$defs/ActionModel"
        }, "title": "Action", "type": "array"
    }
}, "required": ["current_state", "action"
], "title": "AgentOutput", "type": "object"
}
```

### Error Message and Stack Trace (if applicable)

Output:
```json
{
    "current_state": {
        "evaluation_previous_goal": "Determine the current weather in Tokyo.",
        "memory": "The user is interested in the current weather conditions in Tokyo.",
        "next_goal": "Search for the current weather in Tokyo using an online weather service or search engine."
    },
    "action": [
        {
            "search_google": {
                "query": "current weather in Tokyo"
            },
            "go_to_url": null,
            "go_back": null,
            "click_element": null,
            "input_text": null,
            "switch_tab": null,
            "open_tab": null,
            "extract_content": null,
            "done": null,
            "scroll_down": null,
            "scroll_up": null,
            "send_keys": null,
            "scroll_to_text": null,
            "get_dropdown_options": null,
            "select_dropdown_option": null
        }
    ]
}
``` 


Expected output:
```json
{
    "current_state": {
        "evaluation_previous_goal": "Determine the current weather in Tokyo.",
        "memory": "The user is interested in the current weather conditions in Tokyo.",
        "next_goal": "Search for the current weather in Tokyo using an online weather service or search engine."
    },
    "action": [
        {
            "search_google": {
                "query": "current weather in Tokyo"
            }
        }
    ]
}
``` 

### Description

Hey, I want to upgrade our library browser-use to your new 3.0.0 version. 

But with ChatOpenAI gpt-4 outputs all fields for the action key. But we only want to have one action of the keys, e.g. only "search_google". 
This save many tokens and also prevents the model from outputting multiple actions in the same element. 
In version 2.24 it worked and gave us our desired output. 

### System Info

System Information
------------------
> OS:  Darwin
> OS Version:  Darwin Kernel Version 23.1.0: Mon Oct  9 21:33:00 PDT 2023; root:xnu-10002.41.9~7/RELEASE_ARM64_T6031
> Python Version:  3.11.10 (main, Oct 16 2024, 08:56:36) [Clang 18.1.8 ]

Package Information
-------------------
> langchain_core: 0.3.29
> langchain: 0.3.14
> langchain_community: 0.3.14
> langsmith: 0.2.10
> langchain_anthropic: 0.3.1
> langchain_aws: 0.2.10
> langchain_fireworks: 0.2.5
> langchain_google: 0.1.1
> langchain_google_genai: 2.0.8
> langchain_mistralai: 0.2.4
> langchain_ollama: 0.2.2
> langchain_openai: 0.3.0
> langchain_text_splitters: 0.3.4

Optional packages not installed
-------------------------------
> langserve

Other Dependencies
------------------
> aiohttp: 3.11.11
> anthropic: 0.42.0
> async-timeout: 5.0.1
> boto3: 1.35.97
> dataclasses-json: 0.6.7
> defusedxml: 0.7.1
> filetype: 1.2.0
> fireworks-ai: 0.15.8
> google-generativeai: 0.8.3
> httpx: 0.28.1
> httpx-sse: 0.4.0
> jsonpatch: 1.33
> langsmith-pyo3: Installed. No version info available.
> numpy: 1.26.4
> ollama: 0.4.5
> openai: 1.59.7
> orjson: 3.10.14
> packaging: 24.2
> pydantic: 2.10.5
> pydantic-settings: 2.7.1
> PyYAML: 6.0.2
> requests: 2.32.3
> requests-toolbelt: 1.0.0
> SQLAlchemy: 2.0.36
> tenacity: 9.0.0
> tiktoken: 0.8.0
> tokenizers: 0.21.0
> typing-extensions: 4.12.2
> zstandard: 0.23.0
(.venv) magnus@full-speed agentweb %