AzureOpenAI azure_ad_token_provider Keyerror
Labels: name: auto:bug, name: area: models

### System Info

When I use below snippet of code 
```
import os
from azure.identity import DefaultAzureCredential
from azure.identity import get_bearer_token_provider
from langchain.llms import AzureOpenAI
from langchain.chat_models import AzureChatOpenAI

credential = DefaultAzureCredential(interactive_browser_tenant_id=tenant_id, 
                                                              interactive_browser_client_id=client_id, 
                                                              client_secret=client_secret)

token_provider = get_bearer_token_provider(credential, "https://cognitiveservices.azure.com/.default")

endpoint =  "https://xxxx.openai.azure.com"
client = AzureOpenAI(  azure_endpoint=endpoint,
                                       api_version="2023-05-15",
                                       azure_deployment="example-gpt-4",
                                       azure_ad_token_provider=token_provider)
```

I get error : 
```---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
Cell In[36], line 21
     18 # api_version = "2023-05-15"
     19 endpoint =  "https://xxxx.openai.azure.com"
---> 21 client = AzureOpenAI(
     22     azure_endpoint=endpoint,
     23     api_version="2023-05-15",
     24     azure_deployment="example-gpt-4",
     25     azure_ad_token_provider=token_provider,
     26 )

File ~/PycharmProjects/aicc/env/lib/python3.9/site-packages/langchain_core/load/serializable.py:97, in Serializable.__init__(self, **kwargs)
     96 def __init__(self, **kwargs: Any) -> None:
---> 97     super().__init__(**kwargs)
     98     self._lc_kwargs = kwargs

File ~/PycharmProjects/aicc/env/lib/python3.9/site-packages/pydantic/v1/main.py:339, in BaseModel.__init__(__pydantic_self__, **data)
    333 """
    334 Create a new model by parsing and validating input data from keyword arguments.
    335 
    336 Raises ValidationError if the input data cannot be parsed to form a valid model.
    337 """
    338 # Uses something other than `self` the first arg to allow "self" as a settable attribute
--> 339 values, fields_set, validation_error = validate_model(__pydantic_self__.__class__, data)
    340 if validation_error:
    341     raise validation_error

File ~/PycharmProjects/aicc/env/lib/python3.9/site-packages/pydantic/v1/main.py:1102, in validate_model(model, input_data, cls)
   1100     continue
   1101 try:
-> 1102     values = validator(cls_, values)
   1103 except (ValueError, TypeError, AssertionError) as exc:
   1104     errors.append(ErrorWrapper(exc, loc=ROOT_KEY))

File ~/PycharmProjects/aicc/env/lib/python3.9/site-packages/langchain/llms/openai.py:887, in AzureOpenAI.validate_environment(cls, values)
    877             values["openai_api_base"] += (
    878                 "/deployments/" + values["deployment_name"]
    879             )
    880         values["deployment_name"] = None
    881 client_params = {
    882     "api_version": values["openai_api_version"],
    883     "azure_endpoint": values["azure_endpoint"],
    884     "azure_deployment": values["deployment_name"],
    885     "api_key": values["openai_api_key"],
    886     "azure_ad_token": values["azure_ad_token"],
--> 887     "azure_ad_token_provider": values["azure_ad_token_provider"],
    888     "organization": values["openai_organization"],
    889     "base_url": values["openai_api_base"],
    890     "timeout": values["request_timeout"],
    891     "max_retries": values["max_retries"],
    892     "default_headers": values["default_headers"],
    893     "default_query": values["default_query"],
    894     "http_client": values["http_client"],
    895 }
    896 values["client"] = openai.AzureOpenAI(**client_params).completions
    897 values["async_client"] = openai.AsyncAzureOpenAI(
    898     **client_params
    899 ).completions

KeyError: 'azure_ad_token_provider'
```

Ive also tried AzureChatOpenAI , and I get the same error back. 
The error is not reproduced when I use openai library AzureOpenAI . 

Also on openai the azure_ad_token_provider has type azure_ad_token_provider: 'AzureADTokenProvider | None' = None  while in langchain it has type azure_ad_token_provider: Optional[str] = None  which also makes me wonder if it would take as input a different type than string to work with. 

any ideas on how to fix this? Im actually using Azure Service principal authentication, and if I use as alternative field azure_ad_token = credential.get_token(“https://cognitiveservices.azure.com/.default”).token I get token expired after 60min which does not happen with a bearer token, so It is important to me to make the token_provider work.


libraries : 
pydantic                               1.10.12
pydantic_core                      2.10.1
openai                                   1.2.0
langchain                              0.0.342
langchain-core                     0.0.7


### Who can help?

@hwchase17 @agola11

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

import os
from azure.identity import DefaultAzureCredential
from azure.identity import get_bearer_token_provider
from langchain.llms import AzureOpenAI
from langchain.chat_models import AzureChatOpenAI

credential = DefaultAzureCredential(interactive_browser_tenant_id=tenant_id, 
                                                              interactive_browser_client_id=client_id, 
                                                              client_secret=client_secret)

token_provider = get_bearer_token_provider(credential, "https://cognitiveservices.azure.com/.default")

endpoint =  "https://xxxx.openai.azure.com"
client = AzureOpenAI(  azure_endpoint=endpoint,
                                       api_version="2023-05-15",
                                       azure_deployment="example-gpt-4",
                                       azure_ad_token_provider=token_provider)

### Expected behavior

client = AzureOpenAI(  azure_endpoint=endpoint,
                                       api_version="2023-05-15",
                                       azure_deployment="example-gpt-4",
                                       azure_ad_token_provider=token_provider)

should return a Runnable instance which I can use for LLMChain