Using AzureChatOpenAI
Labels: name: auto:bug, name: area: models

### System Info

I have just updated my LangChain packages. Until a few weeks ago, LangChain was working fine for me with my Azure OpenAI resource and deployment of the GPT-4-32K model. As I've gone to create more complex applications with it,  I got stuck at one section where I kept getting the error: "InvalidRequestError: The API deployment for this resource does not exist. If you created the deployment within the last 5 minutes, please wait a moment and try again."

I have given up on this error personally, but I currently have multiple Microsoft employees trying to help me figure it out. However, all of sudden, the basic implementations of LangChain now seem to be creating the same issue. Even though the native/direct API call to Azure OpenAI services is functioning correctly with the same credentials.

I am trying to use the following code to get the basic implementation working again, following directly what is written at https://python.langchain.com/docs/integrations/chat/azure_chat_openai :

model = AzureChatOpenAI(
    azure_deployment="Brian",
    openai_api_version="2023-05-15"
)

message = HumanMessage(
    content="Translate this sentence from English to French. I love programming."
)
print(model([message]))

This keeps getting the same error (as above with "the API deployment does not exist...") and I'm am completely stumped. How has LangChain gone from working to this error without me changing my credentials? And at the same time these credentials still work for the native Azure OpenAI API call. Any help would be massively appreciated.

### Who can help?

@hwchase17 @agola11

### Information

- [X] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

I have put the code above that you can use, but it seems more of an issue about how LangChain uses my Azure OpenAI account.

### Expected behavior

WARNING! azure_deployment is not default parameter.
                    azure_deployment was transferred to model_kwargs.
                    Please confirm that azure_deployment is what you intended.
---------------------------------------------------------------------------
InvalidRequestError                       Traceback (most recent call last)
Cell In[16], line 9
      1 model = AzureChatOpenAI(
      2     azure_deployment="Brian",
      3     openai_api_version="2023-05-15"
      4 )
      6 message = HumanMessage(
      7     content="Translate this sentence from English to French. I love programming."
      8 )
----> 9 print(model([message]))

File /anaconda/envs/azureml_py310_sdkv2/lib/python3.10/site-packages/langchain/chat_models/base.py:606, in BaseChatModel.__call__(self, messages, stop, callbacks, **kwargs)
    599 def __call__(
    600     self,
    601     messages: List[BaseMessage],
   (...)
    604     **kwargs: Any,
    605 ) -> BaseMessage:
--> 606     generation = self.generate(
    607         [messages], stop=stop, callbacks=callbacks, **kwargs
    608     ).generations[0][0]
    609     if isinstance(generation, ChatGeneration):
    610         return generation.message

File /anaconda/envs/azureml_py310_sdkv2/lib/python3.10/site-packages/langchain/chat_models/base.py:355, in BaseChatModel.generate(self, messages, stop, callbacks, tags, metadata, run_name, **kwargs)
    353         if run_managers:
    354             run_managers[i].on_llm_error(e)
--> 355         raise e
    356 flattened_outputs = [
    357     LLMResult(generations=[res.generations], llm_output=res.llm_output)
    358     for res in results
    359 ]
    360 llm_output = self._combine_llm_outputs([res.llm_output for res in results])

File /anaconda/envs/azureml_py310_sdkv2/lib/python3.10/site-packages/langchain/chat_models/base.py:345, in BaseChatModel.generate(self, messages, stop, callbacks, tags, metadata, run_name, **kwargs)
    342 for i, m in enumerate(messages):
    343     try:
    344         results.append(
--> 345             self._generate_with_cache(
    346                 m,
    347                 stop=stop,
    348                 run_manager=run_managers[i] if run_managers else None,
    349                 **kwargs,
    350             )
    351         )
    352     except BaseException as e:
    353         if run_managers:

File /anaconda/envs/azureml_py310_sdkv2/lib/python3.10/site-packages/langchain/chat_models/base.py:498, in BaseChatModel._generate_with_cache(self, messages, stop, run_manager, **kwargs)
    494     raise ValueError(
    495         "Asked to cache, but no cache found at `langchain.cache`."
    496     )
    497 if new_arg_supported:
--> 498     return self._generate(
    499         messages, stop=stop, run_manager=run_manager, **kwargs
    500     )
    501 else:
    502     return self._generate(messages, stop=stop, **kwargs)

File /anaconda/envs/azureml_py310_sdkv2/lib/python3.10/site-packages/langchain/chat_models/openai.py:360, in ChatOpenAI._generate(self, messages, stop, run_manager, stream, **kwargs)
    358 message_dicts, params = self._create_message_dicts(messages, stop)
    359 params = {**params, **kwargs}
--> 360 response = self.completion_with_retry(
    361     messages=message_dicts, run_manager=run_manager, **params
    362 )
    363 return self._create_chat_result(response)

File /anaconda/envs/azureml_py310_sdkv2/lib/python3.10/site-packages/langchain/chat_models/openai.py:299, in ChatOpenAI.completion_with_retry(self, run_manager, **kwargs)
    295 @retry_decorator
    296 def _completion_with_retry(**kwargs: Any) -> Any:
    297     return self.client.create(**kwargs)
--> 299 return _completion_with_retry(**kwargs)

File /anaconda/envs/azureml_py310_sdkv2/lib/python3.10/site-packages/tenacity/__init__.py:289, in BaseRetrying.wraps.<locals>.wrapped_f(*args, **kw)
    287 @functools.wraps(f)
    288 def wrapped_f(*args: t.Any, **kw: t.Any) -> t.Any:
--> 289     return self(f, *args, **kw)

File /anaconda/envs/azureml_py310_sdkv2/lib/python3.10/site-packages/tenacity/__init__.py:379, in Retrying.__call__(self, fn, *args, **kwargs)
    377 retry_state = RetryCallState(retry_object=self, fn=fn, args=args, kwargs=kwargs)
    378 while True:
--> 379     do = self.iter(retry_state=retry_state)
    380     if isinstance(do, DoAttempt):
    381         try:

File /anaconda/envs/azureml_py310_sdkv2/lib/python3.10/site-packages/tenacity/__init__.py:314, in BaseRetrying.iter(self, retry_state)
    312 is_explicit_retry = fut.failed and isinstance(fut.exception(), TryAgain)
    313 if not (is_explicit_retry or self.retry(retry_state)):
--> 314     return fut.result()
    316 if self.after is not None:
    317     self.after(retry_state)

File /anaconda/envs/azureml_py310_sdkv2/lib/python3.10/concurrent/futures/_base.py:451, in Future.result(self, timeout)
    449     raise CancelledError()
    450 elif self._state == FINISHED:
--> 451     return self.__get_result()
    453 self._condition.wait(timeout)
    455 if self._state in [CANCELLED, CANCELLED_AND_NOTIFIED]:

File /anaconda/envs/azureml_py310_sdkv2/lib/python3.10/concurrent/futures/_base.py:403, in Future.__get_result(self)
    401 if self._exception:
    402     try:
--> 403         raise self._exception
    404     finally:
    405         # Break a reference cycle with the exception in self._exception
    406         self = None

File /anaconda/envs/azureml_py310_sdkv2/lib/python3.10/site-packages/tenacity/__init__.py:382, in Retrying.__call__(self, fn, *args, **kwargs)
    380 if isinstance(do, DoAttempt):
    381     try:
--> 382         result = fn(*args, **kwargs)
    383     except BaseException:  # noqa: B902
    384         retry_state.set_exception(sys.exc_info())  # type: ignore[arg-type]

File /anaconda/envs/azureml_py310_sdkv2/lib/python3.10/site-packages/langchain/chat_models/openai.py:297, in ChatOpenAI.completion_with_retry.<locals>._completion_with_retry(**kwargs)
    295 @retry_decorator
    296 def _completion_with_retry(**kwargs: Any) -> Any:
--> 297     return self.client.create(**kwargs)

File /anaconda/envs/azureml_py310_sdkv2/lib/python3.10/site-packages/openai/api_resources/chat_completion.py:25, in ChatCompletion.create(cls, *args, **kwargs)
     23 while True:
     24     try:
---> 25         return super().create(*args, **kwargs)
     26     except TryAgain as e:
     27         if timeout is not None and time.time() > start + timeout:

File /anaconda/envs/azureml_py310_sdkv2/lib/python3.10/site-packages/openai/api_resources/abstract/engine_api_resource.py:155, in EngineAPIResource.create(cls, api_key, api_base, api_type, request_id, api_version, organization, **params)
    129 @classmethod
    130 def create(
    131     cls,
   (...)
    138     **params,
    139 ):
    140     (
    141         deployment_id,
    142         engine,
   (...)
    152         api_key, api_base, api_type, api_version, organization, **params
    153     )
--> 155     response, _, api_key = requestor.request(
    156         "post",
    157         url,
    158         params=params,
    159         headers=headers,
    160         stream=stream,
    161         request_id=request_id,
    162         request_timeout=request_timeout,
    163     )
    165     if stream:
    166         # must be an iterator
    167         assert not isinstance(response, OpenAIResponse)

File /anaconda/envs/azureml_py310_sdkv2/lib/python3.10/site-packages/openai/api_requestor.py:299, in APIRequestor.request(self, method, url, params, headers, files, stream, request_id, request_timeout)
    278 def request(
    279     self,
    280     method,
   (...)
    287     request_timeout: Optional[Union[float, Tuple[float, float]]] = None,
    288 ) -> Tuple[Union[OpenAIResponse, Iterator[OpenAIResponse]], bool, str]:
    289     result = self.request_raw(
    290         method.lower(),
    291         url,
   (...)
    297         request_timeout=request_timeout,
    298     )
--> 299     resp, got_stream = self._interpret_response(result, stream)
    300     return resp, got_stream, self.api_key

File /anaconda/envs/azureml_py310_sdkv2/lib/python3.10/site-packages/openai/api_requestor.py:710, in APIRequestor._interpret_response(self, result, stream)
    702     return (
    703         self._interpret_response_line(
    704             line, result.status_code, result.headers, stream=True
    705         )
    706         for line in parse_stream(result.iter_lines())
    707     ), True
    708 else:
    709     return (
--> 710         self._interpret_response_line(
    711             result.content.decode("utf-8"),
    712             result.status_code,
    713             result.headers,
    714             stream=False,
    715         ),
    716         False,
    717     )

File /anaconda/envs/azureml_py310_sdkv2/lib/python3.10/site-packages/openai/api_requestor.py:775, in APIRequestor._interpret_response_line(self, rbody, rcode, rheaders, stream)
    773 stream_error = stream and "error" in resp.data
    774 if stream_error or not 200 <= rcode < 300:
--> 775     raise self.handle_error_response(
    776         rbody, rcode, resp.data, rheaders, stream_error=stream_error
    777     )
    778 return resp

InvalidRequestError: The API deployment for this resource does not exist. If you created the deployment within the last 5 minutes, please wait a moment and try again.