Repository: langchain
Issue: Error when I try Cohere with Bedrock
Labels: 🤖:bug

### Checked other resources

- [X] I added a very descriptive title to this issue.
- [X] I searched the LangChain documentation with the integrated search.
- [X] I used the GitHub search to find a similar question and didn't find it.
- [X] I am sure that this is a bug in LangChain rather than my code.
- [X] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

### Example Code

This code

```
from langchain_aws import BedrockLLM

llm = BedrockLLM( region_name="us-east-1", model_id="cohere.command-r-plus-v1:0")

llm.invoke(input="What is the recipe of mayonnaise?")
```

Gave the following stack trace.

### Error Message and Stack Trace (if applicable)

ERROR:root:Error raised by bedrock service: An error occurred (ValidationException) when calling the InvokeModel operation: Malformed input request: #: extraneous key [prompt] is not permitted, please reformat your input and try again.
---------------------------------------------------------------------------
ValidationException                       Traceback (most recent call last)
Cell In[27], line 5
      1 from langchain_aws import BedrockLLM
      3 llm = BedrockLLM( region_name="us-east-1", model_id="cohere.command-r-plus-v1:0")
----> 5 llm.invoke(input="What is the recipe of mayonnaise?")

File [~/Library/Caches/pypoetry/virtualenvs/ai-retail-2HZsdhoR-py3.11/lib/python3.11/site-packages/langchain_core/language_models/llms.py:390](http://localhost:8888/lab/workspaces/auto-e/tree/~/Library/Caches/pypoetry/virtualenvs/ai-retail-2HZsdhoR-py3.11/lib/python3.11/site-packages/langchain_core/language_models/llms.py#line=389), in BaseLLM.invoke(self, input, config, stop, **kwargs)
    380 def invoke(
    381     self,
    382     input: LanguageModelInput,
   (...)
    386     **kwargs: Any,
    387 ) -> str:
    388     config = ensure_config(config)
    389     return (
--> 390         self.generate_prompt(
    391             [self._convert_input(input)],
    392             stop=stop,
    393             callbacks=config.get("callbacks"),
    394             tags=config.get("tags"),
    395             metadata=config.get("metadata"),
    396             run_name=config.get("run_name"),
    397             run_id=config.pop("run_id", None),
    398             **kwargs,
    399         )
    400         .generations[0][0]
    401         .text
    402     )

File [~/Library/Caches/pypoetry/virtualenvs/ai-retail-2HZsdhoR-py3.11/lib/python3.11/site-packages/langchain_core/language_models/llms.py:755](http://localhost:8888/lab/workspaces/auto-e/tree/~/Library/Caches/pypoetry/virtualenvs/ai-retail-2HZsdhoR-py3.11/lib/python3.11/site-packages/langchain_core/language_models/llms.py#line=754), in BaseLLM.generate_prompt(self, prompts, stop, callbacks, **kwargs)
    747 def generate_prompt(
    748     self,
    749     prompts: list[PromptValue],
   (...)
    752     **kwargs: Any,
    753 ) -> LLMResult:
    754     prompt_strings = [p.to_string() for p in prompts]
--> 755     return self.generate(prompt_strings, stop=stop, callbacks=callbacks, **kwargs)

File [~/Library/Caches/pypoetry/virtualenvs/ai-retail-2HZsdhoR-py3.11/lib/python3.11/site-packages/langchain_core/language_models/llms.py:950](http://localhost:8888/lab/workspaces/auto-e/tree/~/Library/Caches/pypoetry/virtualenvs/ai-retail-2HZsdhoR-py3.11/lib/python3.11/site-packages/langchain_core/language_models/llms.py#line=949), in BaseLLM.generate(self, prompts, stop, callbacks, tags, metadata, run_name, run_id, **kwargs)
    935 if (self.cache is None and get_llm_cache() is None) or self.cache is False:
    936     run_managers = [
    937         callback_manager.on_llm_start(
    938             self._serialized,
   (...)
    948         )
    949     ]
--> 950     output = self._generate_helper(
    951         prompts, stop, run_managers, bool(new_arg_supported), **kwargs
    952     )
    953     return output
    954 if len(missing_prompts) > 0:

File [~/Library/Caches/pypoetry/virtualenvs/ai-retail-2HZsdhoR-py3.11/lib/python3.11/site-packages/langchain_core/language_models/llms.py:792](http://localhost:8888/lab/workspaces/auto-e/tree/~/Library/Caches/pypoetry/virtualenvs/ai-retail-2HZsdhoR-py3.11/lib/python3.11/site-packages/langchain_core/language_models/llms.py#line=791), in BaseLLM._generate_helper(self, prompts, stop, run_managers, new_arg_supported, **kwargs)
    790     for run_manager in run_managers:
    791         run_manager.on_llm_error(e, response=LLMResult(generations=[]))
--> 792     raise e
    793 flattened_outputs = output.flatten()
    794 for manager, flattened_output in zip(run_managers, flattened_outputs):

File [~/Library/Caches/pypoetry/virtualenvs/ai-retail-2HZsdhoR-py3.11/lib/python3.11/site-packages/langchain_core/language_models/llms.py:779](http://localhost:8888/lab/workspaces/auto-e/tree/~/Library/Caches/pypoetry/virtualenvs/ai-retail-2HZsdhoR-py3.11/lib/python3.11/site-packages/langchain_core/language_models/llms.py#line=778), in BaseLLM._generate_helper(self, prompts, stop, run_managers, new_arg_supported, **kwargs)
    769 def _generate_helper(
    770     self,
    771     prompts: list[str],
   (...)
    775     **kwargs: Any,
    776 ) -> LLMResult:
    777     try:
    778         output = (
--> 779             self._generate(
    780                 prompts,
    781                 stop=stop,
    782                 # TODO: support multiple run managers
    783                 run_manager=run_managers[0] if run_managers else None,
    784                 **kwargs,
    785             )
    786             if new_arg_supported
    787             else self._generate(prompts, stop=stop)
    788         )
    789     except BaseException as e:
    790         for run_manager in run_managers:

File [~/Library/Caches/pypoetry/virtualenvs/ai-retail-2HZsdhoR-py3.11/lib/python3.11/site-packages/langchain_core/language_models/llms.py:1502](http://localhost:8888/lab/workspaces/auto-e/tree/~/Library/Caches/pypoetry/virtualenvs/ai-retail-2HZsdhoR-py3.11/lib/python3.11/site-packages/langchain_core/language_models/llms.py#line=1501), in LLM._generate(self, prompts, stop, run_manager, **kwargs)
   1499 new_arg_supported = inspect.signature(self._call).parameters.get("run_manager")
   1500 for prompt in prompts:
   1501     text = (
-> 1502         self._call(prompt, stop=stop, run_manager=run_manager, **kwargs)
   1503         if new_arg_supported
   1504         else self._call(prompt, stop=stop, **kwargs)
   1505     )
   1506     generations.append([Generation(text=text)])
   1507 return LLMResult(generations=generations)

File [~/Library/Caches/pypoetry/virtualenvs/ai-retail-2HZsdhoR-py3.11/lib/python3.11/site-packages/langchain_aws/llms/bedrock.py:1206](http://localhost:8888/lab/workspaces/auto-e/tree/~/Library/Caches/pypoetry/virtualenvs/ai-retail-2HZsdhoR-py3.11/lib/python3.11/site-packages/langchain_aws/llms/bedrock.py#line=1205), in BedrockLLM._call(self, prompt, stop, run_manager, **kwargs)
   1200         run_manager.on_llm_end(
   1201             LLMResult(generations=[all_generations], llm_output=llm_output)
   1202         )
   1204     return completion
-> 1206 text, tool_calls, llm_output = self._prepare_input_and_invoke(
   1207     prompt=prompt, stop=stop, run_manager=run_manager, **kwargs
   1208 )
   1209 if run_manager is not None:
   1210     run_manager.on_llm_end(
   1211         LLMResult(generations=[[Generation(text=text)]], llm_output=llm_output)
   1212     )

File [~/Library/Caches/pypoetry/virtualenvs/ai-retail-2HZsdhoR-py3.11/lib/python3.11/site-packages/langchain_aws/llms/bedrock.py:840](http://localhost:8888/lab/workspaces/auto-e/tree/~/Library/Caches/pypoetry/virtualenvs/ai-retail-2HZsdhoR-py3.11/lib/python3.11/site-packages/langchain_aws/llms/bedrock.py#line=839), in BedrockBase._prepare_input_and_invoke(self, prompt, system, messages, stop, run_manager, **kwargs)
    838     if run_manager is not None:
    839         run_manager.on_llm_error(e)
--> 840     raise e
    842 if stop is not None:
    843     text = enforce_stop_tokens(text, stop)

File [~/Library/Caches/pypoetry/virtualenvs/ai-retail-2HZsdhoR-py3.11/lib/python3.11/site-packages/langchain_aws/llms/bedrock.py:826](http://localhost:8888/lab/workspaces/auto-e/tree/~/Library/Caches/pypoetry/virtualenvs/ai-retail-2HZsdhoR-py3.11/lib/python3.11/site-packages/langchain_aws/llms/bedrock.py#line=825), in BedrockBase._prepare_input_and_invoke(self, prompt, system, messages, stop, run_manager, **kwargs)
    823         request_options["trace"] = "ENABLED"
    825 try:
--> 826     response = self.client.invoke_model(**request_options)
    828     (
    829         text,
    830         tool_calls,
   (...)
    833         stop_reason,
    834     ) = LLMInputOutputAdapter.prepare_output(provider, response).values()
    836 except Exception as e:

File [~/Library/Caches/pypoetry/virtualenvs/ai-retail-2HZsdhoR-py3.11/lib/python3.11/site-packages/botocore/client.py:569](http://localhost:8888/lab/workspaces/auto-e/tree/~/Library/Caches/pypoetry/virtualenvs/ai-retail-2HZsdhoR-py3.11/lib/python3.11/site-packages/botocore/client.py#line=568), in ClientCreator._create_api_method.<locals>._api_call(self, *args, **kwargs)
    565     raise TypeError(
    566         f"{py_operation_name}() only accepts keyword arguments."
    567     )
    568 # The "self" in this scope is referring to the BaseClient.
--> 569 return self._make_api_call(operation_name, kwargs)

File [~/Library/Caches/pypoetry/virtualenvs/ai-retail-2HZsdhoR-py3.11/lib/python3.11/site-packages/botocore/client.py:1023](http://localhost:8888/lab/workspaces/auto-e/tree/~/Library/Caches/pypoetry/virtualenvs/ai-retail-2HZsdhoR-py3.11/lib/python3.11/site-packages/botocore/client.py#line=1022), in BaseClient._make_api_call(self, operation_name, api_params)
   1019     error_code = error_info.get("QueryErrorCode") or error_info.get(
   1020         "Code"
   1021     )
   1022     error_class = self.exceptions.from_code(error_code)
-> 1023     raise error_class(parsed_response, operation_name)
   1024 else:
   1025     return parsed_response

ValidationException: An error occurred (ValidationException) when calling the InvokeModel operation: Malformed input request: #: extraneous key [prompt] is not permitted, please reformat your input and try again.

### Description

I''m just trying to get Cohere Command R+ to work! I need it to summarize text and to be used in `create_stuff_documents_chain`

### System Info

System Information
------------------
> OS:  Darwin
> OS Version:  Darwin Kernel Version 23.5.0: Wed May  1 20:12:58 PDT 2024; root:xnu-10063.121.3~5/RELEASE_ARM64_T6000
> Python Version:  3.11.9 (v3.11.9:de54cf5be3, Apr  2 2024, 07:12:50) [Clang 13.0.0 (clang-1300.0.29.30)]

Package Information
-------------------
> langchain_core: 0.3.12
> langchain: 0.3.3
> langchain_community: 0.3.2
> langsmith: 0.1.136
> langchain_aws: 0.2.2
> langchain_text_splitters: 0.3.0
> langgraph: 0.2.38

Optional packages not installed
-------------------------------
> langserve

Other Dependencies
------------------
> aiohttp: 3.10.10
> async-timeout: Installed. No version info available.
> boto3: 1.35.43
> dataclasses-json: 0.6.7
> httpx: 0.27.2
> jsonpatch: 1.33
> langgraph-checkpoint: 2.0.1
> langgraph-sdk: 0.1.33
> numpy: 1.26.4
> orjson: 3.10.7
> packaging: 24.1
> pydantic: 2.9.2
> pydantic-settings: 2.6.0
> PyYAML: 6.0.2
> requests: 2.32.3
> requests-toolbelt: 1.0.0
> SQLAlchemy: 2.0.36
> tenacity: 8.5.0
> typing-extensions: 4.12.2