delayedkarma/langchain-issues huggingface dataset:
get_openai_callback total_cost BROKEN

### System Info

Basically, when using "llm.generate" in combination with get_openai_callback the total_cost just outputs 0.

Code Snippet
```
from langchain.chat_models import ChatOpenAI
from langchain.schema import (
    AIMessage,
    HumanMessage,
    SystemMessage
)
from langchain.callbacks import get_openai_callback

chat = [{"role": "user", "content": "What's the weather like in Boston?"}]

for message in chat:
    if message["role"] == "assistant":
        messages.append(AIMessage(content=message["content"]))
    elif message["role"] == "user":
        messages.append(HumanMessage(content=message["content"]))

with get_openai_callback() as cb:
    res = llm.generate([messages])

print(cb) # Tokens Used is okey
print(cb) # Total Cost is Always 0
```

### Who can help?

@agola11 It's a Callback Issue. (That's why I am tagging you)

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```from langchain.chat_models import ChatOpenAI
from langchain.schema import (
    AIMessage,
    HumanMessage,
    SystemMessage
)
from langchain.callbacks import get_openai_callback

chat = [{"role": "user", "content": "What's the weather like in Boston?"}]

for message in chat:
    if message["role"] == "assistant":
        messages.append(AIMessage(content=message["content"]))
    elif message["role"] == "user":
        messages.append(HumanMessage(content=message["content"]))

with get_openai_callback() as cb:
    res = llm.generate([messages])

print(cb) # Tokens Used is okey
print(cb) # Total Cost is Always 0```


### Expected behavior

It should work the same way it works with chains or agents.
kiley-poole/Update OpenAI Chat Model to work with GPT-3-16k and GPT-4-32k

With the release of the new 16k context length Gpt-3 turbo model, I was running into issues with Buffer Summary Memory. I kept receiving the following error. 
`NotImplementedError: get_num_tokens_from_messages() is not presently implemented for model gpt-3.5-turbo-16k.See https://github.com/openai/openai-python/blob/main/chatml.md for information on how messages are converted to tokens.`
Turns out the `get_num_tokens_from_messages` function is reliant on the hard coded values for the models. It current has support for gpt-3.5 and 4, but neither longer context model. I've added support for both of these. I also updated the existing model version numbers to 0613 (the most current set) and updated the model costs.
About map_reduce.py

### System Info

When I'm using "gpt-3.5-turbo-16k" model,This model supports 16k token.However, using the mp-reduce algorithm, if the answer obtained at A time exceeds 4000 tokens, this will be reported. "A single document was longer than the context length,we cannot handle this." Error.
![image](https://github.com/hwchase17/langchain/assets/38323944/d315804b-1c2c-4942-a519-1b49f43e3d0e)
I don't think the token_max parameter changes for different models

### Who can help?

@hwchase17 @agola11 Hope to get help, this is very troublesome for my use

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

My code:
chain_one = load_summarize_chain(chat, chain_type="map_reduce",return_intermediate_steps=True,verbose=True, map_prompt=PROMPT, combine_prompt=combine_prompt)
x = chain_one({"input_documents": documents},return_only_outputs=True)

documents chunk_size is 4000 tokens

ERROR:File D:\conda\envs\ai\Lib\site-packages\langchain\chains\combine_documents\map_reduce.py:37, in _split_list_of_docs(docs, length_func, token_max, **kwargs)
     32     raise ValueError(
     33         "A single document was longer than the context length,"
     34         " we cannot handle this."
     35     )
     36 if len(_sub_result_docs) == 2:
---> 37     raise ValueError(
     38         "A single document was so long it could not be combined "
     39         "with another document, we cannot handle this."
     40     )
     41 new_result_doc_list.append(_sub_result_docs[:-1])
     42 _sub_result_docs = _sub_result_docs[-1:]

ValueError: A single document was so long it could not be combined with another document, we cannot handle this.


### Expected behavior

I hope that when I use the large token model, these errors will not occur
Updated index.rst to reflect langchain's LLM independence

<!--
Thank you for contributing to LangChain! Your PR will appear in our release under the title you set. Please make sure it highlights your valuable contribution.

Replace this with a description of the change, the issue it fixes (if applicable), and relevant context. List any dependencies required for this change.

After you're done, someone will review your PR. They may suggest improvements. If no one reviews your PR within a few days, feel free to @-mention the same people again, as notifications can get lost.

Finally, we'd love to show appreciation for your contribution - if you'd like us to shout you out on Twitter, please also include your handle!
-->

<!-- Remove if not applicable -->

Fixes #930  (issue)

#### Before submitting

<!-- If you're adding a new integration, please include:

1. a test for the integration - favor unit tests that does not rely on network access.
2. an example notebook showing its use


See contribution guidelines for more information on how to write tests, lint
etc:

https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
-->

#### Who can review?

Tag maintainers/contributors who might be interested:
@hwchase17 
<!-- For a quicker response, figure out the right person to tag with @

  @hwchase17 - project lead

  Tracing / Callbacks
  - @agola11

  Async
  - @agola11

  DataLoaders
  - @eyurtsev

  Models
  - @hwchase17
  - @agola11

  Agents / Tools / Toolkits
  - @hwchase17

  VectorStores / Retrievers / Memory
  - @dev2049

 -->
Update dev container

Fixes https://github.com/hwchase17/langchain/issues/6172

As described in https://github.com/hwchase17/langchain/issues/6172, I'd love to help update the dev container in this project.

**Summary of changes:**
- Dev container now builds (the current container in this repo won't build for me)
- Dockerfile updates
    - Update image to our [currently-maintained Python image](https://github.com/devcontainers/images/tree/main/src/python/.devcontainer) (`mcr.microsoft.com/devcontainers/python`) rather than the deprecated image from vscode-dev-containers
    - Move Dockerfile to root of repo - in order for `COPY` to work properly, it needs the files (in this case, `pyproject.toml` and `poetry.toml`) in the same directory
- devcontainer.json updates
     - Removed `customizations` and `remoteUser` since they should be covered by the updated image in the Dockerfile
     - Update comments
- Update docker-compose.yaml to properly point to updated Dockerfile
- Add a .gitattributes to avoid line ending conversions, which can result in hundreds of pending changes ([info](https://code.visualstudio.com/docs/devcontainers/tips-and-tricks#_resolving-git-line-ending-issues-in-containers-resulting-in-many-modified-files))
- Add a README in the .devcontainer folder and info on the dev container in the contributing.md

**Outstanding questions:**
- Is it expected for `poetry install` to take some time? It takes about 30 minutes for this dev container to finish building in a Codespace, but a user should only have to experience this once. Through some online investigation, this doesn't seem unusual
- Versions of poetry newer than 1.3.2 failed every time - based on some of the guidance in contributing.md and other online resources, it seemed changing poetry versions might be a good solution. 1.3.2 is from Jan 2023

#### Who can review?

Tag maintainers/contributors who might be interested: @vowelparrot

I was able to run through the various `make` commands mentioned in contributing.md successfully with this updated setup, and please let me know if there are questions or other tests I can help with to ensure everything is running well. Thanks so much!
Fix class promotion

In LangChain, all module classes are enumerated in the `__init__.py` file of the correspondent module. But some classes were missed and were not included in the module `__init__.py`

This PR:
- added the missed classes to the module `__init__.py` files
- `__init__.py:__all_` variable value (a list of the class names) was sorted
- `langchain.tools.sql_database.tool.QueryCheckerTool` was renamed into the `QuerySQLCheckerTool` because it conflicted with `langchain.tools.spark_sql.tool.QueryCheckerTool`
- changes to `pyproject.toml`:
  - added `pgvector` to `pyproject.toml:extended_testing`
  - added `pandas` to `pyproject.toml:[tool.poetry.group.test.dependencies]` 
  - commented out the `streamlit` from `collbacks/__init__.py`, It is because now the `streamlit` requires Python >=3.7, !=3.9.7
- fixed duplicate names in `tools`
- fixed correspondent ut-s

#### Who can review?
@hwchase17
@dev2049
Fixed a link typo /-/route -> /-/routes. and change endpoint format
Labels: name: lgtm

<!--
Thank you for contributing to LangChain! Your PR will appear in our release under the title you set. Please make sure it highlights your valuable contribution.

Replace this with a description of the change, the issue it fixes (if applicable), and relevant context. List any dependencies required for this change.

After you're done, someone will review your PR. They may suggest improvements. If no one reviews your PR within a few days, feel free to @-mention the same people again, as notifications can get lost.

Finally, we'd love to show appreciation for your contribution - if you'd like us to shout you out on Twitter, please also include your handle!
-->

<!-- Remove if not applicable -->

Fixes a link typo from `/-/route` to `/-/routes`. 
and change endpoint format
from `f"{self.anyscale_service_url}/{self.anyscale_service_route}"` to `f"{self.anyscale_service_url}{self.anyscale_service_route}"`
Also adding documentation about the format of the endpoint
#### Before submitting

<!-- If you're adding a new integration, please include:

1. a test for the integration - favor unit tests that does not rely on network access.
2. an example notebook showing its use


See contribution guidelines for more information on how to write tests, lint
etc:

https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
-->

#### Who can review?

Tag maintainers/contributors who might be interested:

<!-- For a quicker response, figure out the right person to tag with @

  @hwchase17 - project lead

  Tracing / Callbacks
  - @agola11

  Async
  - @agola11

  DataLoaders
  - @eyurtsev

  Models
  - @hwchase17
  - @agola11

  Agents / Tools / Toolkits
  - @hwchase17

  VectorStores / Retrievers / Memory
  - @dev2049

 -->
Segmentation fault (core dumped) 

### System Info

Python 3.9.7 
langchain '0.0.200'


### Who can help?

@hwchase17 
@agola11
@eyurtsev

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [X] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction


```
from langchain.embeddings import LlamaCppEmbeddings
from langchain.embeddings import HuggingFaceEmbeddings, SentenceTransformerEmbeddings 
from langchain.embeddings import OpenAIEmbeddings
import json
from langchain.retrievers import SVMRetriever


llama = LlamaCppEmbeddings(model_path="ggml-model-q4_0.bin")

"""
text = "This is a test document."
query_result = llama.embed_query(text)
print(query_result)
doc_result = llama.embed_documents([text])"""



text_list = ['The first Nobel Prize in Physics was awarded in 1901 to Wilhelm Conrad R\u00f6ntgen \"for his discovery of the remarkable rays subsequently named after him\".',
'The Nobel Prize in Physics is a yearly award given by the Royal Swedish Academy of Sciences for those who have made the most outstanding contributions for mankind in the field of physics. It is one of the five Nobel Prizes established by the 1895 will of Alfred Nobel, which are awarded for outstanding contributions in chemistry, physiology or medicine, literature, and physics. These prizes are awarded in Stockholm, Sweden. The first Nobel Prize in Physics was awarded to Wilhelm R\u00f6ntgen in 1901.',
'The next Deadpool movie is set to be released on June 1, 2018.'
]



retriever = SVMRetriever.from_texts(text_list, llama)#llama)

result = retriever.get_relevant_documents("who got the first nobel prize in physics")

print(result)
```



### Expected behavior

Hello All,

how can i use `LlamaCppEmbeddings(model_path="ggml-model-q4_0.bin")` as embedding for the retriever 
i tried using ggml-model-q4_0.bin but got `Segmentation fault (core dumped) `
following contributing.md results in being unable to run make

### System Info

OS: Ubuntu 20.04 LTS
branch: master
Python: 3.9

### Who can help?

@hwchase17 

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

I followed the steps as described in .github/contributing.md:

1. fork the repo
2. clone the fork
3. `conda create -n langchain python=3.9`
4. `conda activate langchain`
5. `pip install -U pip setuptools`
6. `pip install poetry`
7. `poetry completions bash >> ~/.bash_completion`
8. `poetry config virtualenvs.prefer-active-python true`
9. `poetry install -E all` <-- this command failed to unlock the gnome keyring, so I reran according to [poetry#1917](https://github.com/python-poetry/poetry/issues/1917) below:
10. `PYTHON_KEYRING_BACKEND=keyring.backends.null.Keyring poetry install -E all`
11. `make format`

Running `make format` results in the following:

```
Traceback (most recent call last):
  File "/home/kyle/.miniconda3/envs/langchain/bin/make", line 5, in <module>
    from scripts.proto import main
ModuleNotFoundError: No module named 'scripts'
```

`which make` returns a `make` script installed in my conda env bin, with the following contents:

```
#!/home/kyle/.miniconda3/envs/langchain/bin/python
# -*- coding: utf-8 -*-
import re
import sys
from scripts.proto import main
if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
    sys.exit(main())
```

I'm not sure what this is. I'm pretty sure I followed the directions correctly.

### Expected behavior

I should be able to run `make format` without a problem at this point.
Add the ability to run the map_reduce chains  process results step as async 

This will add the ability to add an AsyncCallbackManager (handler) for the reducer chain, which would be able to stream the tokens via the `async def on_llm_new_token` callback method



Fixes # (issue) [5532](https://github.com/hwchase17/langchain/issues/5532) 


 @hwchase17  @agola11 
The following code snippet explains how this change would be used to enable `reduce_llm` with streaming support in a `map_reduce` chain

I have tested this change and it works for the streaming use-case of reducer responses. I am happy to share more information if this makes solution sense. 

```

AsyncHandler
..........................
class StreamingLLMCallbackHandler(AsyncCallbackHandler):
    """Callback handler for streaming LLM responses."""

    def __init__(self, websocket):
        self.websocket = websocket
    
    # This callback method is to be executed in async
    async def on_llm_new_token(self, token: str, **kwargs: Any) -> None:
        resp = ChatResponse(sender="bot", message=token, type="stream")
        await self.websocket.send_json(resp.dict())


Chain
..........
stream_handler = StreamingLLMCallbackHandler(websocket)
stream_manager = AsyncCallbackManager([stream_handler])

streaming_llm = ChatOpenAI(
        streaming=True,
        callback_manager=stream_manager,
        verbose=False,
        temperature=0,
    )
    main_llm = OpenAI(
        temperature=0,
        verbose=False,
    )

    doc_chain = load_qa_chain(
        llm=main_llm,
        reduce_llm=streaming_llm,
        chain_type="map_reduce", 
        callback_manager=manager
    )
    qa_chain = ConversationalRetrievalChain(
        retriever=vectorstore.as_retriever(),
        combine_docs_chain=doc_chain,
        question_generator=question_generator,
        callback_manager=manager,
    )
    
    # Here `acall` will trigger `acombine_docs` on `map_reduce` which should then call `_aprocess_result` which in turn will call `self.combine_document_chain.arun` hence async callback will be awaited
    result = await qa_chain.acall(
         {"question": question, "chat_history": chat_history}
      )
```
Only using GPU:0 for vector embedding.

### System Info

torch.__version__ '2.0.1+cu117'
langchain.__version__ '0.0.199'
transformers.__version__ '4.30.2'

### Who can help?

@hwchase17
@agola11
@eyurtsev


### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [X] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

`source_dir = $DIRNAME
splitter = 'tiktoken'
num_similar = 4

emb_name = 'hkunlp/instructor-xl'
encode_kwargs = {'normalize_embeddings': True}

embeddings = HuggingFaceInstructEmbeddings(
    model_name=emb_name,
    model_kwargs={'device':'cuda'},
)

txt_loader = DirectoryLoader(source_dir, glob="**/*.txt")
documents = txt_loader.load()

text_splitter = RecursiveCharacterTextSplitter(chunk_size=chunk_size, 
                                             chunk_overlap=max_chunk_overlap,
                                            length_function = len)

documents = text_splitter.split_documents(documents)

vectorstore = Chroma.from_documents(documents, embeddings)
vectorstore.similarity_search(query)
`
or after loading vectorstore

`
retriever = vectorstore.as_retriever(search_type="mmr", search_kwargs={"k":num_similar} )

qa = ConversationalRetrievalChain.from_llm(
    hf_pipeline,
    retriever,
    # condense_question_llm = hf_pipeline, #this can be simpler
    return_source_documents=return_source_documents,
    return_generated_question=True,
    combine_docs_chain_kwargs = {'prompt': QA_PROMPT},
    condense_question_prompt = CONDENSE_QUESTION_PROMPT,
)
`
When we do something like this, even with device_map = 'auto' or 'balanced', we see much higher GPU consumption on GPU:0.

### Expected behavior

So I expect to load vectorDB balanced over all available GPUs like LLM pipelines. But, it only use GPU:0 which makes inefficient usage of VRAM of multiple GPUs.
add metaphor search new api fields

Update metaphor search tool with new API interface

ubmitting

<!-- If you're adding a new integration, please include:

1. a test for the integration - favor unit tests that does not rely on network access.
2. an example notebook showing its use


See contribution guidelines for more information on how to write tests, lint
etc:

https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
-->

#### Who can review?

Tag maintainers/contributors who might be interested:

@hwchase17 

<!-- For a quicker response, figure out the right person to tag with @

  @hwchase17 - project lead

  Tracing / Callbacks
  - @agola11

  Async
  - @agola11

  DataLoaders
  - @eyurtsev

  Models
  - @hwchase17
  - @agola11

  Agents / Tools / Toolkits
  - @hwchase17

  VectorStores / Retrievers / Memory
  - @dev2049

 -->
Issue: Update dev container configuration

### Issue you'd like to raise.

Hi folks! 👋 My name is Brigit, and I'm a PM on the VS Code team working on dev containers and [their open spec](https://containers.dev/). Thank you so much for [adding a dev container to this repo](https://github.com/hwchase17/langchain/pull/4035) and [langchainjs](https://github.com/hwchase17/langchainjs/pull/1241) - these are fantastic scenarios!

As we're actively working improvements to dev containers and their spec, we've made some changes to the best practices we recommend. For instance, we host an updated set of [images](https://github.com/devcontainers/images) and [templates](https://github.com/devcontainers/templates) as part of the spec in the [devcontainers org](https://github.com/devcontainers), rather than in [vscode-dev-containers](https://github.com/microsoft/vscode-dev-containers). It looks like the image in this repo uses the [deprecated vscode-dev-containers image](https://github.com/hwchase17/langchain/blob/master/.devcontainer/Dockerfile#L5), and perhaps it could leverage the [Poetry Feature](https://containers.dev/features) instead of Poetry installation scripts in the Dockerfile. 

I also tried building the dev container in this repo both in the VS Code Dev Containers extension and GitHub Codespaces, and it didn't work for me as-is (I was stopped at container build), so I think this would be a great opportunity to ensure the dev container works well for all potential contributors too. 

It looks like langchainjs uses an [updated image from the devcontainers org](https://github.com/hwchase17/langchainjs/blob/main/.devcontainer/devcontainer.json#L6), which is great!

### Suggestion:

I'd love to contribute a PR to this repo with an updated dev container (and perhaps with some additional info in the readme or a mini [.devcontainer](https://github.com/hwchase17/langchain/tree/master/.devcontainer) readme), but I wasn't sure the best tests to ensure the repo runs correctly in an updated dev container. Would you be able to share any recommended steps / commands / checks so that I can ensure any dev container updates work well for how folks should be building and running this repo? Info on how you tested and verified the [original PR](https://github.com/hwchase17/langchain/pull/4035) would be a great help too (so I can try the same steps).

Let me know if there's any other info I can provide, and I can also just open a draft a PR if that'd be easiest for discussion. Thanks so much!

cc @vowelparrot and @jj701 as I see your discussion in https://github.com/hwchase17/langchain/pull/4035.