Repository: lancedb/lancedb

Issue #680: RuntimeError with LanceDB Cloud Version in FastAPI Environment
Created: 2023-12-04T15:49:59Z
@AyushExel @fcakyon
Description
We are experiencing a RuntimeError when integrating the cloud version of LanceDB with FastAPI using our custom LanceDBVectorStore class in the autollm package. This issue occurs specifically when switching from a local LanceDB setup to a cloud-based setup.  We have implemented the feature branch that supports cloud feature of lancedb via overwriting the llama-index class in this branch and worked from there: https://github.com/safevideo/autollm/tree/feature/lancedb-cloud-support
Relevant Details

Link to related issue: Enhancement Request: Cloud Support in LanceDBVectorStore for Llama-Index #671
The issue is encountered in a FastAPI environment.
The LanceDBVectorStore class has been modified to support LanceDB's cloud version (referenced in the previous issue).
The error does not occur with local LanceDB setup but only with the cloud setup.
The cloud setup works fine outside the FastAPI environment, indicating the tables exist and are accessible.

Error Log
INFO:     Started server process [49449]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO:httpx:HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2023-12-04 18:22:06,735 - autollm - INFO - Embedding Token Usage: 6
ERROR:root:Table {name} does not exist.Please first call db.create_table({name}, data)
/home/seekndestroy/mambaforge/envs/ultralytics/lib/python3.10/site-packages/lancedb/remote/db.py:105: RuntimeWarning: coroutine 'RestfulLanceDBClient.post' was never awaited
  logging.error(
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
INFO:     127.0.0.1:33324 - "POST /query HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/home/seekndestroy/mambaforge/envs/ultralytics/lib/python3.10/site-packages/uvicorn/protocols/http/h11_impl.py", line 408, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/home/seekndestroy/mambaforge/envs/ultralytics/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
    return await self.app(scope, receive, send)
  File "/home/seekndestroy/mambaforge/envs/ultralytics/lib/python3.10/site-packages/fastapi/applications.py", line 292, in __call__
    await super().__call__(scope, receive, send)
  File "/home/seekndestroy/mambaforge/envs/ultralytics/lib/python3.10/site-packages/starlette/applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/seekndestroy/mambaforge/envs/ultralytics/lib/python3.10/site-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/home/seekndestroy/mambaforge/envs/ultralytics/lib/python3.10/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/home/seekndestroy/mambaforge/envs/ultralytics/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
    raise exc
  File "/home/seekndestroy/mambaforge/envs/ultralytics/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "/home/seekndestroy/mambaforge/envs/ultralytics/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 20, in __call__
    raise e
  File "/home/seekndestroy/mambaforge/envs/ultralytics/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 17, in __call__
    await self.app(scope, receive, send)
  File "/home/seekndestroy/mambaforge/envs/ultralytics/lib/python3.10/site-packages/starlette/routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "/home/seekndestroy/mambaforge/envs/ultralytics/lib/python3.10/site-packages/starlette/routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "/home/seekndestroy/mambaforge/envs/ultralytics/lib/python3.10/site-packages/starlette/routing.py", line 66, in app
    response = await func(request)
  File "/home/seekndestroy/mambaforge/envs/ultralytics/lib/python3.10/site-packages/fastapi/routing.py", line 273, in app
    raw_response = await run_endpoint_function(
  File "/home/seekndestroy/mambaforge/envs/ultralytics/lib/python3.10/site-packages/fastapi/routing.py", line 190, in run_endpoint_function
    return await dependant.call(**values)
  File "/home/seekndestroy/mambaforge/envs/ultralytics/lib/python3.10/site-packages/autollm/auto/fastapi_app.py", line 105, in query
    response = query_engine.query(user_query)
  File "/home/seekndestroy/mambaforge/envs/ultralytics/lib/python3.10/site-packages/llama_index/core/base_query_engine.py", line 30, in query
    return self._query(str_or_query_bundle)
  File "/home/seekndestroy/mambaforge/envs/ultralytics/lib/python3.10/site-packages/llama_index/query_engine/retriever_query_engine.py", line 170, in _query
    nodes = self.retrieve(query_bundle)
  File "/home/seekndestroy/mambaforge/envs/ultralytics/lib/python3.10/site-packages/llama_index/query_engine/retriever_query_engine.py", line 126, in retrieve
    nodes = self._retriever.retrieve(query_bundle)
  File "/home/seekndestroy/mambaforge/envs/ultralytics/lib/python3.10/site-packages/llama_index/core/base_retriever.py", line 54, in retrieve
    nodes = self._retrieve(query_bundle)
  File "/home/seekndestroy/mambaforge/envs/ultralytics/lib/python3.10/site-packages/llama_index/indices/vector_store/retrievers/retriever.py", line 88, in _retrieve
    return self._get_nodes_with_embeddings(query_bundle)
  File "/home/seekndestroy/mambaforge/envs/ultralytics/lib/python3.10/site-packages/llama_index/indices/vector_store/retrievers/retriever.py", line 164, in _get_nodes_with_embeddings
    query_result = self._vector_store.query(query, **self._kwargs)
  File "/home/seekndestroy/mambaforge/envs/ultralytics/lib/python3.10/site-packages/llama_index/vector_stores/lancedb.py", line 167, in query
    results = lance_query.to_df()
  File "/home/seekndestroy/mambaforge/envs/ultralytics/lib/python3.10/site-packages/deprecation.py", line 260, in _inner
    return function(*args, **kwargs)
  File "/home/seekndestroy/mambaforge/envs/ultralytics/lib/python3.10/site-packages/lancedb/query.py", line 186, in to_df
    return self.to_pandas()
  File "/home/seekndestroy/mambaforge/envs/ultralytics/lib/python3.10/site-packages/lancedb/query.py", line 195, in to_pandas
    return self.to_arrow().to_pandas()
  File "/home/seekndestroy/mambaforge/envs/ultralytics/lib/python3.10/site-packages/lancedb/query.py", line 409, in to_arrow
    return self._table._execute_query(query)
  File "/home/seekndestroy/mambaforge/envs/ultralytics/lib/python3.10/site-packages/lancedb/remote/table.py", line 165, in _execute_query
    return self._conn._loop.run_until_complete(result).to_arrow()
  File "/home/seekndestroy/mambaforge/envs/ultralytics/lib/python3.10/asyncio/base_events.py", line 625, in run_until_complete
    self._check_running()
  File "/home/seekndestroy/mambaforge/envs/ultralytics/lib/python3.10/asyncio/base_events.py", line 584, in _check_running
    raise RuntimeError('This event loop is already running')
RuntimeError: This event loop is already running
Expected Behavior
The application should query the LanceDB cloud database without errors, similar to how it operates with the local database setup.
Environment

LanceDB version: 0.3.4
FastAPI version: 0.104.1
Llama-Index version: 0.9.10

Additional Context

The application functions correctly with a local LanceDB instance.
The issue seems to be related to asynchronous operations in the FastAPI environment.

We are seeking guidance or a fix for this issue, as it is critical for our integration of LanceDB's cloud version in a FastAPI environment. Any insights or suggestions would be greatly appreciated.
Thank you for your attention to this matter.

Duplicate issue #667: Add support for `uvicorn` web server use of the `uvloop` event handler
Created: 2023-11-29T20:12:14Z
Problem
RemoteDBConnection: Event Handler mismatch with uvicorn leads to RuntimeError: this event loop is already running.
Context
RemoteDBConnection._loop is set toasyncio.get_running_loop() here. And their docs, say: "Raise a RuntimeError if there is no running event loop.".
When running a uvicorn web server, it's event loop implementation has three options [auto| asyncio| uvloop] and it defaults to auto which has a preference toward uvloop, and the lancedb DBConnection loop event handler is only asyncio which is likely causing a mismatch.
So when you run with the default event loop option, for example uvicorn app:app --reload. You will get this stack trace:
  File "/Users/amanmibra/Desktop/dt/deeptruth/src/api/utils/ss.py", line 3, in lancedb_predict
    pred = table.search(embedding).limit(k).to_pandas()
  File "/Users/amanmibra/anaconda3/envs/audio/lib/python3.10/site-packages/lancedb/query.py", line 195, in to_pandas
    return self.to_arrow().to_pandas()
  File "/Users/amanmibra/anaconda3/envs/audio/lib/python3.10/site-packages/lancedb/query.py", line 409, in to_arrow
    return self._table._execute_query(query)
  File "/Users/amanmibra/anaconda3/envs/audio/lib/python3.10/site-packages/lancedb/remote/table.py", line 165, in _execute_query
    return self._conn._loop.run_until_complete(result).to_arrow()
  File "uvloop/loop.pyx", line 1511, in uvloop.loop.Loop.run_until_complete
  File "uvloop/loop.pyx", line 1504, in uvloop.loop.Loop.run_until_complete
  File "uvloop/loop.pyx", line 1377, in uvloop.loop.Loop.run_forever
  File "uvloop/loop.pyx", line 518, in uvloop.loop.Loop._run
RuntimeError: this event loop is already running.
For the time being, setting the event handler to asyncio explicitly resolves the issue uvicorn app:app --reload --loop asyncio, but it would likely be useful to add support for uvloop, especially since uvicorn is a popular web server.

Closing comment (2023-12-12T00:07:34Z): This duplicates #680