Repository: Marker-Inc-Korea/AutoRAG
Issue: How can I add custom embedding model when running web
Root cause: Architectural Constraint

I have custom huggingface embedding model in my pipeline.yaml.  
When running as api server, I can add my custom model like below
```python
autorag.embedding_models["custom model"] = LazyInit(
    HuggingFaceEmbedding, model_name="custom_model"
)
runner = Runner.from_yaml("./6/pipeline.yaml")
runner.run_api_server()
```
But run_web support cli only, so I can't add custom model like above.  
How can I add custom model?
Is there any reason why `run_web` is not supported by the Python SDK, like `run_api_server()`?

@bwook00 You might can help this issue.

@rjwharry Great point! I think there is no docs for this situation. 

First, we might do experiment and leave a comment here again. Thanks. 
(We missed this situation)

If you're willing to work on this issue once you've decided, can I contribute?  
Seems like a good fit for a “good first issue”.

@rjwharry That will be lovely. 
I will assign you to assignee.

There is a streamlit file at `autorag/web.py` 
At `cli.py` we execute the streamlit with `streamlit run web.py` command. 
Because the only thing to execute streamlit was cli, we made just cli method.
You might find a way to start streamlit without using cli. 

I have some suggestions and requests
### Request for style guide
If you have any documentation for the style guide or the '.pylintrc' file, please let me know. My preferences are quite different from yours.
### Suggestions
I'm working on this issue and I noticed that there is a limitation in implementing it with streamlit. Even if I set up a new custom LLM or embedding model, the streamlit application will not recognize the custom model because the streamlit application has to run in a different process.  
So here is my suggestion. Why don't you use [Gradio](https://github.com/gradio-app/gradio) instead? I think it is a good alternative to streamlit and I can run it in the same process, so Gradio knows my custom model.  
I have already tested it on my local and it works fine, so I hope you will consider this. Here's a screenshot of my test
<img width="1467" alt="image" src="https://github.com/user-attachments/assets/642bc32b-c6e0-49ed-a27f-1247075e2a5d">


In the process of changing to gradio, I ran into an issue with event loops. When I run gradio and run the pipeline, all of the `loop.get_event_loop` code throws an error saying: `There is no current event loop in thread 'AnyIO worker thread'`.  
I thought the cause of this error was that there was no event loop running, as shown in the logs, so I changed that code to `asyncio.run()` so that it can be automatically generated when the event loop is not present and now I don't get the error. 
Consequently I changed all modules like below.
```python
# original
loop.get_event_loop()
results = loop.run_until_complete(process_batch(tasks, batch_size=batch))

# new
results = asyncio.run(process_batch(tasks, batch_size=batch))
```
I'm testing to make sure this fix works. I'd appreciate it if you could share your ideas.

> I have some suggestions and requests
> 
> ### Request for style guide
> If you have any documentation for the style guide or the '.pylintrc' file, please let me know. My preferences are quite different from yours.
> 
> ### Suggestions
> I'm working on this issue and I noticed that there is a limitation in implementing it with streamlit. Even if I set up a new custom LLM or embedding model, the streamlit application will not recognize the custom model because the streamlit application has to run in a different process. So here is my suggestion. Why don't you use [Gradio](https://github.com/gradio-app/gradio) instead? I think it is a good alternative to streamlit and I can run it in the same process, so Gradio knows my custom model. I have already tested it on my local and it works fine, so I hope you will consider this. Here's a screenshot of my test <img alt="image" width="1467" src="https://private-user-images.githubusercontent.com/104843295/358842513-642bc32b-c6e0-49ed-a27f-1247075e2a5d.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQwMzI5ODIsIm5iZiI6MTcyNDAzMjY4MiwicGF0aCI6Ii8xMDQ4NDMyOTUvMzU4ODQyNTEzLTY0MmJjMzJiLWM2ZTAtNDllZC1hMjdmLTEyNDcwNzVlMmE1ZC5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwODE5JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDgxOVQwMTU4MDJaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT03MDYyNDU2NGZjY2Y0YWRiNmYxZTgyNzc3ZDdmY2M0NjRmZjliODUxZjg3ZGVlZDg5ZDc2ZjcyZjNhMzZhZGIxJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.yc2N8ExEfbmz6rUx9umhx5yp3w6H1yAmjdAXxW199A0">

1. I made issue for that #621 Thanks for the suggestion.
2. Gradio will be great. Just please keep in mind users can select between two of them and it is not becoming too much complex. 

@rjwharry Maybe gradio is using `anyio`, which is not `asyncio`. So the library itself have some conflict at asyncio event loop.

But yeah, it looks like when you changed all `asyncio.get_event_loop()` to `asyncio.run`, it looks like there is no error from now. 
So it will be great you can fix this.

@rjwharry Can you make a draft PR to our repo? It will be great to see your code 👍 

@vkehfdl1 During test, I got same error(`There is no current event loop in thread 'AnyIO worker thread'`) for specific case. So let me research more about this, and then I'll share you with draft PR as soon as possible

@rjwharry Okay thanks. I hope we can find some way that gradio can use `asyncio` not `anyio`

Plus, at #622, I integrate ruff Linter and reformat all codes. Maybe the code style will be consistent. 

It might be have conflict when merging between two because of all reformat.. 
So be careful... 

I think it is better you get ruff setting and reformat first before merge this. The setting is in `pyproject.toml` 
And use below code.

`ruff check --fix`
`ruff format`