Repository: Marker-Inc-Korea/AutoRAG
Issue: [Hotfix] Window OS error
Root cause: Environment Incompatibility

I had run python -m pytest in terminal

and I got this Error messege


~~~
======================================================================================================= short test summary info =======================================================================================================
FAILED tests/autorag/test_evaluator.py::test_start_trial - OSError: [Errno 22] Invalid argument: 'C:\\Users\\hanpa\\PycharmProjects\\AutoRAG\\0\\retrieve_node_line\\retrieval\\bm25=>top_k_50.parquet'
FAILED tests/autorag/nodes/retrieval/test_run_retrieval_node.py::test_run_retrieval_node - OSError: [Errno 22] Invalid argument: 'C:\\Users\\hanpa\\PycharmProjects\\AutoRAG\\tests\\resources\\test_project\\test_trial\\test_node_line\\retrieval\\bm25=>top_k_4.parquet'
============================================================================================= 2 failed, 26 passed, 23 warnings in 24.79s ==============================================================================================
(venv) PS C:\Users\hanpa\PycharmProjects\AutoRAG>


@Eastsidegunn Is it fix at the latest version of v0.0.11?
If not, how can I reproduce this bug?

@vkehfdl1 sad news...

```
===================================================================================================== short test summary info ===================================================================================================== 
FAILED tests/autorag/evaluate/metric/test_generation_metric.py::test_g_eval_full - assert False
FAILED tests/autorag/test_cli.py::test_evaluator_cli - FileNotFoundError: [WinError 2] 지정된 파일을 찾을 수 없습니다
FAILED tests/autorag/test_deploy.py::test_extract_best_config - PermissionError: [Errno 13] Permission denied: 'C:\\Users\\hanpa\\AppData\\Local\\Temp\\tmpftp1ik4gyaml'
FAILED tests/autorag/test_deploy.py::test_runner - PermissionError: [Errno 13] Permission denied: 'C:\\Users\\hanpa\\AppData\\Local\\Temp\\tmpmjk_8_5byaml'
FAILED tests/autorag/test_evaluator.py::test_test_data_evaluate - PermissionError: [Errno 13] Permission denied: 'C:\\Users\\hanpa\\AppData\\Local\\Temp\\tmppqaa6pai.yaml'
ERROR tests/autorag/nodes/queryexpansion/test_hyde.py::test_hyde_node - NotADirectoryError: [WinError 267] 디렉터리 이름이 올바르지 않습니다: 'C:\\Users\\hanpa\\AppData\\Local\\Temp\\tmpsiqsdf0t\\257c8160-6676-4427-bf62-4a0876e03b36\\data_level0.bin'
ERROR tests/autorag/nodes/queryexpansion/test_query_decompose.py::test_query_decompose_node - NotADirectoryError: [WinError 267] 디렉터리 이름이 올바르지 않습니다: 'C:\\Users\\hanpa\\AppData\\Local\\Temp\\tmprhkmpu46\\a6470dbc-9a65-4db1-b1f1-de9ad83ced7b\\data_level0.bin'
ERROR tests/autorag/nodes/retrieval/test_bm25.py::test_bm25_ingest - PermissionError: [Errno 13] Permission denied: 'C:\\Users\\hanpa\\AppData\\Local\\Temp\\tmpbfvrs4vn.pkl'
ERROR tests/autorag/nodes/retrieval/test_hybrid_cc.py::test_hybrid_cc_node - NotADirectoryError: [WinError 267] 디렉터리 이름이 올바르지 않습니다: 'C:\\Users\\hanpa\\AppData\\Local\\Temp\\tmpra6sxrlz\\resources\\chroma\\03137de3-e224-45f2-b04c-c480db64d825\\data_level0.bin'
ERROR tests/autorag/nodes/retrieval/test_hybrid_rrf.py::test_hybrid_rrf_node - NotADirectoryError: [WinError 267] 디렉터리 이름이 올바르지 않습니다: 'C:\\Users\\hanpa\\AppData\\Local\\Temp\\tmp2fvoaaqk\\resources\\chroma\\29b86527-8ec7-4828-8ce5-803ed4574130\\data_level0.bin'
ERROR tests/autorag/nodes/retrieval/test_hybrid_rrf.py::test_hybrid_rrf_node_deploy - NotADirectoryError: [WinError 267] 디렉터리 이름이 올바르지 않습니다: 'C:\\Users\\hanpa\\AppData\\Local\\Temp\\tmp0iqns3wg\\resources\\chroma\\a04a6abd-80cf-4814-9a18-d8b42eff5ce9\\data_level0.bin'
ERROR tests/autorag/nodes/retrieval/test_run_retrieval_node.py::test_run_retrieval_node - NotADirectoryError: [WinError 267] 디렉터리 이름이 올바르지 않습니다: 'C:\\Users\\hanpa\\AppData\\Local\\Temp\\tmpghaaoznm\\resources\\chroma\\5647983b-6dce-4912-b164-682b48f8c996\\data_level0.bin'
ERROR tests/autorag/nodes/retrieval/test_run_retrieval_node.py::test_run_retrieval_node_only_hybrid - NotADirectoryError: [WinError 267] 디렉터리 이름이 올바르지 않습니다: 'C:\\Users\\hanpa\\AppData\\Local\\Temp\\tmpi7qh9_ml\\resources\\chroma\\0765bde5-d10b-4232-9bc8-e284cef2becf\\data_level0.bin'
ERROR tests/autorag/nodes/retrieval/test_vectordb.py::test_vectordb_retrieval - NotADirectoryError: [WinError 267] 디렉터리 이름이 올바르지 않습니다: 'C:\\Users\\hanpa\\AppData\\Local\\Temp\\tmpjivb_wek\\3284d59d-7f8f-4895-84ae-2b533cf1a57d\\data_level0.bin'
ERROR tests/autorag/nodes/retrieval/test_vectordb.py::test_vectordb_node - NotADirectoryError: [WinError 267] 디렉터리 이름이 올바르지 않습니다: 'C:\\Users\\hanpa\\AppData\\Local\\Temp\\tmp4jwxb4s2\\resources\\chroma\\chroma.sqlite3'
ERROR tests/autorag/nodes/retrieval/test_vectordb.py::test_duplicate_id_ingest - NotADirectoryError: [WinError 267] 디렉터리 이름이 올바르지 않습니다: 'C:\\Users\\hanpa\\AppData\\Local\\Temp\\tmpwwljvl_f\\056c0965-b00e-4918-ab06-20d46cd898d8\\data_level0.bin'
ERROR tests/autorag/test_deploy.py::test_runner - NotADirectoryError: [WinError 267] 디렉터리 이름이 올바르지 않습니다: 'C:\\Users\\hanpa\\AppData\\Local\\Temp\\tmp_xbim295\\resources\\chroma\\chroma.sqlite3'
ERROR tests/autorag/test_deploy.py::test_runner_api_server - NotADirectoryError: [WinError 267] 디렉터리 이름이 올바르지 않습니다: 'C:\\Users\\hanpa\\AppData\\Local\\Temp\\tmpsy3nr5sv\\resources\\chroma\\6306640b-77dc-4663-8071-c30715eb30dd\\data_level0.bin'
ERROR tests/autorag/test_evaluator.py::test_start_trial - NotADirectoryError: [WinError 267] 디렉터리 이름이 올바르지 않습니다: 'C:\\Users\\hanpa\\AppData\\Local\\Temp\\tmpz9a43dk6\\resources\\chroma\\chroma.sqlite3'
========================================================================= 5 failed, 109 passed, 4 skipped, 178 warnings, 14 errors in 1628.99s (0:27:08) ========================================================================== 
(venv)  ...
```

I looked all error on the windows os. 
**I won’t fix this issue**

1. `test_g_eval_full` failed is sometimes happend, because it depends LLM output.
2. `FileNotFoundError` is occured because `tempfile.NamedTemporaryFile()` in Windows.
https://docs.python.org/ko/3/library/tempfile.html
```
On Windows, make sure that at least one of the following conditions are fulfilled:

- delete is false
```
In python docs, it says there is **no way** to use NamedTemporaryFile in Windows withe delete option is True.
If we will not delete tempfile, why do we use this in test? 
This is just **Windows** issue at only testing, and do not need to fix our whole test code. 
*Use Linux or Buy Mac*
3. `NotaDirectoryError` also occured by tempfile. This time `tempdir` is the problem.
When `tempfile` try to delete chroma persistent client, it occurs `ProcessError`
I searched about it, and I couldn’t find solution to delete chroma on the windows process.
No way to fix it, even we do not use tempfile. 
Because there is **no way** to delete chroma persistent client folder in the python code. 
*Use Linux or Buy Mac*

### Conclusion
We are **not support** testing on Windows.
For developing and testing `AutoRAG`, use Linux or buy Mac.

I am assigning this issue as ‘wontfix'

Closing this issue since we will not fix this issue.