Is it possible to accelerate the work of "Vector Storage"?
I uploaded a 66.6 MB PDF file, the preprocessing was fast, literally in a couple of seconds, if not faster. After preprocessing, the TXT weighs ~4MB. After that, vectorization began... It lasts more than 60 minutes. I didn't get it...
Vectorization Settings:
There is a feeling that vectorization does not take up the CPU in full. Or transfer it to a video card.
I'm doing vectorization with a running model that takes away 16VRAM from me...
Is it possible to parallelize this task somehow so that it runs faster?
If I may, I will leave a link to a project where vectorization is faster, perhaps this is not the case at all since I do not understand much, but I will leave it anyway:
https://github.com/BBC-Esq/VectorDB-Plugin-for-LM-Studio
My setup:
• AMD Ryzen 7 7800X3D
• DDR5 32GB 6400
• RTX 4080
Local (transformers.js) option is not accelerated with WebGPU/CUDA since it runs using in Node runtime for ONNX in a single thread.
Multithreaded WASM option is disabled due to causing problems for some people on setups like Linux/Android.
An alternative would be to move the vectorization to a browser side (which reportedly supports hardware acceleration), though I have some doubts about it.