Spaces:

PaddlePaddle

/

UIE-X

like 72

Runtime error

Hi HF folks,

PaddleNLP is releasing a new model + space called UIE-X, which is a multilingual entity extraction + relation extraction model that works for both text and scanned documents. It has pretty decent zero-shot capacity and would make a good space. We are seeking some advice regarding the HF space setup.

• We prefer to have the model stored on HF Model Hub, have a clean one-line loading similar to the Inference API integration and performs the computation in HF Space container. However, due to the multi-stage inference setup and the OCR preprocessor, the latency is just too high (can be more than 100s ) with only CPU inference. Here is a CPU version that you can play with. https://huggingface.co/spaces/linjieccc/uie-test (not loading from HF Hub yet but could be loading from it soon)
• The alternative is do what we did for Ernie-ViLG and Ernie-Layout, which is to run on GPU on-premise in our own DC in China and interact with HF Space through a Rest API. This is much faster than CPU (~5s), but can still be slow when the uploaded file is large. However, we prefer not to use Rest API since it’s opaque and we prefer to be as open as possible. Here is the GPU on-premise version. https://huggingface.co/spaces/PaddlePaddle/UIE-X

Would it be possible for us to go through the community GPU grant, considering our model and solution is completely open-sourced? My guess is that we’d only need this for a while (a few weeks at most) when there is traffic and public interest. (edited)