Spaces:
Paused
Avoid Timeouts While Using BERT
Dear Lennard,
I really like this new dataset, initiative, and also the spaces seems interesting too. In my experience using BERT the latency is so large (even on A10G-Large) that the build times out after 30 minutes, but returns can take more than 4.5 hours for just 3-4 examples... so the best (unfortunately) is to not cache any examples for the build.
Hope this helps + keep up the good work !
Thank you for the advice! I have stopped the space until I have time to address this.
I'm currently working on something else around ExplainableAI and LlaMa2/Mistral and might just exchange the model in the classification. I originally intended it only as a comparison to the random forest I built, but that one, funny enough, also times out on me in the Spaces...
right... so during the build there's a timeout of 30 minutes (nothing you can do to change that, believe me i tried many times) , so if your response times are over 30 minutes - which are most finetunes of BERTs and Llamas or any model that's quite large (more than 3B/7B params) will typically take 30 mins + to repond and basically that times out the space during build if you have like 1 or 2 examples , even.
I'm looking forward to what you'll come up with, so i'll follow along ! hope that's chill ^^ 🚀