castorini

/

rank_zephyr_7b_v1_full

like 8

I used llama_cpp to load the 4-bit gguf version of the model, I used the prompt mentioned on the paper, but I get an empty string as a response, but it seems to work on reranking when I only retrieve less than 5 passages, I guess there is a context window size problem, how to implement the sliding window approach?
I also noticed that the inference time is so long (>1s)