State Space Models are Strong Text Rerankers (2025)

Abstract: Transformers dominate NLP and IR; but their inference inefficiencies and challenges in extrapolating to longer contexts have sparked interest in alternative model architectures. Among these, state space models (SSMs) like Mamba offer promising advantages, particularly time complexity in inference. Despite their potential, SSMs’ effectiveness at text reranking — a task requiring fine-grained query-document interaction and long-context understanding — remains underexplored.This study benchmarks SSM-based architectures (specifically, Mamba-1 and Mamba-2) against transformer-based models across various scales, architectures, and pre-training objectives, focusing on performance and efficiency in text reranking tasks. We find that (1) Mamba architectures achieve competitive text ranking performance, comparable to transformer-based models of similar size; (2) they are less efficient in training and inference compared to transformers with flash attention; and (3) Mamba-2 outperforms Mamba-1 in both performance and efficiency. These results underscore the potential of state space models as a transformer alternative and highlight areas for improvement in future IR applications.

## introduction the transformer architecture @xcite is an established standard within nlp and ir community. compared to recurrent neural networks (rnns) transformers better capture long-range dependencies and also admit large scale pre-training. however, for inference with a sequence of length l and d-dimensional hidden states, transformers cost o(l) time and o(ld) space complexityproving to be less efficient than rnns. recently, there has been a growing interest in developing alternative architectures for modeling sequence data. for example, rwkv @xcite combines the efficient parallelizable training of transformers with the efficient inference of rnns. another notable architecture is the state space model (ssm, @xcite @xcite , which is related to convolutional and recurrent neural networks, and also to signal processing literature. in essence, state space models compress the context into a smaller state of size n , achieving o(1) time complexity and o(n d) space complexity in inference time. however, the capabilities of ssms are limited by the amount of information that can be compressed in its hidden state. to mitigate this, @xcite propose a novel selective state space model named mamba. mamba selectively encodes the input to the hidden state to improve model expressiveness, while also addressing the computation problem with a selective scan method and hardware-aware optimization. @xcite and followup work @xcite @xcite examine the efficacy of mamba models for various sequence modeling tasks, notably language modeling, and also image and audio tasks. the parameterized ssms are able to achieve performance close to transformer-based models of similar sizes while also demonstrating efficiency in training and inference. despite the growing popularity of state space models, their effectiveness in information retrieval remains underexplored. modern search systems typically consist of at least two stages: retrieval and reranking. during retrieval, offline indexes first fetch a preliminary list of candidate documents, which is refined by the reranking model. reranking requires models to understand long context input, and to capture fine-grained query-document interactions. the attention mechanism in the transformer naturally allows for the latter; it allows query tokens to attend to document tokens. in contrast, state space models may fail to model long-range dependencies due to their recurrent nature. in this paper, we examine the following research questions about mamba-1 and mamba-2: to this end, we conduct a rigorous benchmarking study comparing the two model families, across varying architectures, sizes, pre-training objectives, and attention patterns. specifically, we train neural reranking models following established training methodologies outlined in prior literature @xcite @xcite . our experiments allow us to address the two research questions above. we find that: we discuss the implications of our results and point out future directions of transformer alternative architectures for ir tasks. ## background: state space models we will briefly survey state space models and their connection to rnns and transformers. we use structured state space sequence models (s4, @xcite to illustrate the idea behind state space models before describing the mamba models. ## experiments in this section, we describe the experimental setup for passage ( § 4.1) and document reranking ( § 4.3). for performance rq, we report results and analyze the implications in § 4.2 and § 4.4 respectively. then we address efficiency rq in § 4.5. ## conclusion and future work this study investigates the suitability of mamba architectures, a novel class of state space models, for text ranking. our findings demonstrate that mamba models, particularly mamba-2, can achieve competitive performance compared to transformer-based models of comparable size, showcasing their potential as viable alternatives for sequence modeling in ir tasks. while mamba architectures currently exhibit lower training and inference efficiency compared to transformers with flash attention, continuous advancements in model optimization and hardware acceleration have the potential to mitigate these limitations. we picture two future directions of this work: the task direction and the model direction. from the task perspective, the efficacy of state space models, including mamba should be further examined in other ir tasks (e.g., text retrieval). from the model perspective, hybrid models @xcite @xcite have shown promise in certain nlp tasks. we believe the effectiveness of hybrid models should be thoroughly tested. additionally, optimization for state space models is an interesting challenge that may offer substantial improvements.