Before revision:  Building on these insights, we propose a simple neural model that combines the efficiency of dual encoders with some of the expressiveness of  attentional architectures, and explore a sparse-dense hybrid to capitalize on the precision of sparse retrieval.
After revision:  Building on these insights, we propose a simple neural model that combines the efficiency of dual encoders with some of the expressiveness of attentional architectures, and explore sparse-dense hybrid to capitalize on the precision of sparse retrieval.