QAEncoder: Towards Aligned Representation Learning in Question Answering System

## Abstract

Abstract. Modern QA systems entail retrieval-augmented generation (RAG) for accurate and trustworthy responses. However, the inherent gap between user queries and relevant documents hinders precise matching. Motivated by our conical distribution hypothesis, which posits that potential queries and documents form a cone-like structure in the embedding space, we introduce QAEncoder, a training-free approach to bridge this gap. Specifically, QAEncoder estimates the expectation of potential queries in the embedding space as a robust surrogate for the document embedding, and attaches document fingerprints to effectively distinguish these embeddings. Extensive experiments on fourteen embedding models across six languages and eight datasets validate QAEncoder’s alignment capability, which offers a plug-and-play solution that seamlessly integrates with existing RAG architectures and training-based methods.
The code is available at https://github.com/IAAR-Shanghai/QAEncoder .

## 1. Introduction

“What I cannot create, I do not understand.”

— Richard Feynman

Question Answering (QA) systems aim to generate accurate responses to user queries with applications in customer service (Xu et al., ), search engine (Ojokoh and Adebisi, ), healthcare (Guo et al., ) and education (Levonian et al., ), necessitating proficiencies in information retrieval, comprehension, and generation.
Modern QA systems leverage large language models (LLMs) such as ChatGPT (Achiam et al., ), supplemented with retrieval-augmented generation (RAG) to address issues of outdated or hallucinatory information, especially for rapidly evolving knowledge bases (Lewis et al., ; Huang et al., ; Gupta et al., ). The efficacy of RAG hinges on its retrieval module for identifying relevant documents from a vast corpus. Dense retrievers (Lewis et al., ; Hofstätter et al., ), contrasted with keyword-matching-based sparse retrievers (Jones, ; Robertson and Zaragoza, ), have enabled efficient and precise retrieval by mapping queries and documents into a shared vector space.
Despite advancements, a significant challenge that persists is bridging the semantic and syntactic gap between user queries and documents, known as the document-query gap (Zheng et al., ). Three main approaches have emerged to address this challenge:
training-based alignment, document-centric alignment, and query-centric alignment.

Training-based approaches (Dong et al., ; Li et al., ; W et al., ; Zhang et al., ; Khanna and Subedi, ) fine-tune or train embedding models from scratch with dedicated QA datasets to close the representation of relevant queries and documents, but struggle to generalize across new domains (Suprem and Pu, ). Furthermore, catastrophic forgetting necessitates updating embeddings for the previously encoded corpus when new data is learned (Pan et al., ), which is resource-intensive and practically unaffordable.
Document-centric approaches (Wang et al., ; Gao et al., ; Kim and Min, ) generate pseudo-documents for user queries by LLMs, which are then used to retrieve relevant information from the corpus, bridging the document-query gap by LLMs’ zero-shot generalization capabilities.
However, pseudo-document generation is costy and time-consuming, and may include fabricated content (Wang et al., ), increasing the risk of hallucinated information.

Query-centric methods  (Nogueira et al., ; Cheriton, ; Mallia et al., ) generates and indexes document-relevant questions to align the indexed context with user queries in a training-free manner. However, existing solutions are predominantly based on sparse retrievers and have not fully leveraged the potential of dense retrievers. Combining query-centric methods with dense retrievers remains heavily under-explored.
The naive solution can be storing all QA pairs into a vector database, but with several notable challenges:

- C1.
Expanded Index Size.
Storing all QA pairs significantly increase the index size, leading to substantial expansion in storage requirements, especially problematic for large-scale corpora.
- C2.
Prolonged Retrieval Times.
The index expansion also results in extended retrieval times. For dense retrievers, the expanded index size can result in linearly increased search time in both exhaustive and non-exhaustive search (Douze et al., ) and hurt the recall performance in non-exhaustive case (Zhao et al., ).
- C3.
Limited Query Handling.
Although storing QA pairs individually can address predicted queries, this approach lacks robustness when confronted with the wide-ranging and diverse nature of potential queries. Such a method may be unable to effectively handle rephrasing in linguistic style, sentence structure, or even vocabulary, thereby compromising the overall reliability and performance of the system (Alting von Geusau and Bloem, ).

##### Motivation

Inspired by Feynman’s philosophy of learning (Reyes et al., ), we believe that effective information retrieval for QA systems extends beyond mere information storage and fundamentally depends on the active processing, involving the query formulation. This process mirrors human learning, where a deeper understanding of stored knowledge is achieved through thoughtful reflection and inquiry. For instance, the well-established 5W1H framework (Who, What, When, Where, Why, How) (Jinks, ) can be employed to systematically deconstruct information, fostering a comprehensive and nuanced understanding.
In this paper, we continue the research line of query-centric methods and introduce an innovative approach called QAEncoder, motivated by a key observation, termed conical distribution hypothesis. Specifically, for a given document, its potential queries are embedded approximately within a single cluster on some hyperplane in the semantic space, while the document embedding lie on the perpendicular line intersecting the cluster center. Hence, the projection of the document embedding onto this hyperplane, i.e. the cluster center, is significantly closer to the potential queries, and can be utilized to optimize the original document embedding.

As demonstrated in Fig. 1, our method initially generates diversified queries (e.g. 5W1H) and then estimates the cluster center using Monte Carlo methods, i.e. mean pooling. The similarity matrix reveals that, compared to both document-query and query-query similarities, the mean-query similarity is significantly higher than both document-query and query-query similarities. Hence, we advocate using the cluster center as a surrogate for the document embedding in QA systems, which bridges the document-query gap robustly without extra index size and retrieval latency.

Despite these advantages, the basic implementation encounters a critical challenge. While enhancing similarity with user queries, it simultaneously reduces the distinguishability between document representations, as they all become query-like. To address this side effect, we further introduce document fingerprint strategies to infuse unique identities into document representations and enable state-of-the-art performance.

##### Contributions

The main contributions can be summarized as follows:

- •
Methodological Innovations.
We pioneer to bridge the document-query gap in dense retrievers from the query-centric perspective.
Our method, QAEncoder, not only avoids extra index storage, retrieval latency, training cost and hallucination, but also guarantees diversified query handling and robust generalization. We further propose document fingerprint strategies to address the side effect of distinguishability and achieve state-of-the-art performance.
- •
Theoretical Discovery.
We formulate the conical distribution hypothesis, providing a theoretical foundation for the alignment of document and query embeddings, which is validated through extensive empirical analysis and provides deeper insights into semantic spaces in QA systems.
- •
Practical Applications.
QAEncoder is designed as a truly plug-and-play solution, seamlessly integrating with many existing retrieval and RAG architectures. This integration significantly enhances system performance with minimal modifications required.

Figure: Figure 1. Illustration of QAEncoder’s Semantic Alignment Process: Solid lines represent diversified query generation, while dashed lines indicate Monte Carlo estimation with embedding mean pooling. The heatmap depicts the similarity scores among the embeddings of the different queries, the document, and the mean estimation.
Refer to caption: x1.png

## 2. Related Works

##### Retrieval-Augmented QA systems.

Retrieval-Augmented Generation (RAG) significantly improves large language models (LLMs) in question-answering (QA) systems by incorporating a retrieval module that fetches relevant information from external knowledge sources (Févry et al., ; Guu et al., ; Izacard and Grave, ; Zhao et al., ). Retrieval models have evolved from early sparse retrievers, such as TF-IDF (Jones, ) and BM25 (Robertson and Zaragoza, ), which rely on word statistics and inverted indices, to dense retrieval strategies (Lewis et al., ) that utilize neural representations for enhanced semantic matching . Advanced methods, such as Self-RAG (Asai et al., ) which determines if additional information is required and evaluates the relevance of retrieved content, and RAG-end2end (Siriwardhana et al., ) that jointly trains the retriever and generator, represent significant developments in this area. However, these methods still ignore the inherent semantic gap between queries and documents.

##### Training-based alignment.

Training-based approaches bridge the document-query gap generally by contrastive learning  (Xiong et al., ; Qu et al., ) or knowledge distillation  (Zhang et al., ; Khanna and Subedi, ). For instance,  (Dong et al., ) show parameter sharing of the query encoder and the document encoder improves overall performance by projecting queries and documents into shared space. Dual-Cross-Encoder (Li et al., ) and Query-as-context (W et al., ) train embedding models from scratch with paired document-query samples. E5 (Wang et al., ) and GTE (Zhang et al., ) concatenate different prompts before queries for instruction-tuned embedding models that enhance downstream task adaptability, especially for QA systems.
However, training-based methods face high costs, generalization difficulties (Suprem and Pu, ), and catastrophic forgetting issue (Pan et al., ). Catastrophic forgetting, where the encoder continually trained on new data lose previously function, is particularly detrimental as it necessitates the costly re-encoding of entire corpora.

##### Document-centric alignment.

Document-centric methods, such as HyDE (Gao et al., ) and Query2doc (Wang et al., ), dynamically transform user queries into pseudo-documents using large language models for both sparse and dense retrievers. QA-RAG (Kim and Min, ) advances by implementing a two-way retrieval mechanism that utilizes both user query and pseudo-documents for respective retrieval. However, their effectiveness is highly dependent on the quality of pseudo-documents generated, which are susceptible to hallucinations, especially for latest information. Furthermore, invoking LLMs for each query imposes substantial computational costs and increases latency, leading to degraded user experience.

##### Query-centric alignment.

The seminal work, Doc2Query (Nogueira et al., ), mainly focus on the vocabulary mismatch problem for sparse retrievers by expanding the document with keywords in predicted queries. The subsequent DocT5Query (Cheriton, ) improves by training a T5 model to predict queries. Based on DocT5Query, DeepImpact (Mallia et al., ) further assigns weights on keywords with neural estimation for more precise sparse representations. Though these approaches have succeed for sparse retrievers, the integration with dense retrieval systems remains largely unexplored.

Figure: Figure 2. QAEncoder Architecture. Left: Corpus documents are embedded using QAEncoder to obtain query-aligned representations for indexing. User queries are encoded with a vanilla encoder and used to retrieve relevant documents. Right: Internal mechanism of QAEncoder. QAEncoder addresses the document-query gap by generating a diverse set of queries for each document to create semantically aligned embeddings. Additionally, document fingerprint strategies are employed to ensure document distinguishability.
Refer to caption: extracted/5888105/images/pipeline_new.png

## 3. Method

### 3.1. Problem Formulation

Given a query $q$ and a document corpus $\mathcal{D}=\{d_{1},d_{2},...,d_{i},...,d_{N}\}$, our task is to retrieve a subset of $K$ most relevant documents $\mathcal{D}_{+}=\{d{i_{1}},d_{i_{2}},...,d_{i_{j}},...,d_{i_{K}}\}$ through vector search.
We define our embedding model as $\mathcal{E}(\cdot)$, which maps each document $d$ and query $q$ from the textual space $\mathcal{C}$ to a vector space $\mathbb{R}^{r}$.
The semantic relevance is quantified by the cosine similarity between the query $q$ and each document $d$ in the embedding space, defined as:

$$ (1) $sim(q,d)=\frac{\mathcal{E}(q)^{T}\mathcal{E}(d)}{\|\mathcal{E}(q)\|\|\mathcal{ E}(d)\|}.$ $$

Furthermore, for each document $d$ in our dataset, we invoke the query generator $\mathcal{Q}(\cdot)$ multiple times to generate $n$ predicted queries $\{q_{i}\}_{i=1}^{n}$, where the cluster center in embedding space is captured by $\mathbb{E}[\mathcal{E}(\mathcal{Q}(d))]$ and estimated by Monte Carlo sampling $\overline{\mathcal{E}(\mathcal{Q}(d))}$.

Figure: Figure 3. Conical Distribution Hypothesis Validation. The figure presents three visualizations supporting the conical distribution hypothesis : (a): t-SNE visualization of queries derived from various documents in the embedding space, illustrating distinct clustering behavior. (b): Angular distribution of document and query embeddings, showing the distribution of angles between $v_{d}=\mathcal{E}(d)-\mathbb{E}[\mathcal{E}(\mathcal{Q}(d))]$ and $v_{q_{i}}=\mathcal{E}(q_{i})-\mathbb{E}[\mathcal{E}(\mathcal{Q}(d))]$. The angles form a bell curve just below 90°, supporting that $v_{d}$ is nearly orthogonal to each $v_{q_{i}}$ and serves as the normal vector to a hyperplane $\mathcal{H}$. (c): 3D visualization illustrating the conical distribution of document (black point) and query (colored points) embeddings within a unit sphere. The star indicates the queries’ cluster center.
Refer to caption: x2.png

### 3.2. Conical Distribution Hypothesis

In this subsection, we formally define the highly simplified conical distribution hypothesis and validates its reasonableness with empirical analysis detailed in the appendix A.5.

###### Hypothesis 1 (Conical Distribution Hypothesis) .

For any document $d$, the potential queries approximately form a single cluster on some hyperplane $\mathcal{H}=\{x\in\mathbb{R}^{r}\mid w\cdot x=b\}$ in the semantic space, where $w\in\mathbb{R}^{r}$ is the normal vector and $b\in\mathbb{R}$ is the bias term. Furthermore, the document embedding $\mathcal{E}(d)$ lies on the perpendicular line intersecting the cluster center $\mathbb{E}[\mathcal{E}(\mathcal{Q}(d))]$. Formally, the relationship can be represented as:

$$ (2) $\mathcal{E}(d)\approx\mathbb{E}[\mathcal{E}(\mathcal{Q}(d))]+\lambda w,\quad \lambda\in\mathbb{R}.$ $$

Furthermore, when the stronger hypothesis assuming the clusters follow the Gaussian distribution is adopted, quantitative analysis results is derived in the appendix.

###### Hypothesis 1 (Conical Distribution Hypothesis) .

### 3.3. QAEncoder

Building upon the conical distribution hypothesis, we introduce a novel encoding strategy, $\text{QAE}_{\text{base}}$, which represents the document by the cluster center $\mathbb{E}[\mathcal{E}(\mathcal{Q}(d))]$ of potential queries, instead of the document embedding $\mathcal{E}(d)$ itself. Formally, the transformation is defined as follows:

$$ (3) $\text{QAE}_{\text{base}}(d)=\mathbb{E}[\mathcal{E}(\mathcal{Q}(d))]\approx \overline{\mathcal{E}(\mathcal{Q}(d))}=\frac{1}{n}\sum_{i=1}^{n}\mathcal{E}(q_ {i}).$ $$

However, the ideal embedding model should not only bring related entries closer in the embedding space but also separate unrelated entries as much as possible. Despite the $\text{QAE}_{\text{base}}$ model enhances document-query relevance compared to the original document embedding $\mathcal{E}(d)$, it poses the distinguishability issue.

#### 3.3.1. Document Fingerprint Strategy

The distinguishability issue arises because incorporating additional query semantics into document representations suppresses their unique characteristics and renders unrelated documents more similar. To address this issue, we further introduce the document fingerprint strategy, which involves three variations that enhance the uniqueness of document representations from different perspectives.

- •
Embedding fingerprint - $\text{{QAE}}_{\text{{emb}}}$.
The QAEemb strategy manipulates within the embedding space and reintroduces unique identity of the original document, i.e. the document embedding $\mathcal{E}(d)$. Specifically, $\text{QAE}_{emb}$ considers both the cluster center, $\text{QAE}_{\text{base}}$, and the document embedding, $\mathcal{E}(d)$, balancing their contributions using a hyperparameter $\alpha$. The adjusted embedding is formulated as follows:
(4)
$\text{QAE}_{\text{emb}}(d)=(1-\alpha)\cdot\mathcal{E}(d)+\alpha\cdot\text{QAE}
_{\text{base}}(d)\approx(1-\alpha)\cdot\mathcal{E}(d)+\alpha\cdot\frac{1}{n}
\sum_{i=1}^{n}\mathcal{E}(q_{i}).$
Due to the linearity of the inner product space, the cosine similarity, i.e. the inner product between the user query $q$ and $\text{QAE}_{\text{emb}}(d)$, is mathematically equivalent to the weighted sum of the similarities between the user query $q$ and both $\mathcal{E}(d)$ and $\text{QAE}_{\text{base}}(d)$, controlled by hyperparameter $\alpha$.
- •
Textual fingerprint - $\text{{QAE}}_{\text{{txt}}}$.
The $\text{QAE}_{\text{txt}}$ strategy focuses on the textual space and injects the document identity in a more straightforward manner.
Let us define the length of text $c$ in textual space as $|c|$. Before embedding, each document $d$ is enriched by concatenating it with predicted queries such that the ratio of the length of the predicted queries and the length of the original document is about $\beta$. Then, the final embedding is derived as the average representation of these enriched documents.
(5)
$\displaystyle d^{*}=\text{concat}(d,\{q_{j}\}_{j=1}^{k}),\quad s.t.\;|\text{
concat}(\{q_{j}\}_{j=1}^{k})|\approx\beta|d|.$
$\displaystyle\text{QAE}_{\text{txt}}(d)=\frac{1}{n}\sum\limits_{i=1}^{n}
\mathcal{E}(d_{i}^{{}^{*}})=\frac{1}{n}\sum\limits_{i=1}^{n}\mathcal{E}(\text{
concat}(d,\{q_{j}\}_{j=1}^{k})).$
For each $d^{*}$, $\{q_{j}\}_{j=1}^{k}$ are randomly and independently sampled from the query generator $\mathcal{Q}(\cdot)$.
We point out that the hyperparameter $\beta$ determines the proportion of the original document and the potential queries in the concatenated text. When $\beta$ is low, the concatenated text degrades to the original document; when $\beta$ is high, the document information is overwhelmed by the repeated queries, where the repetition also hinders the query matching.
- •
Hybrid fingerprint - $\text{{QAE}}_{\text{{hyb}}}$.
The hybrid approach, QAEhyb, seeks to combine the benefits of both the embedding and textual fingerprints for more sophisticated and nuanced document representation.
Although QAEemb combines the document embedding $\mathcal{E}(d)$ and the cluster center $\mathbb{E}[\mathcal{E}(\mathcal{Q}(d))]$ through linear interpolation, inherent differences between these embeddings suggest that a simple linear interpolation should be suboptimal. Therefore, we explore the potential of substituting the document embedding $\mathcal{E}(d)$ in Equation 4 with QAEtxt, which fuses the semantics of both documents and queries.
(6)
$\text{QAE}_{\text{hyb}}(d)=(1-\alpha)\cdot\text{QAE}_{\text{txt}}(d)+\alpha
\cdot\text{QAE}_{\text{base}}(d).$
Note that a more straightforward alternative can be $\text{QAE}_{\text{hyb'}}(d)=(1-\alpha)\cdot\mathcal{E}(d)+\alpha\cdot\text{QAE
}_{\text{txt}}(d)$, which replaces $\text{QAE}_{\text{base}}$ in Equation 4 with $\text{QAE}_{\text{txt}}$ that integrates more document information. However, this substitution faces insufficient query semantics and limited improvements.

In our implementation, all the calculated embeddings are normalized for standardized cosine similarity measurement. Our experiments confirm $\text{QAE}_{\text{emb}}$ and $\text{QAE}_{\text{hyb}}$ outperform $\text{QAE}_{\text{base}}$ and $\text{QAE}_{\text{txt}}$.

## 4. Experiments

##### Datasets and Metrics.

To rigorously assess the effectiveness of QAEncoder, we employ six well-known datasets: Natural Questions (NQ) (Kwiatkowski et al., ), SQuAD (Rajpurkar et al., ), ELI5 (Fan et al., ), TriviaQA (Joshi et al., ), HotPotQA (Yang et al., ) and MSMARCO (Nguyen et al., ). These datasets are extensively used for evaluating machine reading comprehension and QA systems, providing a solid foundation for performance evaluation. However, due to classical datasets are frequently utilized for pre-training or fine-tuning embedding models, they fall short in objectively assessing and accurately reflecting the generalized alignment capabilities for queries and documents, particularly for rapidly evolving and updated knowledge base. Recognizing this limitation, we further test on two latest news datasets, the Chinese dataset CRUD-RAG (Lyu et al., ) and multilingual dataset FIGNEWS (Zaghouani et al., ) covering English, Arabic, French, Hindi and Hebrew. Following the previous work, we evaluate on the development subset; the evaluation metrics include mean reciprocal rank (MRR), mean average precision (MAP), and normalized discounted cumulative gain (NDCG), capturing both the recall and ranking capabilities. We leave the result on MSMARCO, TriviaQA, HotPotQA, FIGNEWS(French), FIGNEWS(Hindi), FIGNEWS(Hebrew) datasets and MAP metric in the appendix for space reason. Refer to the appendix for more implementation details.

### 4.1. Main Results

We mainly compare QAEncoder against the vanilla encoders. Query-centric methods for sparse retrievers are also presented. Our comparison involves the following approaches:

- •
Sparse retrievers - BM25 (Robertson and Zaragoza, ), Doc2Query (Cheriton, ) and DeepImpact (Mallia et al., ).
- •
Dense retrievers - The state-of-the-art embedding models such as BGE models (Xiao et al., ) by BAAI, E5 models (Wang et al., ) by Microsoft, GTE models (Zhang et al., ) by Alibaba-NLP, Jina models (Günther et al., ) by Jina AI; Other well-known models like Contriever models (Izacard et al., ) by Facebook Research, BCEmbedding models (NetEase Youdao, ) by NetEase Youdao and the popular Text2Vec models (Xu, ). These models feature multilingual understanding and task-specific instruction tuning capabilities. We integrate them with QAEncoder to bridge the document-query gap.

#### 4.1.1. Experiments On Classical Datasets

**Table 1. Retrieval performance on classical datasets NQ, SQuAD and ELI5. Higher is better, the best one is bolded. Hyperparameters including QAEncoder variants and weight terms $\alpha,\;\beta$ are optimized simultaneously for six classical datasets. - denotes default or null values.**
| Model | Param | NQ | SQuAD | ELI5 |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| MRR@8 | NDCG@8 | MRR@8 | NDCG@8 | MRR@8 | NDCG@8 |  |  |
| Sparse |  |  |  |  |  |  |  |
| BM25 | - | 17.9 | 12.8 | 46.2 | 48.2 | 12.6 | 11.5 |
| Doc2Query | - | 18.2 | 13.6 | 45.5 | 48.2 | 12.6 | 11.8 |
| DeepImpact | - | 21.8 | 15.7 | 47.8 | 50.0 | 12.9 | 11.9 |
| Dense |  |  |  |  |  |  |  |
| bge-large-en-v1.5 | - | 87.0 | 68.0 | 76.2 | 79.9 | 57.7 | 55.8 |
| QAEhyb, $\alpha=0.15,\;\beta=0.5$ | 88.2 | 69.1 | 78.2 | 81.8 | 59.1 | 57.3 |  |
| multilingual-e5-large | - | 86.0 | 66.3 | 86.2 | 88.9 | 39.0 | 37.6 |
| QAEhyb, $\alpha=0.15,\;\beta=1.5$ | 86.1 | 66.7 | 84.9 | 87.9 | 46.4 | 43.3 |  |
| gte-base-en-v1.5 | - | 86.2 | 67.6 | 68.1 | 72.4 | 54.5 | 51.6 |
| QAEhyb, $\alpha=0.3,\;\beta=0.5$ | 85.5 | 67.4 | 74.8 | 78.7 | 57.0 | 55.1 |  |
| jina-embeddings-v2-small-en | - | 82.4 | 63.3 | 69.5 | 73.6 | 54.3 | 51.6 |
| QAEhyb, $\alpha=0.15,\;\beta=0.5$ | 83.3 | 64.1 | 72.4 | 76.2 | 53.8 | 51.6 |  |
| contriever | - | 78.8 | 60.8 | 64.8 | 69.4 | 51.3 | 49.8 |
| QAEemb, $\alpha=0.45$ | 84.0 | 65.7 | 74.9 | 78.9 | 55.7 | 54.4 |  |
| mcontriever | - | 52.1 | 37.6 | 49.3 | 54.9 | 43.0 | 40.6 |
| QAEhyb, $\alpha=0.45,\;\beta=0.75$ | 61.4 | 45.9 | 64.7 | 69.5 | 51.0 | 48.3 |  |
| bce-embedding-base-v1 | - | 74.4 | 55.5 | 77.2 | 81.3 | 47.3 | 45.0 |
| QAEemb, $\alpha=0.3$ | 76.4 | 56.8 | 77.1 | 81.1 | 50.1 | 48.4 |  |
| text2vec-base-multilingual | - | 53.8 | 36.6 | 40.9 | 45.6 | 38.2 | 34.7 |
| QAEhyb, $\alpha=0.6,\;\beta=0.5$ | 66.6 | 47.2 | 56.2 | 60.6 | 43.4 | 40.8 |  |

As shown in Table 1, for sparse retrievers, Doc2Query exhibits some improvements over the standard BM25 method. However, the improvements are not as significant as DeepImpact. Conversely, the DeepImpact method, which builds on Doc2Query and incorporates BERT for term weight assignment, achieves better performance across all metrics and datasets. This not only suggests that non-neural retrievers have limited alignment capability, but also highlights the importance of weight adjustment of document and query information.

Compared with sparse retrievers, dense retrievers generally outperform significantly. For the state-of-the-art embedding models such as BGE, E5, and GTE, integrating them with QAEncoder can result in considerable performance enhancements, particularly for rare or unseen datasets. For instance, the multilingual-e5-large model witnesses an MRR increase from 39.0 to 46.4 on the ELI5 dataset, due to the ELI5 dataset is heavily down-sampled during the fine-tuning; while the gte-base-en-v1.5 model improves its MRR from 68.1 to 74.8 on the SQuAD dataset, due to the SQuAD dataset is not included in the fine-tuning data. It suggests that even state-of-the-art embedding models suffer limited generalization, while QAEncoder enjoys robust performance boost. For other popular models as Contriever, BCEmbedding, and Text2Vec, QAEncoder significantly improves them due to their poor generalization caused by relatively smaller training datasets. E.g., the Contriever and mContriever models achieve improvements of 10.5 and 15.5 MRR points on the SQuAD dataset respectively.

#### 4.1.2. Experiments On Latest Datasets

In scenarios such as search engines, financial analysis, and news QA, large volumes of new data constantly emerge and are indexed into retrieval bases for accurate and up-to-date response. Hence, the alignment capability for previously unseen user queries and relevant documents is crucial for embedding models in RAG systems. We experiment on the latest news datasets, the multilingual dataset FIGNEWS and the Chinese dataset CRUD-RAG, to avoid data leakage and mimic the real-world scenarios.

**Table 2. Retrieval performance on the latest datasets FIGNEWS and CRUD-RAG. Higher is better, the best one is bolded. Hyperparameters including QAEncoder variants and weight terms $\alpha,\;\beta$ are optimized simultaneously for all lastest datasets. - denotes default or null values.**
| Model | Param | FIGNEWS(English) | FIGNEWS(Arabic) | CRUD-RAG(Chinese) |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| MRR@8 | NDCG@8 | MRR@8 | NDCG@8 | MRR@8 | NDCG@8 |  |  |
| bge-m3 | - | 74.3 | 78.2 | 77.7 | 80.5 | 86.8 | 88.9 |
| QAEtxt, $\beta=1.5$ | 77.1 | 80.8 | 80 | 82.8 | 89 | 90.8 |  |
| multilingual-e5-small | - | 70.9 | 74.7 | 74 | 77.1 | 81.7 | 84.2 |
| QAEhyb, $\alpha=0.3,\;\beta=0.5$ | 73.5 | 77.5 | 78.8 | 81.2 | 89.1 | 90.6 |  |
| multilingual-e5-base | - | 74.7 | 77.9 | 72.2 | 75.7 | 86.1 | 88.1 |
| QAEemb, $\alpha=0.3$ | 77.4 | 80.7 | 77.1 | 80 | 88.5 | 90.5 |  |
| multilingual-e5-large | - | 73.8 | 77.5 | 76.6 | 79.9 | 85.7 | 88.1 |
| QAEhyb, $\alpha=0.15,\;\beta=1.25$ | 77 | 80.4 | 82.1 | 84.9 | 89.1 | 91.1 |  |
| gte-multilingual-base | - | 65.3 | 69.9 | 73.2 | 76.2 | 82.9 | 85.6 |
| QAEhyb, $\alpha=0.15,\;\beta=1.5$ | 75.3 | 79 | 76.2 | 79 | 85.5 | 88 |  |
| mcontriever | - | 32.8 | 36.3 | 40.1 | 44.2 | 71.7 | 75.8 |
| QAEhyb, $\alpha=0.45,\;\beta=1.25$ | 61.2 | 65.4 | 68.2 | 71.9 | 88.7 | 90.5 |  |
| bce-embedding-base-v1 | - | 58.9 | 63.3 | - | - | 76.8 | 80.3 |
| QAEhyb, $\alpha=0.3,\;\beta=0.5$ | 66.6 | 70.6 | - | - | 85.9 | 88.1 |  |
| text2vec-base-multilingual | - | 38.5 | 43 | 27.7 | 31.4 | 12.1 | 13.7 |
| QAEemb, $\alpha=0.75$ | 55.2 | 59.3 | 51.3 | 54.8 | 55.3 | 58.6 |  |

As illustrated in Table 2, for the latest datasets, QAEncoder significantly enhances the alignment of user queries with relevant documents across both state-of-the-art embedding models and other popular open-source models. For instance, the gte-multilingual-base model’s MRR metric on the FIGNEWS(English) dataset increases from 65.3 to 75.3. Similarly, the mContriever model’s MRR on the FIGNEWS(English) dataset improves from 32.8 to 61.2. Besides, the text2vec-base-multilingual model’s MRR on the CRUD-RAG dataset rose from 12.1 to 55.3. These results confirm QAEncoder’s remarkably generalized alignment capability across various embedding models and multilingual datasets.

### 4.2. Analysis and Discussion

For a more comprehensive assessment, we also report results of various QAEncoder ablations, i.e. QAEemb, QAEtxt, QAEhyb and QAEnaive which directly stores predicted queries. Finally, we also discuss the relationship between QAEncoder and training-based as well as document-centric methods.

#### 4.2.1. Ablation Studies

**Table 3. Performance comparison of QAEncoder variants on the latest datasets FIGNEWS and CRUD-RAG. Higher is better, the best one is bolded. Hyperparameters are optimized simultaneously across all the latest datasets. $n$ indicates the number of predicted queries in QAnaive.**
| Model | Param | FIGNEWS(English) | FIGNEWS(Arabic) | CRUD-RAG(Chinese) |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| MRR@8 | NDCG@8 | MRR@8 | NDCG@8 | MRR@8 | NDCG@8 |  |  |
| bge-m3 | QAEemb, $\alpha=0.3$ | 76.2 | 80.0 | 80.0 | 82.6 | 88.7 | 90.6 |
| QAEtxt, $\beta=1.5$ | 77.1 | 80.8 | 80.1 | 82.8 | 89.0 | 90.8 |  |
| QAEhyb, $\alpha=0.15,\;\beta=1.5$ | 77.3 | 80.7 | 80.6 | 83.3 | 89.4 | 91.2 |  |
| QAEnaive, $n=10$ | 76.8 | 79.5 | 76.9 | 79.2 | 85.9 | 87.8 |  |
| multilingual-e5-large | QAEemb, $\alpha=0.45$ | 77.8 | 81.1 | 79.7 | 82.7 | 89.7 | 91.4 |
| QAEtxt, $\beta=1.5$ | 75.6 | 79.0 | 80.7 | 83.7 | 88.3 | 90.4 |  |
| QAEhyb, $\alpha=0.15,\;\beta=1.25$ | 77.0 | 80.5 | 82.1 | 84.9 | 89.1 | 91.1 |  |
| QAEnaive, $n=10$ | 76.8 | 79.4 | 76.5 | 79.2 | 84.9 | 86.8 |  |

We present the performance comparison of QAEncoder variants on two state-of-the-art multilingual embedding models in Table 3.
Generally, QAEhyb and QAEemb outperform the QAEtxt and QAEnaive approaches.
For instance, for the bge-m3 model, QAEhyb consistently outperforms other variants. Conversely, the multilingual-e5-large model performs best with QAEemb. However, the best performance differences between QAEemb, QAEtxt, and QAEhyb are not substantial, demonstrating the robustness of our approach to hyperparameter variations.
Regarding QAEnaive, it evidently underperforms other ablations, despite storing 10 times the number of embedding vectors. This leads to unacceptable storage management overhead and recall latency in large-scale production systems.
We also provide some more granular ablation experiments in Figure 4 and the appendix.

Figure: Figure 4. Ablation on $\alpha$ and $n$ hyperparameters for QAEemb and QAEnaive on FIGNEWS(English) dataset.
Refer to caption: extracted/5888105/images/Ablation_alpha.png

#### 4.2.2. Training-based and Document-centric Methods

Training-based approaches mainly include two categories, fine-tuning on QA datasets and multi-task instruction datasets. We choose mContriever-msmarco and multilingual-e5-large-instruct as representative models respectively. Finally, we consider HyDE (Gao et al., ), a well-known document-centric method for comparison.

**Table 4. Performance comparison of QAEncoder against training-based and document-centric methods on the latest datasets FIGNEWS and CRUD-RAG. Higher is better, the best one is bolded. Hyperparameters $\alpha,\;\beta$ are optimized simultaneously across the two latest datasets. $n$ denotes the number of pseudo-documents in HyDE. - indicates default or null values.**
| Model | Param | FIGNEWS(English) | FIGNEWS(Arabic) | CRUD-RAG(Chinese) |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| MRR@8 | NDCG@8 | MRR@8 | NDCG@8 | MRR@8 | NDCG@8 |  |  |
| mContriever-msmarco | - | 65.9 | 70.0 | 70.1 | 73.5 | 85.0 | 87.4 |
| QAEhyb, $\alpha=0.3,\;\beta=0.75$ | 72.1 | 76.3 | 77.2 | 80.2 | 88.6 | 90.6 |  |
| multilingual-e5-large-instruct | - | 66.7 | 70.7 | 74.9 | 78.1 | 79.8 | 82.5 |
| QAEhyb, $\alpha=0.15,\;\beta=1.5$ | 75.5 | 79.4 | 80.7 | 83.5 | 88.9 | 90.6 |  |
| mContriever | - | 32.8 | 36.3 | 40.1 | 44.2 | 71.7 | 75.8 |
| HyDE, $n=8$ | 24.9 | 27.6 | 35.1 | 40.4 | 70.7 | 74.2 |  |
| multilingual-e5-large | - | 73.8 | 77.5 | 76.6 | 79.9 | 85.7 | 88.2 |
| HyDE, $n=8$ | 63.4 | 67.7 | 68.2 | 73.8 | 81.5 | 83.9 |  |

Training-based and query-centric methods operate at training time and indexing time, respectively. Therefore, integrating these approaches cloud lead to more improvements.
As illustrated in Table 4, both types of fine-tuned models significantly benefit from the QAEncoder. For instance, the mContriever-msmarco model improves MRR from 70.1 to 77.2 on FIGNEWS(Arabic); the multilingual-e5-large-instruct model’s MRR increases 8.8 and 9.1 MRR points on the FIGNEWS(English) and CRUD-RAG(Chinese) datasets, respectively.

For document-centric methods, we investigate the widely-reported hallucination phenomenon on latest datasets (Wang et al., ; Gao et al., ; Kim and Min, ). Table 4 shows the recall performance heavily decreases for all the latest datasets and both models, attributed to the hallucination of pseudo-documents. Besides, the LLM invocation for pseudo-documents is both time-consuming and costly. In our case, the time for single LLM invocation is greater than 2000ms while the time for vector search is less than 10ms. These highlight the irreplaceable importance and practicality of QAEncoder method.

## 5. Conclusion

In this paper, we introduced QAEncoder, a training-free approach to bridge the document-query gap for more advanced QA systems.
Based on our conical distribution hypothesis and document fingerprint strategies, QAEncoder substitutes document embeddings with the expectation of query embeddings, enriched with document information.
Extensive experiments across multiple datasets, languages and embedding models confirm that QAEncoder not only avoids extra index storage, retrieval latency, training costs, and hallucination, but also ensures robust generalization and diversified query handling.

## Appendix A Appendix

### A.1. Dataset Details

- •
MS-MARCO is a large-scale dataset specifically designed for machine reading comprehension. It comprises real user queries collected through Bing search, paired with corresponding passages retrieved from a comprehensive web document collection.
We evaluated our model on the passage retrieval task using a corpus of approximately 8.8M passages.
- •
Natural Questions (NQ) is a widely used dataset comprising real user queries submitted to the Google search engine, paired with relevant Wikipedia pages. Each query is annotated with a long answer (typically a paragraph) and a short answer (one or more entities). Each question-answer pair corresponds to a single Wikipedia page, ensuring clarity in evaluating information retrieval performance.
- •
TriviaQA is a dataset of question-answer-evidence triples, with over 650K entries, including 95K question-answer pairs created by trivia enthusiasts. The evidence documents, typically sourced from Wikipedia or other web sources, average six per question.
- •
HotPotQA contains 113K Wikipedia-based question-answer pairs that require multi-hop reasoning across multiple pages. Each pair includes a set of supporting sentences, which are treated as evidence documents.
- •
ELI5 is a question-answering dataset consisting of question-answer-evidence triples, where the questions are complex and often require detailed, multi-sentence responses. It contains 270K diverse and intricate questions that necessitate explanatory answers. To support each response, web search results are utilized as evidence documents, enhancing the reliability of the provided answers.
- •
SQuAD is a widely used reading comprehension dataset comprising 107K question-answer pairs derived from 536 Wikipedia articles. Each question are specific text segments from the relevant
articles. In this dataset, the Wikipedia articles function as evidence documents, supplying the essential context required for accurate response retrieval.
- •
CRUD-RAG is a benchmark specifically designed for evaluating RAG systems, incorporating the latest high-quality news data that were not included in the training phase of the language models. It comprises more than 80K news articles sourced from prominent Chinese news websites, all published after July 2023.
From the set of queries generated by GPT-4o-mini for each document, we randomly sample one to serve as the test query. The original news is designated as the evidence documents for recall evaluation, ensuring queries are associated with exactly relevant documents.(^1^11LlamaIndex adopts the same common practice and provides templated workflow, i.e. generating queries for recall and rerank evaluation. See [https://www.llamaindex.ai/blog/boosting-rag-picking-the-best-embedding-reranker-models-42d079022e83](https://www.llamaindex.ai/blog/boosting-rag-picking-the-best-embedding-reranker-models-42d079022e83).)
- •
FIGNEWS is a multilingual news post dataset designed to examine bias and propaganda within news articles across different languages. It consists of 15,000 publicly available news posts collected from verified blue-check accounts between October 7, 2023, and January 31, 2024. The dataset includes posts in five languages—English, Arabic, Hebrew, French, and Hindi—distributed evenly across 15 batches, each containing 1,000 posts. Each batch consists of 200 posts for each language. Similar to CRUD-RAG, we randomly sample one predicted query generated by GPT-4o-mini for each document and use the original news as the evidence documents for recall evaluation.

### A.2. Metric Details

For SQuAD, CRUD-RAG and FIGNEWS benchmarks, MAP metric is equivalent to MRR, as there is only one related document per query.

- •
Mean Reciprocal Rank (MRR)
Mean Reciprocal Rank is a statistic measure used to evaluate the effectiveness of a retrieval system by calculating the reciprocal of the rank at which the first relevant result appears. The mathematical formulation is:
$\text{MRR}=\frac{1}{|Q|}\sum_{i=1}^{|Q|}\frac{1}{\text{rank}_{i}}$
Where: - $|Q|$ is the number of queries. - $\text{rank}_{i}$ is the rank position of the first relevant document for the $i$-th query.
- •
Mean Average Precision (MAP)
Mean Average Precision is a measure that combines precision and recall. It computes the average precision value for a set of queries and then averages these values. The mathematical formulation is:
$\text{MAP}=\frac{1}{|Q|}\sum_{i=1}^{|Q|}\text{AP}(i),$
Where: - $|Q|$ is the number of queries. - $\text{AP}(i)$ is the average precision for the $i$-th query.
The average precision for a single query is given by:
$\text{AP}=\frac{1}{R}\sum_{k=1}^{n}P(k)\cdot\text{rel}(k),$
Where: - $R$ is the total number of relevant documents for the query. - $n$ is the number of retrieved documents. - $P(k)$ is the precision at cut-off $k$. - $\text{rel}(k)$ is an indicator function equating to 1 if the document at rank $k$ is relevant, otherwise 0.
- •
Normalized Discounted Cumulative Gain (NDCG)
Normalized Discounted Cumulative Gain is a measure of ranking quality that takes into account the positions of the relevant documents. It is based on the concept of discounting the relevance of documents based on their position in the result list. The mathematical formulation is:
$\text{NDCG}=\frac{DCG_{p}}{IDCG_{p}},$
Where: - $DCG_{p}$ is the Discounted Cumulative Gain at position $p$. - $IDCG_{p}$ is the Ideal Discounted Cumulative Gain at position $p$, which is the DCG score of the perfect ranking.
The Discounted Cumulative Gain at position $p$ is given by:
$DCG_{p}=\sum_{i=1}^{p}\frac{2^{\text{rel}_{i}}-1}{\log_{2}(i+1)},$
Where: - $\text{rel}_{i}$ is the relevance score of the document at rank $i$.
The Ideal Discounted Cumulative Gain $IDCG_{p}$ is computed in the same way as $DCG_{p}$, except that the documents are ideally sorted by relevance.

### A.3. Baseline Details

- •
BM25 (Robertson and Zaragoza, ) is the traditional lexical retriever based on term relevance and frequency, regarded as the most popular variation of TF-IDF.
- •
DocT5Query (Cheriton, ) appends generated queries to the document before building the inverted index of BM25.
- •
DeepImpact (Mallia et al., ) leverages DocT5Query to enrich the document collection and estimates the semantic importance of tokens with fine-tuned BERT.
- •
BGE models (Xiao et al., )by BAAI, Jina models (Günther et al., ) by Jina AI, E5 models (Wang et al., ) by Microsoft, GTE models (Zhang et al., ) by Alibaba-NLP are the most advanced embedding models, featuring multilingual understanding and task-specific instruction tuning capabilities. We choose both vanilla encoders and QA-specific instruction-tuned encoders for test.
- •
Contriever models (Izacard et al., ) are developed by Facebook Research, including Contriever, mContriever, Contriever-msmacro and mContriever-msmacro. mContriever serves as the multilingual version of Contriever. Contriever-msmacro and mContriever-msmacro are further fine-tuned on the MS-MACRO dataset for bridging the document-query gap.
- •
BCEmbedding models (NetEase Youdao, ), developed by NetEase Youdao, are bilingual and crosslingual embedding models in English and Chinese. BCEmbedding serves as the cornerstone of Youdao’s RAG-based QA system, QAnything, an open-source project widely integrated in commercial products like Youdao Speed Reading and Youdao Translation. We choose bce-embedding-base for test.
- •
Text2Vec models (Xu, ) is a popular open-source project that implements Word2Vec (Mikolov et al., ), RankBM25, BERT (Devlin et al., ), Sentence-BERT (Reimers and Gurevych, ), CoSENT and other text representation models. We test its most prominent model, text2vec-base-multilingual, which supports multiple languages, including German, English, Spanish, French, Italian, Dutch, Polish, Portuguese, Russian, and Chinese.
- •
Training-based approaches mainly include two categories, fine-tuning on QA datasets or multi-task instruction datasets. For fine-tuning on QA datasets, we choose mContriever-msmarco for test, an enhanced variant of the mContriever model that has been fine-tuned on the MS-MARCO.
The second category involves fine-tuning models on multi-task instruction datasets, where distinct prompt prefixes are appended to the input text, enabling the model to effectively differentiate between various tasks. In this category, we test the multilingual-e5-large-instruct (Wang et al., ) developed by Microsoft, which leverages synthetic instruction data (Wang et al., ) for fine-tuning.
- •
Document-centric methods instruct LLMs to generate a pseudo-document for each query. The pseudo-document aims to capture relevant information but does not correspond to a real document and may contain inaccuracies and hallucinations. Subsequently,the pseudo-document is encoded, and its embedding is utilized to retrieve similar real documents based on vector similarity. We choose Hyde (Gao et al., ) for test.

### A.4. Prompts

We employ specialized prompts to instruct gpt-4o-mini as the question and pseudo-document generator. Only the English version is presented due to LaTeX compilation issues with non-English languages.

### A.5. Proof of the Conical Distribution Hypothesis

This subsection provides the proof of the Conical Distribution Hypothesis, which proposes that potential queries form a distinct cluster on a hyperplane in semantic space.

###### Proof.

Our validation is structured from three core aspects:

- •
Single-cluster sub-hypothesis verification.
As illustrated in Fig. 3(a), we validate the single-cluster sub-hypothesis by visualizing the embedding space using t-SNE dimensionality reduction techniques. This visualization displays that the predicted queries for each document form distinct and cohesive clusters (different colored). And these clusters are notably distant from the clusters of other documents, thereby supporting the single-cluster sub-hypothesis.
- •
Perpendicular sub-hypothesis verification.
To further assess the perpendicular sub-hypothesis, let $v_{d}=\mathcal{E}(d)-\mathbb{E}[\mathcal{E}(\mathcal{Q}(d))]$ and $v_{q_{i}}=\mathcal{E}(q_{i})-\mathbb{E}[\mathcal{E}(\mathcal{Q}(d))]$ be the vectors from the cluster center to the document embedding and the individual query embedding, respectively. As illustrated in Fig. 3(b), the degree distribution between vector $v_{d}$ and vector $v_{q_{i}}$ exhibits a bell-shaped curve. The mean value is slightly less than 90 degrees, and the primary range of distribution lies between 75 and 100 degrees, which confirms that $v_{d}$ is approximately orthogonal to each $v_{q_{i}}$ and can be regarded as the normal vector to some hyperplane $\mathcal{H}$.
- •
Conical distribution in unit sphere demonstration.
Finally, we illustrate the highly simplified conical distribution hypothesis within the unit sphere embedding space, as most embedding models utilize normalized embedding vectors. As depicted in Fig. 3(c), the embeddings of potential queries form a cluster on the surface of the unit sphere, with each point color-coded. The center of the cluster is indicated by a star, while the document embedding is represented by a black point positioned above the cluster. It is evident that these elements form a distorted cone, aligning with the above hypothesis and the degree distribution experiment.

∎

###### Proof.

### A.6. Strong Conical Distribution Hypothesis

In this subsection, we further substantiate the original hypothesis that the potential queries adhere to a Gaussian distribution:
For any document $d$, the potential queries in the embedding space approximately follow a Gaussian distribution, characterized by a mean $\mu$ and covariance matrix $\Sigma$. Refer to A.6.3 for detailed validation.

#### A.6.1. Main theorem

Building on this Gaussian assumption, we derive bounds on the cosine similarity between potential query embeddings and both the document embedding and the mean vector.

###### Theorem 1 .

(Concentration Inequalities for Cosine Similarities in Embedding Spaces) Let $\mathbf{q}\sim\mathcal{N}(\mu,\Sigma)$ denote a random vector representing the distribution of potential queries of document $d$ in the unit sphere embedding space, where $\mu\in\mathbb{R}^{r}$ is the mean vector and $\Sigma\in\mathbb{R}^{r\times r}$ is the covariance matrix. Let $\mathbf{d}\in\mathbb{R}^{r}$ be the embedding of document $d$, and let $\theta$ be the angle between $\mu$ and $\mathbf{d}$ such that $\cos(\theta)=\mu^{\top}\mathbf{d}$. Assume that both $\mu$ and $\mathbf{d}$ are unit vectors. Then, the following properties hold:

- (1)
The concentration inequality for the cosine similarity measure of $\mathbf{q}$ with $\mathbf{d}$ :
(7)
$\mathbb{P}\left(\left|\mathbf{q}^{\top}\mathbf{d}-\cos(\theta)\right|\geq t
\right)\leq 2\exp\left(-\frac{t^{2}}{2\mathbf{d}^{\top}\Sigma\mathbf{d}}\right).$
- (2)
The concentration inequality for the cosine similarity measure of $\mathbf{q}$ with $\mu$ :
(8)
$\mathbb{P}\left(\left|\mathbf{q}^{\top}\mu-1\right|\geq t\right)\leq 2\exp
\left(-\frac{t^{2}}{2\mu^{\top}\Sigma\mu}\right).$
- (3)
Non-Negativity of the Difference in Similarities:
(9)
$\mathbf{q}\top\mu-\mathbf{q}\top\mathbf{d}=\mathbf{q}^{\top}\mu(1-\cos(\theta)
)>0.$

###### Remark 1 .

The theorem provides a robust theoretical foundation for the QAEncoder’s capabilities in computing the similarity between documents and queries.
These concentration inequalities in equation 7 and equation 8 show that cosine similarities between $\mathbf{q}$, $\mathbf{d}$, and $\mu$ are concentrated around their expected values.
Inequality 9 indicates that the similarity between the $\mathbf{q}$ and $\mu$ is always greater than between $\mathbf{q}$ and $\mathbf{d}$. This confirms that using the mean vector as a projection in QAEncoder better captures the semantic relationship between queries and documents. This theoretical result aligns with the experimental findings 1, further validating QAEncoder’s effectiveness .

###### Theorem 1 .

###### Remark 1 .

#### A.6.2. Proof of Similarity Bounds

###### Proof.

Given the setup where $\mathbf{q}\sim\mathcal{N}(\mu,\Sigma)$ is an $r$-dimensional Gaussian random vector with mean $\mu$ and covariance $\Sigma$, and the angle between another unit vector $\mathbf{d}$ and $\mu$ is $\theta$.

The cosine of the angle between $\mathbf{q}$ and $\mathbf{d}$ is given by:

$$ $\cos(\phi_{qd})=\mathbf{q}^{\top}\mathbf{d}.$ $$

Since $\mathbf{q}$ is Gaussian, based on Lemma 2,the inner product $\mathbf{q}^{\top}\mathbf{d}$ is a linear transformation of $\mathbf{q}$ and hence is a normal distribution with mean $\mu^{\top}\mathbf{d}$ and variance $\mathbf{d}^{\top}\Sigma\mathbf{d}$. Given that $\mu$ and $\mathbf{d}$ are unit vectors and the angle between $\mu$ and $\mathbf{d}$ is $\theta$, we have:

$$ $\mu^{\top}\mathbf{d}=\cos(\theta).$ $$

Thus, $\mathbf{q}^{\top}\mathbf{d}$ can be approximated as:

$$ $\mathbf{q}^{\top}\mathbf{d}\sim\mathcal{N}(\cos(\theta),\mathbf{d}^{\top} \Sigma\mathbf{d}).$ $$

The concentration inequality for the cosine value between $\mathbf{q}$ and $\mathbf{d}$ follows from Hoeffding’s inequality for zero-mean sub-Gaussian random variables, which can be expressed as:

$$ $\mathbb{P}\left(\left|\mathbf{q}^{\top}\mathbf{d}-\cos(\theta)\right|\geq t \right)\leq 2\exp\left(-\frac{t^{2}}{2\mathbf{d}^{\top}\Sigma\mathbf{d}}\right).$ $$

Similarly, the cosine of the angle between between $\mathbf{q}$ and $\mu$ can be expressed as:

$$ $\cos(\phi_{q\mu})=\mathbf{q}^{\top}\mu.$ $$

Given that $\mathbf{q}\sim\mathcal{N}(\mu,\Sigma)$, based on Lemma 2, the inner product $\mathbf{q}^{\top}\mu$,
representing the cosine of the angle between $\mathbf{q}$ and $\mu$
is a linear transformation of a Gaussian random vector with mean:

$$ $\mathbb{E}[\mathbf{q}^{\top}\mu]=\mu^{\top}\mu=1,$ $$

since $\mu$ is a unit vector, and variance:

$$ $\mathrm{Var}[\mathbf{q}^{\top}\mu]=\mu^{\top}\Sigma\mu.$ $$

Thus, $\mathbf{q}^{\top}\mu$ can be approximated as:

$$ $\mathbf{q}^{\top}\mu\sim\mathcal{N}(1,\mu^{\top}\Sigma\mu).$ $$

Applying the similar Hoeffding’s inequality, the concentration inequality can be derived similarly:

$$ $\mathbb{P}\left(\left|\mathbf{q}^{\top}\mu-1\right|\geq t\right)\leq 2\exp \left(-\frac{t^{2}}{2\mu^{\top}\Sigma\mu}\right).$ $$

Notably, we observe that:

$$ $\mathbf{q}^{\top}\mathbf{d}=(\mathbf{q}^{\top}\mu)(\mu^{\top}\mathbf{d})=( \mathbf{q}^{\top}\mu)\cos(\theta).$ $$

Therefore, comparing $\mathbf{q}^{\top}\mu$ and $\mathbf{q}^{\top}\mathbf{d}$, we find:

$$ $\mathbf{q}^{\top}\mu-\mathbf{q}^{\top}\mathbf{d}=\mathbf{q}^{\top}\mu(1-\cos( \theta)).$ $$

Since $\mathbf{q}\neq\mathbf{d}$, it follows that $\cos(\theta)<1$.
Moreover, $\mathbf{q}^{\top}\mu>0$ because $\mathbf{q}$ is a Gaussian random vector centered at $\mu$, which implies that $\mathbf{q}$ generally aligns positively with its mean $\mu$. Given that both $1-\cos(\theta)>0$ and $\mathbf{q}^{\top}\mu>0$, we have:

$$ $\mathbf{q}^{\top}\mu-\mathbf{q}^{\top}\mathbf{d}=\mathbf{q}^{\top}\mu(1-\cos( \theta))>0.$ $$

∎

###### Proof.

#### A.6.3. Normality Test

To assess whether potential queries follow a Gaussian distribution in the embedding space,we employ two statistical tests: the Chi-Squared Q-Q Plot and the Anderson-Darling Test.

##### Chi-Squared Q-Q Plot Verification

We employ the Chi-Squared Q-Q Plot to assess whether the squared Mahalanobis distances 3 conform to the chi-squared distribution. By leveraging Lemma 4, We compare the observed $D^{2}$ values with the theoretical quantiles of the chi-squared distribution to assess the conformity of the data to a high-dimensional Gaussian model.
We conclude that close alignment of the sample points along the 45-degree reference line indicates support for the original hypothesis.

Figure: Figure 5. Q-Q plot against chi-squared distribution
Refer to caption: x3.png

Direct applications of high-dimensional normality tests, such as the HZ test, often lead to Type I errors in our case. E.g., testing on 768-dimensional normal samples revealed that HZ test demands high ratios of sample size to dimensionality, and it particularly susceptible to Type I errors when the sample size is not sufficiently large. Hence, we opted to perform uni-variate normality assessments on marginal distribution instead.

##### Anderson-Darling Test

To further evaluate the normality of marginal distributions, we conduct the Anderson-Darling test across all dimensions of the embeddings. The specific steps are as follows:

- (1)
Null Hypothesis:
•
$H_{0}$: Each dimension’s marginal distribution follows a Gaussian distribution.
•
$H_{1}$: At least one dimension’s marginal distribution does not follow a Gaussian distribution.
- (2)
Testing Results:
Following the Anderson-Darling test, the results across all dimensions failed to reject $H_{0}$. This suggests that each dimension’s marginal distribution can statistically be considered Gaussian, thereby supporting our hypothesis that potential queries conform to a high-dimensional Gaussian distribution.

In summary, through the verification of squared Mahalanobis distances using the Chi-Squared Q-Q Plot and the evaluation of marginal distributions with the Anderson-Darling test, we validate the plausibility that potential queries conform to a high-dimensional Gaussian distribution within the embedding space.

#### A.6.4. Some properties of Gaussian distribution

The statement and proof of our main results contain some mathematical concepts. This section introduces these concepts, covering the fundamental lemmas and definitions essential for understanding our analysis.

###### Lemma 0.

(Tong, )
Let $\mathbf{x}$ follow a multivariate Gaussian distribution:

$$ $\mathbf{x}\sim\mathcal{N}(\mu,\Sigma),$ $$

where $\mu$ is the mean vector and $\Sigma$ is the covariance matrix. For any linear transformation $A\mathbf{x}+\mathbf{b}$, the result is also multivariate normal:

$$ $\mathbf{y}=A\mathbf{x}+\mathbf{b}\sim\mathcal{N}(A\mu+\mathbf{b},A\Sigma A^{ \top}).$ $$

###### Remark 2 .

The lemma shows that multivariate Gaussians remain Gaussian under linear transformations.

###### Definition 0.

(Squared Mahalanobis Distance (McLachlan, ))
Assuming $\mathbf{x}\sim\mathcal{N}(\mu,\Sigma)$ in a high-dimensional Gaussian distribution, the squared Mahalanobis distance $D^{2}$ is defined as:

$$ $D^{2}=(\mathbf{x}-\mu)^{T}\Sigma^{-1}(\mathbf{x}-\mu).$ $$

###### Remark 3 .

This metric quantifies the distance between the observed value and the mean.

###### Lemma 0.

(Chi-Squared Distribution of Squared Mahalanobis Distance (Mardia et al., )
)
If $\mathbf{x}$ is a $r$-dimensional Gaussian random variable, then the squared Mahalanobis distance $D^{2}$ follows a chi-squared distribution:

$$ $D^{2}\sim\chi^{2}_{r},$ $$

where $r$ denotes the dimensionality of the variable.

###### Remark 4 .

The property that the squared Mahalanobis distance follows a chi-squared distribution can be viewed as a form of dimensionality reduction. By mapping a high-dimensional Gaussian variable to a scalar that encodes its deviation from the mean, adjusted for the covariance structure, this transformation reduces the complexity of the multivariate data while preserving key statistical properties in a single distance metric.

###### Lemma 0.

###### Remark 2 .

###### Definition 0.

###### Remark 3 .

###### Lemma 0.

###### Remark 4 .

### A.7. Missing Figures and Tables

**Table 5. Complete retrieval performance across six classical datasets: NQ, SQuAD, ELI5, HotPotQA, MSMARCO, and TriviaQA. Higher is better, the best one is bolded. Hyperparameters including QAEncoder variants and weight terms $\alpha,\;\beta$ are optimized simultaneously for six classical datasets. - denotes default or null values.**
| Model | Param | NQ | SQuAD | ELI5 | HotPotQA | MSMARCO | TriviaQA |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| MRR@8 | MAP@8 | NDCG@8 | MRR@8 | MAP@8 | NDCG@8 | MRR@8 | MAP@8 | NDCG@8 | MRR@8 | MAP@8 | NDCG@8 | MRR@8 | MAP@8 | NDCG@8 | MRR@8 | MAP@8 | NDCG@8 |  |  |
| Sparse |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| BM25 | - | 17.9 | 10.4 | 12.8 | 46.2 | 46.2 | 48.2 | 12.6 | 9.6 | 11.5 | 13.8 | 9.9 | 11.1 | 53.2 | 53.1 | 62.1 | 9.8 | 6.7 | 7.8 |
| Doc2Query | - | 18.2 | 10.9 | 13.6 | 45.5 | 45.5 | 48.2 | 12.6 | 9.7 | 11.8 | 13.3 | 9.6 | 10.9 | 54.4 | 54.4 | 64.0 | 10.1 | 7.0 | 8.3 |
| DeepImpact | - | 21.8 | 12.9 | 15.7 | 47.8 | 47.9 | 50.0 | 12.9 | 9.9 | 11.9 | 14.4 | 10.4 | 11.6 | 64.4 | 64.1 | 71.8 | 11.8 | 8.2 | 9.5 |
| Dense |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| bge-large-en-v1.5 | - | 87.0 | 83.2 | 68.0 | 76.2 | 76.2 | 79.9 | 57.7 | 55.8 | 55.8 | 93.4 | 90.1 | 83.3 | 75.4 | 75.4 | 81.3 | 78.4 | 75.0 | 67.5 |
| QAEhyb, $\alpha=0.15,\;\beta=0.5$ | 88.2 | 84.2 | 69.1 | 78.2 | 78.2 | 81.8 | 59.1 | 56.7 | 57.3 | 94.2 | 91.0 | 83.5 | 73.8 | 73.8 | 79.9 | 80.2 | 77.5 | 70.3 |  |
| multilingual-e5-large | - | 86.0 | 82.3 | 66.3 | 86.2 | 86.2 | 88.9 | 39.0 | 37.8 | 37.6 | 95.3 | 92.8 | 85.4 | 73.6 | 73.6 | 80 | 76.8 | 73.9 | 66.1 |
| QAEhyb, $\alpha=0.15,\;\beta=1.5$ | 86.1 | 81.9 | 66.7 | 84.9 | 84.9 | 87.9 | 46.4 | 45.6 | 43.3 | 94.3 | 91.0 | 83.4 | 70.1 | 70.1 | 77.3 | 79.3 | 77.0 | 68.9 |  |
| gte-base-en-v1.5 | - | 86.2 | 82.5 | 67.6 | 68.1 | 68.1 | 72.4 | 54.5 | 52.5 | 51.6 | 93.4 | 89.7 | 78.4 | 76.9 | 76.9 | 82.4 | 77.7 | 74.8 | 67.2 |
| QAEhyb, $\alpha=0.3,\;\beta=0.5$ | 85.5 | 81.6 | 67.4 | 74.8 | 74.8 | 78.7 | 57.0 | 54.8 | 55.1 | 92.9 | 88.9 | 76.9 | 76.0 | 76.0 | 81.4 | 78.8 | 76.1 | 69.2 |  |
| jina-embeddings-v2-small-en | - | 82.4 | 79.3 | 63.3 | 69.5 | 69.5 | 73.6 | 54.3 | 52.4 | 51.6 | 92.9 | 88.3 | 78.1 | 65.3 | 65.3 | 73.3 | 74.6 | 72.7 | 64.4 |
| QAEhyb, $\alpha=0.15,\;\beta=0.5$ | 83.3 | 79.9 | 64.1 | 72.4 | 72.4 | 76.2 | 53.8 | 52.3 | 51.6 | 91.7 | 87.3 | 76.6 | 68.2 | 68.2 | 75.7 | 75.9 | 73.9 | 65.5 |  |
| contriever | - | 78.8 | 75.0 | 60.8 | 64.8 | 64.8 | 69.4 | 51.3 | 49.2 | 49.8 | 89.0 | 84.6 | 74.1 | 55.4 | 55.4 | 63.6 | 71.2 | 68.6 | 61.4 |
| QAEemb, $\alpha=0.45$ | 84.0 | 80.2 | 65.7 | 74.9 | 74.9 | 78.9 | 55.7 | 54.0 | 54.4 | 89.8 | 86.2 | 75.1 | 67.1 | 67.1 | 74.2 | 76.4 | 73.9 | 67.4 |  |
| mcontriever | - | 52.1 | 48.7 | 37.6 | 49.3 | 49.3 | 54.9 | 43.0 | 41.2 | 40.6 | 83.0 | 78.3 | 67.9 | 49.6 | 49.6 | 56.7 | 59.4 | 56.8 | 49.7 |
| QAEhyb, $\alpha=0.45,\;\beta=0.75$ | 61.4 | 58.5 | 45.9 | 64.7 | 64.7 | 69.5 | 51.0 | 49.1 | 48.3 | 85.2 | 80.9 | 69.5 | 65.1 | 65.1 | 71.8 | 70.4 | 67.6 | 60.4 |  |
| bce-embedding-base-v1 | - | 74.4 | 70.0 | 55.5 | 77.2 | 77.2 | 81.3 | 47.3 | 46.1 | 45.0 | 83.0 | 79.2 | 67.8 | 70.9 | 70.9 | 77.7 | 67.8 | 65.2 | 57.3 |
| QAEemb, $\alpha=0.3$ | 76.4 | 72.3 | 56.8 | 77.1 | 77.1 | 81.1 | 50.1 | 48.4 | 48.4 | 84.1 | 79.7 | 68.0 | 71.2 | 71.2 | 78 | 69.4 | 66.5 | 58.4 |  |
| text2vec-base-multilingual | - | 53.8 | 51.1 | 36.6 | 40.9 | 40.9 | 45.6 | 38.2 | 37.0 | 34.7 | 51.2 | 49.0 | 37.8 | 52.4 | 52.4 | 59.8 | 41.8 | 40.1 | 33.3 |
| QAEhyb, $\alpha=0.6,\;\beta=0.5$ | 66.6 | 63.7 | 47.2 | 56.2 | 56.2 | 60.6 | 43.4 | 42.1 | 40.8 | 72.0 | 69.2 | 52.7 | 67.8 | 67.8 | 74.2 | 57.2 | 55.5 | 47.1 |  |

**Table 6. Comprehensive retrieval performance on the latest datasets FIGNEWS and CRUD-RAG. Higher is better, the best one is bolded. Hyperparameters including QAEncoder variants and weight terms $\alpha,\;\beta$ are optimized simultaneously for six latest datasets. - denotes default or null values.**
| Model | Param | FIGNEWS(English) | FIGNEWS(Arabic) | CRUD-RAG(Chinese) | FIGNEWS(French) | FIGNEWS(Hindi) | FIGNEWS(Hebrew) |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| MRR@8 | NDCG@8 | MRR@8 | NDCG@8 | MRR@8 | NDCG@8 | MRR@8 | NDCG@8 | MRR@8 | NDCG@8 | MRR@8 | NDCG@8 |  |  |
| bge-m3 | - | 74.3 | 78.2 | 77.7 | 80.5 | 86.8 | 88.9 | 73.4 | 76.9 | 58.4 | 63.8 | 78.6 | 81 |
| QAEtxt, $\beta=1.5$ | 77.1 | 80.8 | 80 | 82.8 | 89 | 90.8 | 76.7 | 79.9 | 62.5 | 67.5 | 79.8 | 82.4 |  |
| multilingual-e5-small | - | 70.9 | 74.7 | 74 | 77.1 | 81.7 | 84.2 | 72.8 | 76.1 | 66.6 | 70.1 | 52.3 | 57.2 |
| QAEhyb, $\alpha=0.3,\;\beta=0.5$ | 73.5 | 77.5 | 78.8 | 81.2 | 89.1 | 90.6 | 77.4 | 80.1 | 74.1 | 77.6 | 59.4 | 64 |  |
| multilingual-e5-base | - | 74.7 | 77.9 | 72.2 | 75.7 | 86.1 | 88.1 | 70.9 | 74.3 | 57.6 | 62.4 | 72.6 | 75.6 |
| QAEemb, $\alpha=0.3$ | 77.4 | 80.7 | 77.1 | 80 | 88.5 | 90.5 | 76.6 | 79.8 | 61.3 | 66 | 77.7 | 80.4 |  |
| multilingual-e5-large | - | 73.8 | 77.5 | 76.6 | 79.9 | 85.7 | 88.1 | 70.4 | 74 | 52.7 | 58.4 | 73.9 | 77.1 |
| QAEhyb, $\alpha=0.15,\;\beta=1.25$ | 77 | 80.4 | 82.1 | 84.9 | 89.1 | 91.1 | 77.2 | 80.4 | 60.3 | 65 | 77.6 | 80.6 |  |
| gte-multilingual-base | - | 65.3 | 69.9 | 73.2 | 76.2 | 82.9 | 85.6 | 62.8 | 67 | 52.1 | 57.8 | 65.9 | 69.3 |
| QAEhyb, $\alpha=0.15,\;\beta=1.5$ | 75.3 | 79 | 76.2 | 79 | 85.5 | 88 | 66.6 | 70.7 | 55.8 | 61.4 | 73.9 | 77.2 |  |
| mcontriever | - | 32.8 | 36.3 | 40.1 | 44.2 | 71.7 | 75.8 | 34.7 | 38.6 | 26.8 | 31.4 | 49.2 | 53.7 |
| QAEhyb, $\alpha=0.45,\;\beta=1.25$ | 61.2 | 65.4 | 68.2 | 71.9 | 88.7 | 90.5 | 64.5 | 68.5 | 50.3 | 55.6 | 69.8 | 73 |  |
| bce-embedding-base-v1 | - | 58.9 | 63.3 | - | - | 76.8 | 80.3 | - | - | - | - | - | - |
| QAEhyb, $\alpha=0.3,\;\beta=0.5$ | 66.6 | 70.6 | - | - | 85.9 | 88.1 | - | - | - | - | - | - |  |
| text2vec-base-multilingual | - | 38.5 | 43 | 27.7 | 31.4 | 12.1 | 13.7 | 33.2 | 37.5 | 15.3 | 18.9 | 12.3 | 14.6 |
| QAEemb, $\alpha=0.75$ | 55.2 | 59.3 | 51.3 | 54.8 | 55.3 | 58.6 | 49.1 | 53.8 | 35.9 | 40.6 | 46.7 | 51 |  |

**Table 7. Complete performance comparison of QAEncoder variants on latest datasets FIGNEWS and CRUD-RAG. Higher is better, the best one is bolded. Hyperparameters are optimized simultaneously across the six latest datasets. $n$ indicates the number of predicted queries in QAnaive.**
| Model | Param | FIGNEWS(English) | FIGNEWS(Arabic) | CRUD-RAG(Chinese) | FIGNEWS(French) | FIGNEWS(Hindi) | FIGNEWS(Hebrew) |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| MRR@8 | NDCG@8 | MRR@8 | NDCG@8 | MRR@8 | NDCG@8 | MRR@8 | NDCG@8 | MRR@8 | NDCG@8 | MRR@8 | NDCG@8 |  |  |
| bge-m3 | QAEemb, $\alpha=0.3$ | 76.2 | 80.0 | 80.0 | 82.6 | 88.7 | 90.6 | 75.0 | 78.5 | 60.9 | 65.9 | 78.9 | 81.7 |
| QAEtxt, $\beta=1.5$ | 77.1 | 80.8 | 80.1 | 82.8 | 89.0 | 90.8 | 76.7 | 79.9 | 62.5 | 67.5 | 79.8 | 82.4 |  |
| QAEhyb, $\alpha=0.15,\;\beta=1.5$ | 77.3 | 80.7 | 80.6 | 83.3 | 89.4 | 91.2 | 76.4 | 79.7 | 61.5 | 66.3 | 79.7 | 82.4 |  |
| $n=10$ | 76.8 | 79.5 | 76.9 | 79.2 | 85.9 | 87.8 | 71.7 | 73.8 | 62.2 | 65.8 | 68.0 | 71.3 |  |
| multilingual-e5-large | QAEemb, $\alpha=0.45$ | 77.8 | 81.1 | 79.7 | 82.7 | 89.7 | 91.4 | 77.0 | 79.9 | 58.2 | 63.1 | 78.0 | 80.9 |
| QAEtxt, $\beta=1.5$ | 75.6 | 79.0 | 80.7 | 83.7 | 88.3 | 90.4 | 76.0 | 79.3 | 59.7 | 64.8 | 77.0 | 79.7 |  |
| QAEhyb, $\alpha=0.15,\;\beta=1.25$ | 77.0 | 80.5 | 82.1 | 84.9 | 89.1 | 91.1 | 77.2 | 80.4 | 60.3 | 65.1 | 77.6 | 80.6 |  |
| $n=10$ | 76.8 | 79.4 | 76.5 | 79.2 | 84.9 | 86.8 | 70.4 | 73.1 | 61.5 | 65.4 | 69.3 | 71.8 |  |

**Table 8. The table illustrates a comprehensive performance comparison of QAEncoder against training-based and document-centric methods on the latest datasets: FIGNEWS and CRUD-RAG. Higher is better, the best one is bolded. Hyperparameters $\alpha,\;\beta$ are optimized simultaneously across the six latest datasets. $n$ denotes the number of pseudo-documents in HyDE. - indicates default or null values.**
| Model | Param | FIGNEWS(English) | FIGNEWS(Arabic) | CRUD-RAG(Chinese) | FIGNEWS(French) | FIGNEWS(Hindi) | FIGNEWS(Hebrew) |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| MRR@8 | NDCG@8 | MRR@8 | NDCG@8 | MRR@8 | NDCG@8 | MRR@8 | NDCG@8 | MRR@8 | NDCG@8 | MRR@8 | NDCG@8 |  |  |
| mContriever-msmarco | - | 65.9 | 70.0 | 70.1 | 73.5 | 85.0 | 87.4 | 66.3 | 70.0 | 48.6 | 53.5 | 69.3 | 72.6 |
| QAEhyb, $\alpha=0.3,\;\beta=0.75$ | 72.1 | 76.3 | 77.2 | 80.2 | 88.6 | 90.6 | 74.2 | 77.7 | 58.7 | 63.5 | 76.3 | 79.4 |  |
| multilingual-e5-large-instruct | - | 66.7 | 70.7 | 74.9 | 78.1 | 79.8 | 82.5 | 65.8 | 70.1 | 48.0 | 53.1 | 69.5 | 72.8 |
| QAEhyb, $\alpha=0.15,\;\beta=1.5$ | 75.5 | 79.4 | 80.7 | 83.5 | 88.9 | 90.6 | 75.7 | 79.2 | 58.9 | 64.1 | 79.3 | 82.1 |  |
| mContriever | - | 32.8 | 36.3 | 40.1 | 44.2 | 71.7 | 75.8 | 34.7 | 38.6 | 26.9 | 31.4 | 49.2 | 53.7 |
| HyDE,$n=8$ | 24.9 | 27.6 | 35.1 | 40.4 | 70.7 | 74.2 | 25.6 | 28.6 | 11.6 | 14.3 | 41.8 | 47.0 |  |
| multilingual-e5-large | - | 73.8 | 77.5 | 76.6 | 79.9 | 85.7 | 88.2 | 70.4 | 74.0 | 52.7 | 58.4 | 73.9 | 77.1 |
| HyDE,$n=8$ | 63.4 | 67.7 | 68.2 | 73.8 | 81.5 | 83.9 | 58.2 | 62.6 | 45.5 | 51.0 | 65.3 | 69.6 |  |

Figure: Figure 6. Ablation on $\alpha$ and $\beta$ hyperparameters for QAEhyb on FIGNEWS(English) dataset.
Refer to caption: x4.png

Figure: Figure 7. Ablation on $\beta$ hyperparameter for QAEtxt on FIGNEWS(English) dataset.
Refer to caption: extracted/5888105/images/Ablation_beta.png

## Peer Reviews

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: Please see weaknesses.
soundness: 3
strengths: 1.	This paper proposes a simple but novel document embedding method to reduce the gap b/w query-document matching, based on a set of queries automatically generated for a given document. It is well-motivated, and it is interesting to see that the simple idea has not been suggested to my best of knowledge, while the work is indeed novel.
2.	The experiments show that the proposed QAencoding leads to improvements over the baseline dense retrieval, while more baselines need to be included.
summary: The paper proposes a simple but novel document encoding method based on a set of these corresponding (automatically generated) queries, QAEncoder, which generates a set of queries first given a document according to 5W1H, and considers the mean embedding of queries as a document embedding. The experiment results show that the retrieval performance is substantially improved.
weaknesses: 1.	The paper assumes Hypothesis, but it is not clear how this hypothesis is connected to the final form of QAencoder. The hypothesis says “the document embedding lies in the perpendicular line intersecting the cluster center”, but it is not clear whether the final embedding vector (Eq. 3 and other equations) fulfills this condition, where lambda is positive. Specially, it seems that the conical assumption hypothesis is not important on this paper, because it doesn’t have a serious role to my understanding. All the methods could be presented, without the conical assumption hypothesis. More explanation is desirable, because this part makes the paper less-readable.  
2.	More baselines need to be provided. In particular, different from the proposed direction, the reverse direction (i.e. query2document style) also needs to be compared. 
3.	Only mean pooling of query embeddings is used. However, taking the mean pooling may make individual query semantics blurred. Advanced pooling or matching methods for query-to-query may be examined and discussed.

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 3
questions: 1. Why not combine $QAE_{emb}$ and $QAE_{txt}$?
2. How can you find the most suitable hyperparameters in different scenarios? Is it cumbersome to rely on repeated searches? Do you have any insights on simpler methods?
3. Maybe the QAEncoder framework can be used in retrieval model's training?
soundness: 3
strengths: 1. Excellent architecture and visualization figures.
2. The main experiment includes many different models and hyperparameters.
3. A series of ablation experiments were conducted.
4. The proposed method is effective and allows pre-computation of the (without query) embedding part, accelerating the inference process.
summary: The paper introduces QAEncoder, a training-free method designed to enhance alignment in RAG systems by addressing the document-query gap. The approach is based on the conical distribution hypothesis, which suggests that potential queries and documents form a cone-like structure in embedding space. QAEncoder estimates the cluster center of potential queries as a surrogate for document embeddings, improving alignment without increasing storage or retrieval latency. The method integrates seamlessly with existing RAG architectures and includes strategies like document fingerprinting to maintain document distinguishability. Extensive experiments demonstrate QAEncoder's effectiveness across multiple languages and datasets.
weaknesses: 1. The descriptions of several QAE fingerprints are somewhat confusing.
2. There are relatively many hyperparameters, and the model's performance varies significantly across different scenarios and hyperparameter settings.

---

confidence: 5
contribution: 2
initial_rating: 5
presentation: 3
questions: 1. Is there any statistics showing pseudo queries generated by GPT-4o-mini for each dataset (total number, number per doc, length, etc.)? If each doc comes with multiple generated queries, it can also be costly.
2. Why was the study not run on more comprehensive benchmarks like BEIR?
3.  Could the authors provide more detail on the computational resources and time required during indexing? Insight into this would clarify its applicability to large-scale datasets.
soundness: 2
strengths: 1. The paper introduces a training-free method to improve query-document alignment, which can be valuable for systems where retraining is impractical. It seems to improve the performance of unsupervised models more, helping the cases that fewer fine-tuning data is available.
2. The authors conduct experiments across various datasets, languages, and embedding models, demonstrating the general applicability of QAEncoder.
3. QAEncoder is designed as a plug-and-play solution, which can be easily integrated into existing systems without significant modifications.
summary: This paper proposes QAEncoder, a training-free method aimed at bridging the semantic gap between user queries and documents in dense retrieval systems for question answering (QA). The authors introduce the conical distribution hypothesis, suggesting that potential queries and documents form a cone-like structure in the embedding space. By estimating the mean of potential query embeddings, QAEncoder generates a surrogate embedding for each document to improve alignment with user queries. Additionally, the paper introduces document fingerprint strategies to maintain document distinguishability. Extensive experiments across multiple datasets, languages, and embedding models demonstrate that QAEncoder enhances retrieval performance without additional training or significant computational overhead.
weaknesses: 1. The performance gain over supervised models is marginal. In certain cases, it seems to worsen the scores on MSMARCO and HotpotQA (shown in the appendix).
2. If I understand correctly, the pseudo-queries are generated based on the corpus used for testing. This is equivalent to domain adaptation, so I think more domain adaptation baselines should be included, such as GPL, Combination of AdaLM and IDF, AugTriever.
3. A basic but reasonable baseline setting is also missing, which fine-tunes each model using the pairs of generated queries and documents. Even though it is fine-tuned and domain-adapted, it saves the cost for serving new documents.
4. Although QAEncoder doesn’t require training, generating multiple queries and embeddings per document could lead to computational strain. A closer look at the resources required would help readers understand if this approach scales well. Need an extra table to show the time of usage.

---

confidence: 4
contribution: 2
initial_rating: 3
presentation: 2
questions: 1. Could the authors please provide more clarity on the concept of the "document-query gap" discussed in this paper and how the proposed method addresses it? Is this gap intended to be reflected through performance improvements, or is it an issue that might not be fully captured by existing benchmarks?

2. I am asking Q1 because I am a bit unclear about the motivation behind this training-free augmentation approach. In what scenario is it necessary, and does it address an issue that benchmarks might not fully capture? While the idea is interesting, the improvement seems somewhat inconsistent and not particularly significant, especially considering the high cost involved (e.g., using GPT-4 to generate a pseudo query for each document).

3. Following Q2, could you provide the number of generated tokens or text chunks required to complete the tasks? Given a similar cost, would it be more effective to label a training set and conduct in-domain training instead?

4. Could you please share the reasoning behind focusing on the cross-lingual scenario rather than comparing the method using other well-established retrieval benchmark?
soundness: 2
strengths: 1. The authors propose several methods for augmenting document embeddings, which could be valuable. Some of their findings are interesting and may provide insights into more effective techniques for representation augmentation.

2. The authors also present various hypotheses and theories, thoughtfully connecting them in a manner that could serve as a useful reference for future research and exploration.
summary: The authors propose QAEncoder, a training-free approach that utilizes a large language model to generate a pseudo query set from each document in the corpus. They introduce various methods to augment the document representations embedded by the retriever (referred to as "document fingerprints" in the paper) based on the generated queries. The approach is evaluated on QA and cross-lingual retrieval benchmarks, comparing the performance of retrieval models augmented with this method to the original ones.
weaknesses: 1. The evaluation process has several issues:

    - The evaluation metric does not align with the well-established metrics used in prior research, such as retrieval accuracy for NQ and SQuAD, or MRR@10 and NDCG@10 for MS MARCO. Changing it to MRR@8 seems unnecessary, and the authors should either justify this choice or also provide results using the well-established metrics.

    - Most of the retrieval baselines are outdated or unsupervised (e.g., Contriever) or are cross-lingual retrievers. It would strengthen the evaluation if more general neural retrieval models, such as DPR, were included. Additionally, some advanced models are evaluated only in their multilingual versions, and it’s not entirely clear how this focus aligns with the authors' overall motivation.

2. The improvements are minor, considering that the benchmarked retrievers are either weak due to limited training data in their multilingual versions or are outdated, unsupervised models. This is particularly concerning given that the authors use the most recent GPT-4-mini for their enhancements. From another perspective, the proposed solution does not appear to be cost-effective.

3. Some concepts are overly wrapped and lack consensus in the field. For example, the "conical distribution hypothesis" occupies a significant amount of space but does not appear relevant or convincing. Although it may not be false, it is not currently reflected or validated during the retriever’s pre-training or tuning process. Instead, the validity of this hypothesis and the proposed solution might heavily depend on the prompt used to generate the pseudo query and how well it aligns with downstream tasks. Additionally, the term "fingerprint" could benefit from further clarification, as it seems to refer to augmenting document embeddings with pseudo queries. If this interpretation is incorrect, additional explanation would help, as the term might be perceived as vague, making the concept harder to grasp. Demonstrating the solution in a simple, direct manner could enhance clarity.