Multi-Field Adaptive Retrieval
Authors: Millicent Li
Tongfei Chen
Benjamin Van Durme
Patrick Xia
Venue: ICLR 2025 Spotlight (2025)
Keywords: information retrieval
hybrid retrievers
semi-structured data
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: We introduce a framework for retrieval over structured documents that adaptively accommodates multiple scorers.

Abstract: Document retrieval for tasks such as search and retrieval-augmented generation typically involves datasets that are _unstructured_: free-form text without explicit internal structure in each document. However, documents can have some structure, containing fields such as an article title, a message body, or an HTML header. To address this gap, we introduce Multi-Field Adaptive Retrieval (mFAR), a flexible framework that accommodates any number and any type of document indices on _semi-structured_ data. Our framework consists of two main steps: (1) the decomposition of an existing document into fields, each indexed independently through dense and lexical methods, and (2) learning a model which adaptively predicts the importance of a field by conditioning on the document query, allowing on-the-fly weighting of the most likely field(s). We find that our approach allows for the optimized use of dense versus lexical representations across field types, significantly improves in document ranking over a number of existing retrievers, and achieves state-of-the-art performance for multi-field structured data.

Published as a conference paper at ICLR 2025
MULTI-FIELD ADAPTIVE RETRIEVAL
Millicent Li Tongfei Chen"; Benjamin Van Durme', Patrick Xia²
Northeastern University, *Augment Code, 'Microsoft 1i mi meleenthhasseernccoo patricck...em//rroofft.coo
ABSTRACT
Document retrieval for tasks such search and typically involves datasets that are unstructured: free-form text without explicit in- ternal structure each document. However, documents can have some structure, containing fields such as an article title. a message body, or an HTML header. To address this gap, we introduce Multi-Fiees Adaptive Retrieval (MFAR), flexible framework that accommodates any number and any type of document indices on semi-structured data. Our framework consists of two main steps: (1) the decom- position of an existing document into fields, each indexed independently through dense and lexical methods, and (2) learning a model which adaptively predicts the importance of field by conditioning weighting of the most likely field(s). We find that our approach allows for the op- timized use of dense versus lexical representations across field types, significantly improves in document ranking over a number of existing retrievers, and achieves state-of-the-an performance for multi-field semi-structured data.
generation
the document query, allowing on-the-fly fly
INTRODUCTION
The task of document retrieval has many traditional applications, like web search or question answer- ing, but there has also been renewed interest as part of LLM workflows, like sttrieval-augmente generation (RAG). An area of study is focused on increasing the complexity and naturalness of queries (Yang et al., 2018 Qi et al. 2019 Jeong et al., 2024 Lin et al., 2023). Another less stud- ied area considers the increased complexity of the documents Jiang et al. 2024 Wu et al. 22024b). This represents a challenge compared to prior datasets for retrieval, like MS MARCO (Nguyen et al.] 2016), which contain chunks of text that highly related the query. Retrieval is done by either searching over the documents via lexical match (Robertson et al. 1994) or with dense retrievers that embed text into vector representations (Karpukhin al. 2020 Ni et al. 2022 Izacard al.) 2022). Relatedly, some approaches (Gao et al. 2021; Chen et al.] 2022) explore the benefits of a hybrid solution, but these options are not mainstream. In this work, we revisit both hybrid models and methods for retrieval of more complex documents.
Our motivation for this direction derives from two observations: 1) documents do have structure: fields like titles, timestamps, headers, authors, etc. and queries can refer directly to this structure; and 2) different scoring method may be beneficial for each of these fields, as not every field is necessary to answer each query. More specifically, our goal is to investigate retrieval semi-structured data. Existing work on retrieval for semi-structured data with dense representations focus on directly em- bedding semi-stture structured knowledge into the model through pretraining approaches (Li et al.) [023 Su et al. 2024). but we would like a method which can more flexibly combine existing pretrained models and scorers. Similarly, there has been prior interest in multi-ficld retrieval, although these works focused on retrieval with solely lexical or sparse features or early neural models (Robertson et al. 1994 Zaragoza al. 2004 Zamani et al., 2018).
In this work, we demonstrate how multi-field documents can be represented through paired views and a per-field basis, with learned mechanism that maps queries weighted combinations of these views. Our method, Multi-Fielk Adaptive Retrieval (MFAR) is a retrieval approach that can accommodate any number of fields and any number of scorers (such as one lexical and one
f Work done while at Microsoft
https://gi thub. cwm.mimmcmmmmmmmmm / www.onncoooooooooooooooooom

---

Published as a conference paper at ICLR 2025
Datasel
Example Query
Example Document
MS MARCO
aleve maximam dose What is Piebaldiom?
You should take one tablet every 8 to 10 hours until symptoms abate.... Pichaldism is rare autosomal dominant disorder of melanocyte develop- characterized by congenital white forclock multiple ...
RioASQ
STaRK- Amazon
Looking, for chess strateyy guide from The House Staunton that offers lics against Old Indian and Modern de- Jenses. Any recommenddation??
Title: Beating the King's Indian
Defense Description: This book also tells you how to play against the Old In- dian and Modern defenses.
Brand: The House of Staunton
Reviews: [[ (reviewerlD: 1234, text:... [reviewerlD:: 1235, text:...s... ...
STaRK
Doet any research from the Indian Maritime University touch upon energy level transitions within the scope of Configuration Interaction?
Thile Radiative transition rates for the forbidden lines in Fe II Abstract: We report electric quadrupole
magnetic dipole transitions among the levels belonging to 3d 6 4s, 3d 7 and 3d 5 4s 2 configurations of Fe in a large scale configuration interaction (CI) calculation.
Authons: N.C. Deb, Hibbert (Indian Maritime University)
STaRK-Prime
What drugs target the CYP344 enzyme
Name: Ivermeetin
and are used to treot strongyloddiavis?
Entiry Type: drug
Details: (Description: Ivermeetin broad-spectrum anti-parasite modi-
cation. It was first marketed under... Half Life: 16 hours) }
TTarget: gene/protein
Indication: For the treatment of intestinal stronggloidiasis due [Cytochrome P-450 CYP3A Inducers, Lactones, ...]
Category:
Figure 1: Traditional documents for retrieval (top), like in MS MARCO (Nguyen al. 2016) and BioASQ (Nentidis et al., 2023), are unsiructured: free-form text that tends to directly answer the queries. Documents in the STaRK datasets (bottom) (Wu et al., 2024b), are semi-structured. each contains multiple fields. The queries require information from some of these fields, it is important to both aggregate evidence across multiple fields while ignoring irrelevant ones.
vector-based) for each field. Additionally, we introduce light-weight component that adaptively weights the most likely fields, conditioned on the query. This allows us to exhaustively include all fields and scorers at inference and let the model determine the relative importance. MFAR obtains significant performance gains over existing state-of-tha -hhee--arre- art baselines. Unlike prior work, our simple approach does not require pretraining and offers some controllability test-time. Concretely, our contributions are:
1. We introduce novel framework for document retrieval, MFAR, that is aimed at semi- structured data with any number of fields. Notably, MFAR is able to mix lexical and vector- based scorers between the query and the document's fields.
2. We find that hybrid mixture scorers performs better than using dense or lexical-based scorers alone; we also find that encoding documents with our multi-field approach can result in better performance than encoding the entire document as a whole. As result, MFAR achieves state-of-the-ee performance on STaRK, dataset for semi-structured doc- ument retrieval.
3. We introduce an adaptive weighting technique that conditions on the query, weighting more the fields most related the query and weighting less the fields that are less important.
4. Finally, we analyze the performance our models trained from our framework; we control the availability of scorers at test-time in an ablation study to measure the importance of the individual fields in the corpus.
2 MULTI-FIELD RETRIEVAL
While semi-structured documents is a broad term more generally, in this work, we focus docu- ments that can be decomposed into fiedds where each field has a name and a value. As an example in Figure 1] for the STaRK-Prime document, Entity Type would be a field name and its value would be "drug." The values themselves can have additional nested structure, like Category has a list of terms as its value. Note that this formulation of semi-structured multi-field document is broad, as it not only includes objects like knowledge base entries, but also free-form text (chat messages, emails) along with their associated metadata (timestamps, sender, etc) and tabular data.
2

---

Published as a conference paper at ICLR 2025
Document D
Query Q
Abstract: "The CH3170H isotopic form of methanol has been investigated as a far-infrared ..."
Are there any publications from Edmonds Community College on Laser pumping methods that explore various emission spectro?
Institution: chMel Jackson": ["Central
Washington University"], "Andrew J. Nichols":
("Edmonds Community College"], "D "D'Artagnonn R.
Womack"; ["Edmonds Community College"]3
Area of Study: ["Wavelengthh "Center
Frequency", "Laser pumping", "Optoelectronics" "Optics", "Physics", "Optical pumping", "Finite impulse response", "Laser"),
abstract stmm (0.5,
document sim; 0.5
institution sim: (0.8, 1.0)
Field weights: [(0.3, 0.2), (0.4, 0.0), (0.0, 0.1)]
Score(Q, D) 0.5
area study sim: (0.5, 0.5)
Score D) - 0.57
Standard Retriever
mFAR (Ours)
Figure 2: Document D and query Q are examples from the STaRK-MAG dataset. Parts of the query (highlighted) correspond with specific fields from D. Traditional retrievers (A) would score the entire document against the query (e.g. through vector similarity). In (B), our method, mFAR, first decomposes D) into fields and scores each field separately against the query using both lexical- and vector-based scorers. This yields pair of field-specific similarity scores, which are combined using our adaptive query conditioning approach to produce a document-level similarity score.
Formally, consider corpus of documents C [d1, d2..... dn} and set of associated fields F (fi. f2. .... fm} that make up each document d, i.e., d = [f xx I E F), where x is the value for that field. Then, given natural- latuuael query q. would like scoring function s(q.d) d) that can be used to rank the documents in such that the most relevant documents to q score highest (or within the top-k). q may ask about values from any subset of fields, either lexically semantically.
2.1 STANDARD RETRIEVER AND CONTRASTIVE LOss
Traditionally, is indexed in its entirety. The retriever can employ either lexical (Robertson et al., 1994) or dense (embedding -based)d)
based) (Lee al., 2019 Karpukhin et al. (0200) scorer. lexical scorer like BM25 (Robertson et al. 1994) directly computes s(q, d) based on term frequencies. For a dense scorer, document and query encoders are used to embed d and q, and simple similarity function, in our case an unnormalized dot product, is used to compute s(g. d).
Document and query encoders can be finetuned by using a contrastive loss (Izacard et al. 2022) which aims to separate positive (relevant) document d against k negative (irrelevant) documents Dj for a given query q. In prior work, shared encoder for the documents and queries trained using this loss, and temperature r is used for training stability:
Le log elqgnd()lT Ed, ED; es(gnd))) 1/T
(1)
Ce is the basic contrastive loss which maximizes P(d q:). Following Henderson al. 2011 Izacard et al.] (2022) and Chen et al. (2020a), we employ in-batch negatives to efficienlly sample those negative documents by treating the other positive documents in the batch (of b documents), dj. as negatives and including them into D,, where, j # and S j S b. Furthermore, following prior work (Yang et al. 2019 Ni et al.) 2022 Chen et al. 2025), we can include a bi-directional loss for P(qi d.). Here, for given positive document d'.. 9j the positive query and the other queries qi.i i become negative queries:
e (gg,d))/
Lb llg e e 1/T > Maa
(2)
3

---

Published as a conference paper at ICLR 2025
The final loss for the (shared) encoder is C Le Lb.
2.2 MFAR: A MULTI-FIELD ADAPTIVE RETRIEVER
Because semi-structured documents can be decomposed into individual fields (d = (xf) we score the query q against each field separately. This score could be computed via lexical or dense (vector- based) methods. This motivates a modification to the standard setup above, where s(q. d) can instead be determined as a weighted combination of field-wise scores and scoring methods,
s(q. d)
Σ Σ FEJ MEM
W th f (q.x))
Here, sf (q.x)) is the score between and field f of using scoring method m E M, and M is the set of scoring methods. For hybrid model, .M- = (lexical, dense). w" is weight, possibly learned, that is associated with field f and scoring method m.
Adaptive field selection. As presented, our method uses weights, wyr that are learned for each field and scorer. This is useful in practice, as not every field in the corpus useful or even asked about, like unrelated numbers internal identifiers. Additionally, queries usually ask about infor- mation contained in small number of fields and these fields change depending on the query.
This motivates conditioning the value of w" also on q so that the weights query by using the query text to determine the most important fields. We use an adaptation function G and let w f Now, the query query-conditioned or adaptive, weighted sum is:
adapt to the given
sq..
Σ Σ G(q.f.m) (q.x)))) JEF MEM
To implement G, let q be dense embedding q, and aM E R be learnable parameters. Then we could define G(q. f m) = a q. We find that learning is more stable with nonlinearity over all fields f and scorers m: G(q. f. m) softmax (amT q)), which is what we use in MFAR.
Multiple scorers and normalization. One objective of ours is seamlessly incorporate scorers using different methods (lexical and dense). However, the distribution of possible scores per scorer can be on different scales. While G can technically learn normalize, we want G focus on query- conditioning. Instead, we experiment with using batch normalization (loffe & Szegedy 2015) per field that whitens the scores and learns new scalars y" and 8 for each field and scorer. Because these scores are ultimately used in the softmax of the contrastive loss, y acts like bias term which modulates the importance of each score while B has no effect.
Note that the score whitening process is not obviously beneficial or necessary, especially if the scorers already share similar distribution (i.e. if we only use dense scorers). We leave the inclusion of normalization as hyperparameter part of our grid search.
Inference At test time, the goal is to rank documents by s(q. d) such that the relevant (gold) documents are highest. Because it can be slow to compute [F[[M]]C] scores for the whole corpus, we use an approximation. We first determine top-k shortlist, Cyy. of documents for each field and
scorer and only compute the full scores for all U UFFF,me M CtT. which results in the final ranking. Note this inexact approximation of the top-k document distinct from traditional late-stage re- ranking methods that rescore the query with each document, which is not the focus of this work.
3 EXPERIMENTS
Our experiments are motivated by the following hypotheses:
1. Taking advantage the multi-field document structure will lead to better accuracy than treating the document in its entirely, single field.
2. Hybrid (a combination of lexical and dense) approaches to modeling will perform better than using only one other.
4

---

Published as a conference paper at ICLR 2025
3.1 DATA
We use STaRK (Wu et al., 2024b), a collection of three retrieval datasets in the domains of prod- uet reviews (Amazon), academic articles (MAG), and biomedical knowledge (Prime), each derived from knowledge graphs. Amazon contains queries and documents from Amazon Product Reviews (He & McAuleyy 2016) and Amazon Question and Answer Data (McAuley al. 2015). MAG con- tains queries and documents about academic papers, sourced from the Microsoft Academic Graph (Wang et al. 2020), obgn-MAG, and obgn-papers 100p 100M (Hu al. 2020). Prime contains queries and documents regarding biomedicine from PrimeKG (Chandak et al.] (022). These datasets are formulated as knowledge graphs in STaRK and are accompanied by complex queries.
In the retrieval baselines (Wu et al. 2024b), node (corresponding to an entity in the knowledge graph) information is linearized into documents that can be encoded and retrieved via dense meth- ods. We likewise treat each node as document. In our work, we preserve each node property or relation as a distinct field for our multi-field models or likewise reformat to a human-readabi doc- ument for single-fiedd models. Compared to Amazon and MAG, we notice that Prime contains a higher number of relation types, i.e. relatively more fields in Prime are derived from knowledge- graph relations than in either Amazon or MAG, where document content is derived from node's properties. In total, there are 22, 8, and 5 fields for Prime, Amazon, and MAG respectively; more details on dataset sizes, preprocessing, exact fields are described in Appendix A
We use t rec_eva| F for evaluation and follow Wu et al. (024b) by reporting Hit@ 1, Recall @ 20, and mean reciprocal rank (MRR). @it@5 is reported in Appendix C.I due space limitations here.
3.2 BASELINES AND PRIOR WORK
We compare primarily to prior work on STaRK, which is set of baselines established by Wu et al. (2024)) and more recent work by Wu et (2024)) Specifically, they include two vector similarity search methods that use OpenAl's -ex.t-embeddi... aoaediing-ddad -002 model, ada-002 and multi-ada- 002. Notably, the latter is also a multi-vector approach, although it only uses two vectors per docu- ment: one to capture node properties and one for relational information. We also include their two LLM-based re-ranking baselines (Claude3 and GPT4 rerankers) on top of ada-002. Although our work does not perform re-ranking, we add these results to show the superiority of finetuning smaller retrievers over using generalist LLMs for reranking.
More recently, AvaTaR (Wu al., 2024a) is an agent-based method which iteratively generates prompts to improve reasoning and scoring of documents. While not comparable with our work, which does not focus on agents nor use models as large, it is the state-of-the-art method for STaRK.
Finally, we use an fff-the-shelf pretrained retrieval encoder, Contriever finetuned on MS MARCO
al. 2022). as baseline for our dense scorer, which we subsequently con- tinue finetuning STaRK. Early experiments showed that Contriever performed better than other dense retrievers. We use BM25 (Robertson et al., 1994 Li 2024) as a lexical baseline. These use the single-field formatting described Section
3.3 EXPERIMENTAL SETUP
MFAR affords combination of lexical and dense scorers across experiments. Similarly to our baselines, we use BM25 as our lexical scorer and the dot product of Contriever embeddings as our dense scorer. We use shared embedding model for both the query and document and for creating q when computing the adaptation function G. Because of potential differences across datasets, we initially consider four configurations that take advantage of MFAR's 's ability to accommodate multiple fields or scorers: MFARDeense uses all fields and the dense scorer, MFARLesicall uses all fields and the lexical scorer, MFARAI uses all fields and both scorers, and MFAR, uses both scorers but the single-field (Sec. B.I document representation. Based on our final results and analysis, we additionally create and evaluate MFARA which consists of both single-document and multi- field representation for both lexical and dense scoring methods. This results in five MFAR R models that use [FF, [F], 2) and 2 + scorers respectively. For each dataset (and across models),
2 htttsa///gituub..cc///mmtttttott _val
httes://huggingff.cc..o/o/oooaap co/
5

---

Published as a conference paper at ICLR 2025
Comparing our method (MFAR) against baselines and state-of-the-art methods the STaRK test sets. ada-002 and multi-ada-002 are based on vector similarity: +CCludee3 GPT4] further adds an LLM reranking step on top of ada-002. AvaTaR is an agent-based iterative frame- work. Contriever-FT is finetuned Contriever model, which is also the encoder finetuned in MFAR. MFAR superior against prior methods and datasets, and earns substantial margin on average across the benchmark. 6 In Wu et al. (2024b), these are reranker models that are only run on random 10% subset of queries.
Table 1:
MAG
Average
Model
1
R@
MRR
H@I
R@200
MRR
H@1 1
R@20
MRR
H@1
R@:0
MRR
ada-002
0.392
0.533
0.542
0.291
0.484
0.386
0.126
0.360
0.214
0.270
0.459
0.381
multi-ada-002
0.401
0.551
0.516
0.259
0.508
0.369
0.151
0.381
0.235
0.270
0.480
0.373
Claude3
0.455
0.538
0.589
0.365
0.484
0.442
0.178
0.356
0.263
0.313
0.459
0.421
GPT4
0.448
0.554
0.557
0.409
0,486
0.490
0.183
0.341
0.266
0.347
0.460
0.465
AvaTaR agent
0.499
0.606
0,587
0.444
0.506
0.512
0,184
0.393
0.267
0.376
0.502
0.455
BM25
0.483
0.584
0,589
0.471
0,689
0.572
0,167
0.410
0.255
0.374
0.561
D.462
Contricver-FT
0.383
0.530
0.497
0.371
0,578
0.475
0.325
0.600
0.427
0.360
0.569
0,467
LeARL
0.332
0.491
0.443
0.657
0.532
0.257
0.347
0.549
MFARDene
0.390
0.555
0,512
0.467
0.669
0.564
0,375
0.698
0.485
0.411
0.641
0.520
MFAR;
0.574
0.663
0.681
0.503
0,721
0.603
0.227
0.495
0.327
0.435
0.626
0.537
MFARAII
0.412
0.585
0.542
0.490
0.717
0.582
0.409
0.683
0.512
0.437
0.662
0.545
0.530
0.663
0.643
0.559
0.741
0.643
0.400
0.726
0.520
0.496
0,710
0.602
run a grid search over learning rates and whether to normalize and select the best model based on the development set.
Because Contriever has 512-token context window, we prioritize maximizing this window size for each field, which ultimately reduces the batch size we can select for each dataset, resulting in 96 for Amazon and Prime, and 192 for MAG. More details on the exact hyperparameters for each run are in Appendix B
4 RESULTS
We report the results from our MFAR models in Table compared against against prior methods and baselines. Our best models- bothlls--oohhhoo make use of both scorers- peee--perfomm significantly better than prior work and baselines: MFAR, on Amazon, and MFAR and MFARAI+2 on the other datasets. This includes surpassing re-ranking based methods and the strongest agentic method, AvaTaR, MFAR performs particularly well on Prime (+20% for H @ )). Comparatively, all models based on ada-002 have extended context windows of 2K tokens, but MFAR, using an encoder that has much smaller context window size (512), still performs significantly better. Furthermore, our gains cannot be only attributed to finetuning or full reliance on lexical scorers since the MFAR models perform better against the already competitive BM25 and finetuned Contriever baselines.
We find that the adoption a hybrid approach benefits recall, which we can attribute to successful integration of BM25's 's scores. Individually, BM25 already achieves higher R@20 than most vector- based methods. The MFAR models retain and further improve on that performance. Recall is especially salient for tasks such RAG where collecting documents in the top-k are more important than surfacing the correct result at the top.
Revisiting our hypotheses from Section we can compare the various configurations of MFAR. Noting that BM25 is akin to a single-field, lexical baseline and Contriever-FT is single-field, dense baseline, we can observe the following:
Multi-field vs. Single-field. A side-by-side comparison of the single-field models against their multi-field counterparts shows mixed results. If we only consider dense scorers, MFARDeene pro- duces the better results than Contriever-FT across all datasets. To our knowledge, this is the first pos- itive evidence in favor multi-fich field methods in dense retrieval. For MFARResiee in both Amazon and MAG, the BM25 baseline performs especially well, and we do not see consistent improvements. This specific phenomenon has been previously noted by Robertson et al.] (2004), who describe BM25F, variant of BM25 that aggregates multi-field information in principled manner.

---

Published as a conference paper at ICLR 2025
Specifically in BM25, the scores are length normalized. For some fields, like institution, repetition does not imply a stronger match, and so treating the institution field separately (and predicting high weights for it) could lead to high scores for negative documents. A multi-field sparse representation, then, may not always be the best solution, depending on the dataset. We further try using BM25F instead but find lackluster performance likely due to undertuned weights (Appendix D). Given the number of fields in STaRK datasets, tuning field weights is a challenging and open problem that appears less tractable for BM25F than for MFAR. Finally, we note that combining multi-field with single-field can lead to further gains, as demonstrated by MFARAII+2 (and Appendix C.2)
Hybrid is best. Across both multi-field (MFARAU vs. Deue or MFAR and single- field models (MFAR2 vs. BM25 or Contriever-FTT) and across almost every dataset, there is an increase in performance when using both scorers over a single scorer type, validating our earlier hypothesis. This reinforces findings from prior work (Gao et al. 2021 Kuzi et al. 2020) that hybrid methods work well. The one exception (Prime, single-field) may be challenging for single-field models, possibly due to the relatively higher number of fields in the dataset and the semantics of the fields, as we investigate more in Section 5.3] However, in the multi-field setting for Prime, we again see hybrid perform best. This provides evidence for our original motivation: that hybrid models are suitable for and positively benefits certain semi-structured, multi-field documents.
ANALYSIS
Next, we take deeper look into why MFAR leads to improvements. We first verify that model is indeed adaptive to the queries by showing that query conditioning is a necessary component of MFAR. Because the field weights are naturally interpretable and controllable, we can manually set the weights to perform a post-hoe analysis of the model, which both shows us which fields of the dataset are important for the given queries and whether the model is benefiting from the dense or lexical scorers, or both, for each field. Finally, we conduct qualitative analysis to posit reasons why MFAR holds an advantage.
Our analyses, along with the quantitative results, lead us to experiment with a combination of single- field and multi-field document representations in Appendix [C22] like MFARAUH2 We find that these combinations offer additional gains over MFARAIl and that even just combination of single-field lexical and multi-field dense improves over only MFARAl or MFAR2.
5.1 Is QUERY-CONDITIONEI ADAPTATION NECESSARY?
We designed MFAR with mechanism for adaptive field selection: for test-time query, the model makes a weighted prediction over the fields to determine which ones are important. In this sec- tion, analyze whether this adaptation is necessary achieve good performance. To do so, MFAR against an ablated version which does not have the ability to predict query-specific weights but can still predict global, field-specific weights by directly learning wT from Equation 3 This allows the model to still emphasize (or de-emphasize) certain fields globally they are deemed important (or unimportant).
Table 2: The test scores of MFARAU without query conditioning (QC) and the % relative change without it.
Amazon
MAG
STaRK Avg.
R@:20
H@
R
R@
MRR
MFARAB
0.412
0.585
0.542
0.490
0.717
0.582
0.409
0.683
0.512
0.437
0.662
0.545
No QC
0.346
0.547
0.473
0.428
0.662
0.528
0.241
0.596
0.368
0.338
0.602
0.456
Loss (%)
-16.0
-6.5
-12.7
-12.7
-7.7
-9.3
-41.1
-12.7
-28.1
-22.6
-9.1
-16.3
In Table 2 we present the details for MFARAn and find that query conditioning is indeed necessary for performance gains across all datasets. Omitting it results in substantial losses on the metrics on each dataset and for the STaRK average. This extends to the other models too. We also find lower scores on STaRK average across the 3 metrics (H@) 1, @@ MRR): -10% -6%, -8% for MFA R Desse and -17% 11%, -14% for the MFAR MFARLesiae Rexical-
7

---

Published as a conference paper at ICLR 2025
5.2 WHICH FIELDS AND SCORERS ARE IMPORTANT?
The interpretable design of our MFAR framework enables to easily control the used fields and scorers after model has been trained. Specifically, we can mask (zero) out any subset of the weights wiy used to compute s(q. d) (Equation 4p. For example, setting n 0 for each ff would force the model to only use the dense scores for each field. We can interpret a drop in performance as a direct result of excluding certain fields scorers, and thus we can measure their contribution (or lack thereof). In this dcep-dive analysis, we re-evaluate MFARAu's 's performance on each dataset after masking out entire scoring methods (lexical dense), specific fields (title, abstract, etc), and even specific field and scoring method (e.g. title with dense scorer).
Scorers We present results on the three STaRK datasets in Table 3 MFARAU on Amazon is heavily reliant on the dense scores, Knowing the results from Table [ this may be unsurprising because MFARe did perform the worst. While the model leans similarly towards dense scores for Prime, on MAG, it relies more on the lexical scores. This shows that each dataset may benefit from different scorer. Further, this may not be expected a priori: we would have expected Prime to benefit most from the lexical scores, as that biomedical dataset contains many initialisms and IDs that are not clearly semantically meaningful. This demonstrates the flexibility and adaptivity of MFAR to multiple scoring strategies.
We see the performance of
From Table 1 we observe that MFARAn outperforms MFARDeese by small margin (0.435 vs. 0.411 for average H@ 1), and so one may suspect MFARAU is heavily relying on the dense scores. However, MFARAIl with w lexiical masked out performs substantially worse on each dataset (Table 0.326 average) than MFAR Dense, suggesting that a nontrivial amount of the performance on MFAR is attributable to lexical scores. Thus, unlike late-stage reranking or routing models for retrieval, the coexistence of dense and lexical scorers (or even individual fields) during training likely influences what the model and encoder learns.
Table 3: Performance of MFARAl with entire scoring methods masked out at test-time.
Amazon
MAG
Prime
Masking
H@1
R@20
MRR
H@
R@20
MRR
H@1
R@20
MRR
None
0.412
0.586
0.542
0.490
0.717
0.582
0.409
0.683
0.512
Dense only: w deeiical = 0
0.389
0.553
0.512
0.257
0.481
0.355
0.331
0.635
0.352
Lexical only: w dndense 0
0.271
0.452
0.386
0.352
0.602
0.446
0.267
0.500
0.442
Fields By performing similar analysis at a fine-grained field-level, we can identify which parts of the document are asked about or useful. For each field fi, we can set wlexical = 0. w w-dense 0, or both. We collect few interesting fields from each dataset into Table with all fields in Appendix
We find that behaviors vary depending on the field. For some fields (MAG's authors, Amazon's title), masking out of the scorers results in almost no change. However, masking out the other one results in a sizeable drop of similar magnitude to masking out both scorers for that field. In this case, one interpretation is that auther (q, d) and glexical (q. d' are not useful within MFARAn-
To simplify the model, one may suggest removing any s (q. d) where setting w" = 0 results in no drop. However, we cannot do this without hurting the model. In other words, low deltas do not signify low importance. For some fields (e.g. Amazon's 's qa, MAG's title, or Prime's phenotype ab- sent), when the lexical or dense scorers are zeroed out individually, the scores are largely unaffected. However, completely removing the field by zeroing both types of scorers results in noticeable drop. In many cases, we observe that masking out entire fields yields a larger drop than masking out either one individually. This type of behavior could be a result of MFAR redundantly obtaining the same similarity information using different scorers. On the contrary, there is also information overlap across fields, and so in some cases, it is possible in some cases to remove entire fields, especially in Prime (e.g. enzyme) and Amazon, without substantial drops.
8

---

Published as a conference paper at ICLR 2025
Table 4: For each dataset, the absolute change (delta) of masking out certain fields and scorers from MFARAul for H@ I and R@@ 20. For each field, we zero out either the lexical scorer. the dense scorer, or both. The raw scores on all metrics for all fields in each dataset are in Appendix
W
lexical 0
by d.dense
Both
Field
H@1
R@20
H@1
@@20
H@
@@20
Amazon
qa
0
0
-0.031
-0.041
title
0.002
-0.003
-0.022
-0.031
-0,023
-0.024
authors
-0.152
-0.117
0
-0.101
-0.086
MAG
title
-0.011
-0,003
-0.017
-0.014
-0.076
0,063
phenotype absent
-0.001
-0.002
0
-0.033
-0.030
Prime
enzyme
0
-0.004
-0.006
Query: Whick gene or protein is not expressed in female gonadal tissue?
2FARg:
MEAAABE:
neme: NUDT
HSP90AB3P
type: gene/protein
lype: gene/protein
expression present: {anatomy: female gonad
expression absent: [anatomy: [cerehellum, female gonad])
Query: Does Arxiv have any research papers from Eckerd College on the neutron scattoring of 6lf in Neutron physies?
MFARLasial
MFA Rpemei
MFARAn:
Abstract: Abstract A pin-
Abstract: Abstract Measurements of neu- tron clastic and inclostic scattering cross sections from 54Fc
Abstract: scattering of 6He from a proton target using a microscopic folding optical po- tential, in which the 6He nucleus described in terms of 4He-core with two additional neu- valence p p-shell. contrasi to the previous work of that nature, all contribations from the interaction the valence neutrons
hole small-angle neutron scattering (SANS) spectrometer. installed cold neutron source of the 20 MW China Mianyang Revearhh (CMRR) in the Institute of Nuclear Physics...
performed for nine incident neutron energies between ...
Citd Papers: Nentron scattering differ- ential cross sections for 23 Na from to 4.5 McV. Neutron inclastic scattering on 54Fc
Authors: Mei Peng (China Academy of Engincering Physics), Guanyun Yan (China Academy of Engineering Physics). Qiang Tian (China Academy Engineering Physics).... ...
Authors: Weppner (Eckerd College). A. Orazbayev (Ohio University). Ch. Elster (Ohio
Area of Study: [Elastic scattering,
Physies, Inclastic scaftering. Neutron, Direct coupling, Atomic physics, Seatter- ing....
University)
[elustic scattering, physics, neutron, atomic physies, scattering)
Study;
Figure 3: Snippets from the highest-scoring document selected by various MFAR. Top: a single- field hybrid model (MFAR2) vs. MFAR MFARAll picks correctly while MFAR, is possibly confused by negation in the query. Bottom: Snippets from configurations of MFAR with access to different scorers Only MFARAll correctly makes use of both lexical and semantic matching across fields.
5.3 QUALITATIVE ANALYSIS
Multi-field gives semantic meaning for choice of field, as compared single-field. In Figure 3 (top), the query is looking for either a gene or protein that is not expressed. With MFARAI- the retriever matches a longer text more accurately than MFAR₂ does. Both MFAR and MFAR, correctly match female gonad. However, MFARAI selects the field that refers to the absence of an expression, which learned by the model. In MFAR, R2, because the lexical scorer cannot distinguish between present and absent, MFAR, incorrectly ranks the negative document higher.
Hybrid excels when both lexical matching and semantic similarity is required. In Figure (bottom), MFARAIl has the advantage over MFAR by having the ability to lexically match Eck- erd College. Furthermore, MFARAI is still able to semantically match the abstract of the document. While MFARDense also finds a close fit, it unable to distinguish this incorrect but similar example from the correct one.
We likewise observe the drawbacks of lexical-only scoring. One limitation of BM25 that the frequency of successive term matching results in increased scores. Because Physics is keyword with high frequency in the authors list, it results in a high score for this document even though is not used in the same sense semantically. On the other hand, MFARAi correctly matches the specific institution because the final scores are based on a weighted combination of lexical and dense scorers, which may reduce or the impact of high lexical scores.

---

Published as a conference paper at ICLR 2025
6 RELATED WORK
Structured and Semi-structured Retrieval Forms of structured and semi-structured retrieval have been explored in variety of tasks and domains. In particular, focus multi-field
trieval, a form of semi-structured retrieval (Zaragoza et al. (2004) for which prior sparse approaches include the aforementioned BM25F (Robertson et al. (2004), learned sparse representations (Zamani et al. 2018), and Bayesian approaches (Piwowarski & Gallinari 2003). Lin et al. (2023) approach similar task using dense retrievers and with primary focus on query decomposition to support a weighted combination of "expert" retrievers. In contrast to their system in which weights are hand- picked and each retriever is independently trained, our MFAR model is learned end-to-end with a single shared encoder and learned weights, which aids scalability.
Table retrieval (Zhang & Balog 2021 BBhaavvtula et al.] 2015) Pasupat & Liang 2015 Herzig al.. 2021] [Shraga et al. Chen et al. (2020b) is structured retrieval task which adopts similar methods as multi-field retrieval (see Appendix F for an evaluation of MFAR for this task). Table retrieval does not necessarily require table-specific model design, as linearized forms of the table can be adequate for competitive performance (Wang et al., (022) and many table retrieval datasets have been seen during encoder pretraining (e.g. DPR (Karpukhin et al.] (2200) has been trained on Wikipedia). Beyond tabular data, other structured retrieval tasks include code search (Husain et al.) 2020) and knowledge graph datasets like shopping (Reddy al. 2022) The latter is similar the Amazon subset of STaRK. Separately, books QA dataset used by Lin ct al. (2023) is multimodal, which is not our focus.
Besides decomposition, which targets parts of the structure, like fields, with specialized parameters, prior work has also investigated modifying the training process through generating pseudo-queries based on Wikipedia formatting (Su et al., (024) and incorporating auxiliary alignment objectives between the document and a natural language description (Li et al. 2023). These methods generally assume that there exist semantically repetitive information (e.g. table and description of table). We do not make this assumption and focus on post-training methods that can use off-the-shelf encoders.
Hybrid Methods The combination of both types of lexical and dense scorers has previously been found be complementary. leading to performance gains (Gao al., [021] Kuzi et al.) 2020 Lee et al.] (023). Notably, Kuzi et al. (2020) points out that long documents are challenging for lexical-based methods and suggests document chunking as a possible remedy in future work. We implicitly segment the document by taking advantage of the multi-field structure inherently present in documents, and unlike those past works, our work the first to demonstrate the strength of hybrid-based methods in multi-field setting. Alternative hybrid retrieval setups combine both dense and lexical features with dense encoder, trained end-to-end (Lin & Lin 2023 2023 whereas we explicitly use existing lexical scorers.
Shen et al.
7 CONCLUSION
We present MFAR, a novel framework for retrieval over multi-field data by using multiple scorers, each independently scoring the query against part (field) of semi-structured document. These scorers can be lexical-based or dense-based, and each field can be scored by both types. We in- troduce an interpretable and controllable query-conditioned predictor of weights used to adaptively sum over these scores. On three large-scale datasets, we find that MFAR can achieve significant performance gains over existing methods due to multi-field advantages and the inclusion of hy- brid combination of scorers, leading to state-of-the-arr performance. Through our analysis, we find that the best models benefit from both access to both scorers and the ability to weight each field conditioned on the query, further verifying our method.
Our primary goal is to study the challenging and emerging problem of retrieval for multi-field semi- structured data and to introduce flexible framework to approach it. Having laid the groundwork, future work can include more specialized individual scorers, scale up to more scorers in other modal- ities like vision or audio, and add other algorithmic improvements to the weighted integration of scores across scorers. Then, MFAR would be a step towards retrieval of any type of content, which can further aid applications for general search agent-based RAG.
10

---

Published as a conference paper at ICLR 2025
8 ACKNOWLEDGEMENTS
The authors thank the reviewers for the engaging exchange and their helpful feedback. The authors also thank Nick Craswell, Jason Eisner, Jenny Liang, Shirley Wu, Jay DeYoung. Somin Wadhwa, and Byron Wallace, for the insightful discussions to improve the paper. ML conducted this work as a research intern in the MSAI group at Microsoft. ML is also supported by an NSF Graduate Research Fellowship.
REFERENCES
Chandra Sekhar Bhagavatula, Thanapon Noraset, and Doug Downey. Tabel: Entity linking in web tables. In The Semantic Web ISWC 2015: 14th International Semantic Web Conference, Beth- lehem, PA, USA, October 11-15, 2015, Proceedings, Part pp. 425-441, Berlin, Heidelherg, doi: 10. 10.1007/998--3331-22507--4 119-111007-665
2015. Springer- Verlag. ISBN 978-3 978-3-319-25006-9 https://dbi.org///0 100710007990
URL
Payal Chandak, Kexin Huang. and Marinka Zitnik. Building knowledge graph to enable precision medicine. bioRviv, 2022. doi: 10.1101/2022.0..01 4899988 URL https://wwww http:/ww..bp...////////////w/.tt
05 469928
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In Hal Daumé III and Aarti Singh (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Pro- ceedings of Machine Learning Research, pp. 1597-1607. PMLR, 13-18 Jul 2020a. URL hktps://proceeste
wwl..press/ve
Tongfei Chen, Ankita Sharma, Adam Pauls, and Benjamin Van Durme. Hierarchical corpus en- coder: Fusing generative retrieval and dense indices, 2025. URL
abs/2502.1881
Xilun Chen, Kushal Lakhotia, Barlas Oguz, Anchit Gupta, Patrick Lewis, Stan Peshterliev, Yashar Mehdad, Sonal Gupta, and Wen-tau Yih. Salient phrase aware dense retrieval: Can a dense retriever imitate sparse one? In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Findings of the Association for Computational Linguistics: EMNLP 2022, pp. 250-262, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics. doi: 10. 10.116653/1122222.nniiggg
URL
Eincings-emnip.11
Zhiyu Chen, Mohamed Trabelsi, Jeff Heflin, Yinan Xu, and Brian D. Davison. Table search using
deep contextualized language model. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '20, pp- 589-598, New York, NY, USA, 2020b. Association for Computing Machinery. ISBN 9781450380164, doi: 10.1145/3397271 0110/33397771341044. 33010444. URL htRLhhtps:://o..
www.oo../000...... 1141111915317111 .ww.3997777111111111 3401044]
Luyu Gao, Zhuyun Dai, Tongfei Chen, Zhen Fan, Benjamin Van Durme, and Jamie Callan. Comple- ment lexical retrieval model with semantic residual embeddings. In Advances in Information Re- trieval: 43rd European Conference on IR Research, ECIR 2021, Virtual Event, March 28 April 1, 2, Proceedings, Part I, pp. 146-166 160, Berlin, Heidelberg, 2021. Springer- Verlag. ISBN 978- 1007/
3-030-72112-4 doi: 10.1007/978-3-000000
URL
978 3-
10.
Ruining He and Julian McAuley. Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. In Proceedings of the 25th International Conference World Wide Web, WWW '16, pp. 507-517, Republic and Canton Geneva, CHE, 2016. In- ternational World Wide Web Conferences Steering Committee. ISBN 9781450341431. doi: 11tp:///0///.11//2222222 10.1145/2872427. 288833377 URL/attps://dl /ttt:s///.o..crr..00111////2//21 228777222222228838388
Matthew Henderson, Rami Al-Rfou, Brian Strope, Yun hsuan Sung. Laszlo Lukacs, Ruiqi Guo, Sanjiv Kumar, Balint Miklos, and Ray Kurzweil. Efficient natural language response suggestion for smart reply, 2017. URL htttpss//arxvv.orr///et/
00652
11

---

Published as a conference paper at ICLR 2025
Jonathan Herzig, Thomas Müller, Syrine Krichene, and Julian Eisenschlos. Open domain question answering over tables via dense retrieval. In Kristina Toutanova, Anna Rumshisky, Luke Zettle- moyer, Dilek Hakkani-Tur: Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou (eds.), Proceedings of the 2021 Conference of the North American Chapter of the As- sociation for Computational Linguistics: Human Language Technologies, pp. 512-519, Online, June 2021. Association for Computational Linguistics. doi: 100.38 1/662//1/2021/ mnacc-main.44
URL https: http//aaaantttt www.aaaaannnoooo...caaao org/ 2021 naacl main. 437
Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Ad- vances in Neural Information Processing Systems, volume 33, pp. 22118-22113 Curran Associates, Inc., 2020. URL paaper/202// file/ http:///claa.o..eeeee//7777/-eo.
files/
Hamel Husain, Ho-Hsiang Wu, Tiferet Gazit, Miltiadis Allamanis, and Mare Brockschmidt. Code- searchnet challenge: Evaluating the state of semantic code search, 2020. URL https:
//azxi..rrg/ab//1130
09436
Sergey loffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Francis R. Bach and David M. Blei (eds.), Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6-11 July 2015, volume 37 of JMLR Workshop and Conference Proceedings, pp. 448-456. JMLR.org, 2015, URL http::/proceete
Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. Unsupervised dense information retrieval with contrastive learn-
ing. Transactions on Machine Learning Research, 2022. ISSN 2835-8856. URL https:
//open/eviww.ntt/// www.eeeeetooommidmoeee.. Kot.ee////ooommma22----.
Soyeong Jeong, Jinheon Baek, Sukmin Cho, Sung Ju Hwang, and Jong Park. Adaptive-RAG: Learning to adapt augmened large language models through question complexity. In Kevin Duh, Helena Gomez, and Steven Bethard (eds.), Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguisties: Human Lan- guage Technologies (Volume 1: Long Pupers), pp. 7036-7050, Mexico City, Mexico, June URL
2024. Association for Computational Linguistics. doi: www.aaaaanlooooooe 389
Zhengbao Jiang, Zhiqing Sun, Weijia Shi, Pedro Rodriguez, Chunting Zhou, Graham Neubig, Xi Lin, Wen-tau Yih, and Srini Iyer. Instruction-tuned language models are better knowl- edge learners. In Lun- Ku, Andre Martins, and Vivek Srikumar (eds.). Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pa- pers). pp. 5421-5434, Bangkok, Thailand, August 2024. Association for LnaputtttionalLinnuu URL httpppss//acclaaccclooooo/////gp
tics. doi: 10. 11116//111222.44/-11a acl-long: 296
John Foley. FastRank alpha release
httpss///5500ley.mm//2////0/0//0
Fastrank - awp.aalppaaa.... html 2019.
Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. Dense passage retrieval for open-domain question answering. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (eds.), Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 3769-6781. Online, Novem- ber 2020. Association for Computational Linguistics. doi: 10. 120.8666/../00000mmmmmmmmmm.mmm URL hwt.hhpps//aaaaaantllloooooo
ort. 2020 wwwnp-mmaiii. 550
Saar Kuzi, Mingyang Zhang, Cheng Ii, Michael Bendersky, and Mare Najork. Leveraging semantic and lexical matching improve the recall of document retrieval systems: A hybrid approach, 2020. WRRLhttps: //wrxv/aaivvv////////a//aaa/aaaa www.aba////aoooaooaa/..../a..aea /0000. 01195
Dohyeon Lee, Seung-won Hwang, Kyungjae Lee, Seungtack Choi, and Sunghyun Park. On com- plementarity objectives for hybrid retrieval. In Anna Rogers, Jordan Boyd-Graber, and Naoaki
12

---

Published as a conference paper at ICLR 2025
Okazaki (eds.), Proceedings of the 61st Anmual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 13357-13368, Toronto, Canada, July 2023, Asso- ciation for Computational Linguistics. doi:
URL hotps:
//acl aalllantholgyy. wrt.gyy.rgg/22.....
7467
Kenton Lee, Ming-Wes Wei Chang, and Kristina Toutanova. Latent retrieval for weakly supervised open domain question answering. Anna Korhonen, David Traum, and Lluís Märquez (eds.), Proceedings of the 57th Annual Meeting of the Association for Computational Linguisties, pp. 6086-6096, Florence, Italy, July 2019. Association for Computational Linguisties. doi: 10.18653/ v1/P19-1612. htthh:t::: ac antho ogy ogt..ogg/..9911150111
Xinze Li, Zhenghao Liu, Chenyan Xiong, Shi Yu, Yu Gu, Zhiyuan Liu, and Ge Yu. Structure aware language model pretraining improves dense retrieval on structured data. In Anna Rogers, Jordan Boyd-Graber. and Naoaki Okazaki (eds.), Findings of the Association for Computational Linguis- fics: ACL 2023, pp. 11560-11574, Toronto, Canada, July 2023. Association for Computational http:///20222.ninnngaa....... findings-acl.734.. URL https: httpp::s/acclantholoogoooo .ogg
Linguistics. doi: 10. 10.11863//112022..1 2023. fiw.finde findings acl. 734
Kevin Lin, Kyle Lo, Joseph Gonzalez, and Dan Klein. Decomposing complex queries for tip- of of-the-tongue retrieval. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the
Association for Computational Linguistics: EMNLP 2023, pp. 5521-5533, Singapore, December 2023. Association for Computational Linguistics. doi: 10. 10.18653/v1/2023. hwtp////2.2.....n---aiaa 2023. findiniggggss ing, April 2023. ISSN 1046-8188. ftt.://ii.......nngnnin.nens emn 3.. 367
mondings-emnll.666)
BRLLhtpps:// ottp:////..aaaaaooooooo.ooo///o.
Sheng-Chich Lin and Jimmy Lin. A dense iepresentaaiooonnaaeeeevooo for lexical and semantic match-
Xing Han Là. Bm25s: Orders of magnitude faster lexical search via eager sparse scoring, 2024.
URL
03618
Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton van den Hengel. Image-based rec- ommendations on styles and substitutes. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '15, pp. 43-52, New York, NY, USA, 2015, Association for Computing Machinery. ISBN 9781450336215. http:///to.org...///// 1tt.:///...1152766414421 7277666666.2777788
doi:
10.11452776466227767555 /RLettps://dbi.oorr
Anastasios Nentidis, Georgios Katsimpras, Anastasia Krithara, Salvador Lima López, Eulália Farré- Maduell, Luis Gasco, Martin Krallinger, and Georgios Paliouras. Overview of BioASQ 2023: The Eleventh BioASQ Challenge on Large-Scale Biomedical Semantic Indexing and Question An- swering, pp. 227-250. Springer Nature Switzerland, 2023, ISBN 9783031424488 doi: 10.1007/ 031 42448
www.:d.ioogg/1111117.7.1111 778-3-031-42448-99 URL L
Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. Ms marco: A human generated machine reading comprehension dataset. Novem- ber 2016. URL https:/ http://wwww.i..ioott/ httpp//.....cceeeeee//////eeaaaa
/
marco human generated machine reading wwwp:y/opppppppooeeeeeeeeeeeeeee ort. htteteeniiiinnntttteeeeaea dataset/
Jianmo Ni, Chen Qu, Jing Lu, Zhuyun Dai, Gustavo Hernandez Abrego, Ji Ma, Vincent Zhao, Yi Luan, Keith Hall, Ming-Wei Chang, and Yinfei Yang. Large dual encoders are generaliz- able retrievers. In Yoav Goldberg, Zoritsaa Kozareva, and Yue Zhang (eds.), Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 9844-9855, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguis- tics. doi: 10. http:3/w1122222.mmlmmmeee emn.p-main. 669
URL http::/ss//aaaaaahhhooooo org/2022..
Panupong Pasupat and Percy Liang. Compositional semantic parsing on semi-structured tables. In Chengqing Zong and Michael Strube (eds.), Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 1470- 1480, Beijing, China. July 2015. Association for Computational Linguistics. doi: 10.3115 1142. URL https:
7/acl 7/aclantholo htwpchooloyy.oroo/15111111
13

---

Published as a conference paper at ICLR 2025
Benjamin Piwowarski and Patrick Gallinari. A machine learning model for information trieval with structured documents. In Petra Perner and Azriel Rosenfeld (eds.), Machine Learning and Data Mining in Pattern Recognition, pp. 425-438, Berlin, Heidelberg, 2003. Springer Berlin Heidelberg. ISBN 978-3-540-45065-88 URL httpts://////t http::::///ii.r/////// orr//10.100/
13-540-45055-333
Peng Qi, Xiaowen Lin, Leo Mehr, Zijian Wang, and Christopher D. Manning. Answering complex open-domain questions through iterative query generation. In Kentaro Inui, Jing Jiang, Vincent Ng. and Xiaojun Wan (eds.), Proceedings of the 2019 Conference on Empirical Methods Nat- ural Language Processing and the 9th International Joint Conference on Natural Language Pro- cessing (EMNLP-IJCNLP)), pp. 2590-2600 Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10.18653/v//D199 1/1111111.1
URL https: httpps:/aalaathhooooy
org/D19-1261
Chandan K. Reddy, Lluís Mårquez, Fran Valero, Nikhil Rao, Hugo Zaragoza, Sambaran Bandy- opadhyay, Arnab Biswas, Anlu Xing, and Karthik Subbian. Shopping queries dataset: A large- scale esci benchmark for improving product search, 2022. 02066065888
Nils Reimers and Iryna Gurevych. Sentence-BERT Sentence embeddings using Siamese BERT- networks. In Kentaro Inui, Jing Jiang, Vincent Ng. and Xiaojun Wan (eds.), Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th In- ternational Joint Conference on Natural Language Processing 3992, Hong Kong, China, November 2019, Association for Computational Linguistics. doi: 10.18653/11/D19-144 URL /ttp:./ http:///l.aatttlll..o/r/////////
pp. 3982-
Stephen Robertson, Hugo Zaragoza, and Michael Taylor. Simple bm25 extension to multiple weighted fields. In Proceedings of the Thirteenth ACM International Conference on Informa- tion and Knowledge Management, CIKM '04, pp. 42-49, New York, NY, USA, 2004. Associ- ation for Computing Machinery. ISBN 1581 org/00 1115/1031171.1003 1115/1031171.103111
doi: 10.1145/1031171 171 103111 URL
Stephen E. Robertson, Steve Walker, Susan Jones, Micheline Hancock- Beanlle,, and Mike Gatford. Okapi at TREC-3. In Donna K. Harman (ed.), Proceedings of The Third Text REtrieval Confer- ence, TREC 1994, Gaithersburg, Maryland, USA, November 2-4, 1994, volume 500-225 of NIST Special Publication, pp. 109-126. National Institute of Standards and Technology (NIST), 1994. URL http::tttp/erree nist gov/ hwtp//pba////ee ttttttrer3////
Tao Shen, Xiubo Geng, Chongyang Tao, Can Xu, Guodong Long, Kai Zhang, and Daxin Jiang. Unifier: A unified retriever for large-scale retrieval. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 223 pp. 4787-4799, New York, NY, USA, 2023. Association for Computing Machinery. ISBN 9798400701030. doi: 10.1145/ https:/://// /ttp///..1115533000055555030
3580305.3599999
Roee Shraga, Haggai Roitman, Guy Feigenblat, and Mustafa Cannim. Web table retrieval using multimodal deep learning. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development
Information Retrieval, SIGIR '20, pp. 1399-1408, New York, NY, USA, 2020. Association for Computing Machinery. ISBN 9781 9781450380164, doi: 10.1 0197221.3401 120. URL httpss:/:/// http:///to.o..///////// 10. 1145/ 00445/33977711 0111771..411222 3401120
Weihang Su, Qingyao Ai, Xiangsheng Li, Jia Chen, Yiqun Liu, Xiaolong Wu, and Shengluan Hou. Wikiformer: pre-training with structured information of wikipedia for ad-hoc retrieval. In Pro- ceedings of the Thirty-Eightt AAAI Conference on Artificial Intelligence and -hirth-Sixth Confer- ence on Innovative Applications of Artificial Intelligence and Fourleenth Symposium Educa- tional Advances in Artificial Intelligence, AAAT24/AAAT44FS
AAAI Press, 2024. ISBN httplaattp://aao 1
978 778-1-57735-888-99 doi: 10. 10.1609/aai..v3811.7 29899. aaai w18.13111177 29869
Kuansan Wang. Zhihong Shen, Chiyuan Huang, Chieh-Han Wu, Yuxiao Dong, and Anshul Kanakia. Microsoft academic graph: When experts are not enough. Quantitative Science Studies, 1(1):396- 413, 02 2020. ISSN 2641-3337. doi: 10.1162/qss a 00021. URL https: /ooi orq/10. 11 62 62266///___000211111__1___000001 000221
14

---

Published as a conference paper at ICLR 2025
Zhiruo Wang, Zhengbao Jiang, Eric Nyberg, and Graham Neubig. Table retrieval may not ne- cessitate table-specific model design. In Wenhu Chen, Xinyun Chen, Zhiyu Chen, Ziyu Yao, Michihiro Yasunaga. Tao Yu, and Rui Zhang (eds.), Proceedings of the Workshop Struc-
tured and Unstructured Knowledge Integration (SUKI), pp- 36-46, Seattle, USA, July 2022.
Association for Computational Linguisties. doi: 10. 10.18653/1122022.sukiz
URL https:
7/aclanthology hww.hhoooggyyggg/22/./22 wwgp///g..2.... suki 1.557
Shirley Wu, Shiyu Zhao, Qian Huang, Kexin Huang, Michihiro Yasunaga Kaidi Cao. Vas- silis loannidis, Karthik Subbian, Jure Leskovec, and James Y Zou. Avatar: Optimiz- ing IIm agents for tool usage via contrastive reasoning. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomezak, and C. Zhang (eds.), Advances in Neu- ral Information Processing Systems, volume 37, pp. 25981-26010. Curran Associates, Inc., httppe/eeeeeeee//////222222 2024a. URL htttps://proceedd.ne hew.seeeddd.nnn..neneoeeee....ee n httpd/esscccpppaerreeeeeeeeeee File/ wawcceescc00eettt11110033333--e3
Shirley Wu, Shiyu Zhao, Michihiro Yasunaga, Kexin Huang, Kaidi Cao, Qian Huang, Vas- silis Ioamnidis, Karthik Subbian, James Y Zou, and Jure Leskovec. Stark: Benchmarking I1 retrieval on textual and relational knowledge bases. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomezak, and C. Zhang (eds.), Advances in Neural Infor- mation Processing Systems, volume 37, pp. 127129-127153, Curran Associates, Inc., 2024b. ngg neu ccp fi Files wwwps//pppeeee/222222222 pptss//par//2///1eeeee Pape-e-appe--ee Datasetss- and Benchharks.
URL
e607b1419e
Track. pdf
Yinfei Yang, Gustavo Hemandez Abrego, Steve Yuan, Mandy Guo, Qinlan Shen, Daniel Cer, Yun- hsuan Sung, Brian Strope, and Ray Kurzweil. Improving multilingual sentence embedding using bi-directional dual encoder with additive margin softmax. In Proceedings of the Twenty-Kighth International Joint Conference on Artificial Intelligence, IICAI-19, pp. 5370-5378. International Joint Conferences on Artificial Intelligence Organizatiion, 7 2019. doi: 10.24963fjccai.20117744 URL https::/doi...oo wwwpo/g......... /ttt::///o...og//.0222/////a.... 2019/745
Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. HotpotQA: dataset for diverse, explainable multi-hop question answering. In Ellen Riloff, David Chiang. Julia Hockenmaier, and Jun' ichi Tsujii (eds.), Proceed- ings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 2369- 2380, Brussels, Belgium, October- November 2018. Association for Computational Linguistics. doi: (0.18653/v1/D18 1259. URL
Hamed Zamani, Bhaskar Mitra, Xia Song, Nick Craswell, and Saurabh Tiwary. Neural ranking models with multiple document fields. In Proceedings of the Eleventh ACM International Con- ference on Web Search and Data Mining, WSDM 18, pp. 700-708, New York, NY, USA, 2018. Association for Computing Machinery. ISBN 4781450355810. doi: 10.1145/3159652.1199777
URL http:::tt///.m.... / / orgg 1eeee1115111555 3119733
Hugo Zaragoza, Nick Craswell, Michael J. Taylor, Suchi Saria, and Stephen E. Robertson. Microsoft cambridge at TREC 13: Web and hard tracks. In Ellen M. Voorhees and Lori P. Buckland (eds.), Proceedings of the Thirteenth Text REtrieval Conference. TREC 2004, Gaithersburg, Maryland, USA, November 16-19. 2004, volume 500-261 of NIST Special Publication. National Institute of Standards and Technology (NIST), 2004. URL http::/ttttttttt ppapersmi m http:///mmmmooooooomaooaooorreee dge web. ha rd.
ni st wtw.gg/pppb//eeeee//
crosoft
Shuo Zhang and Krisztian Balog. Semantic table retrieval using keyword and table queries. ACM Trans. Web, 15(3), May 2021. ISSN 1559- 1131, doi: 10.1145/3441690 URL https: //doi.
org 10. 1 141114454444 34444/444411 690
15

---

Published as a conference paper at ICLR 2025
A DATASET
Table 5: The corpus size, number of fields, and queries (by split) for each of the STaRK datasets.
For field information, refer to Table in the Appendix.
Dataset
Domain
Num. Documents
Num. Fields
Train
Dev.
Test.
Amazon
products, product reviews 950K
8
6K
1.5K
1.5K
MAG
science papers, authors biomedical entities
700K
8K
2.6K
2.6K
Prime
130K
22
6.1K
2.2K
2.8K
A.1 PREPROCESSING
Technically, STaRK is a dataset of queries over knowledge graphs. The full dataset details are in Table 5 The basclines (Wu et al., 2024b) create a linearized document for each node, which omits some edge and multi-hop information that is available the knowledge graph. AvaTaR (Wu et al. 2024a) operates directly the knowledge graph. As want to operate over semi-structured documents, we need a preprocessing step either on the linearized documents or by processing the graph.
Because parsing documents is error-prone, we decide to reproduce the document creation process from (Wu et al.) 20246). We start with all of the original dataset from STaRK, which come in the form of queries and the associated answer ids in the knowledge graph. Each query requires combination of entity information and relation information from their dataset to answer. However, each dataset handles the entity types differently. The answer to every query for Amazon is the product entity. For MAG, the answer the paper entity. However, Prime has list of possible entities that can answer the query, we include all ten as documents.
We create our set of documents based on the directional paths taken their knowledge graph; there are more than single hop relations, then we take at most two hops for additional entities and relations. For Amazon, since the queries are at most one hop. we do not include additional node information. MAG and Prime, however, include more queries with more than two hops, include information about additional relations and nodes for each document in our dataset.
A.2 FIELDS
We include the list of fields that we used in this work in Table 6 Not every single field available in the STaRK knowledge graph (Wu et al. 2024b) is used because some are not used in the baseline and we try match the baselines as closely as possible. We make some cosmetic changes for space and clarity in the examples in the main body of this paper, including uppercasing field names and replacing underscore with spaces. We also shorten per_..cites__ppppp to "Papers Cited", has.to... expand "type" to "Entity Type."
wwt.eedw_tt......ttttiito. "pa- to "Area of Study" and
Table 6 also lists some information about the length distribution of each field, measured by the Contriever tokenizer. This is useful to know how much information might be lost to the limited window size of Contriever. Furthermore,
list the maximum sequence length used by the dense scorer of MFAR both during training and at test-time. The trade off for sequence length is batch size with respect to GPU memory usage. Our lexical baseline (BM25) does not perform any truncation.
B IMPLEMENTATION DETAILS
During training, we sample & negative example per query. Along with in-batch negatives, this results in 2b negative samples for a batch size of b. This negative document is sampled using Pyserini Lucend
4 100 nearest documents are retrieved, of which the postive documents are removed. The top negative document is then sampled among that remaining set. We apply early
https://giitie wttp::twww...oottttoooii////aaae
16

---

Published as a conference paper at ICLR 2025
Table 6: 'he datasets and list of fields, F, used in this work, along with basic length statisties of the content of those fields. The length of the kth-%ile longest value is listed. For example, k = 50 would be the median length. The MSL is the maximum sequence length threshold we chose for that field in MFAR based either the maximum window size of the encoder (512) or based on covering most (> 99%) documents within the corpus..
Length in Contriever tokens (A (--le)
Dataset
Field
90
95
99
99.9
Max
MSL
also buy
64
217
908
3864
50176
512
also view brand description
86
189
557
1808
21888
512
10
35
207
289
446
1020
5038
512
Amazon
feature
130
171
305
566
1587
512
qa review title
4
5
698
1873
512
1123
2593
12066
58946
630546
512
28
48
75
918
128
abstract
354
410
546
775
2329
512
author aafllraatttiiiiieeiiiii institttttitnnnitttttttnnnn
90
121
341
18908
46791
512
MAG
paper piper__cttes__pppppp
581
863
1785
4412
79414
512
paper has of title
49
52
57
63
90
64
31
34
44
62
9934
64
associated with carrier
10
35
173
706
4985
256
3
4
13
2140
8
contraindication
66
586
3481
128
details
329
823
2446
5005
12319
512
enzyme
4
12
63
5318
64
expression absent expression present indication
4
29
77
12196
64
204
510
670
18306
81931
512
4
25
146
1202
32
interacts with linked to
93
169
446
1324
55110
512
3
4
57
544
8
17
21
38
74
133
64
Prime
off-label use parent-child
4
56
727
49
70
168
714
18585
256
phenotype absent phenotype present ppi
3
4
4
33
1057
8
20
82
372
1931
28920
512
36
438
1563
22432
512
side effect
93
968
5279
128
source
5
6
6
7
8
8
synergistic interaction
4
4
4800
9495
13570
512
larget
4
33
312
5852
64
transporter
4
41
2721
type
7
8
9
9
8
stopping validation loss with patience of 5. We set 0.05 and train with DDP 8x NVIDIA A A100s. Contriever is 11M parameter model, and the additional parameters added through G is negligible F7)8) scaling linearly in the number of fields.
We use separate learning rates (LRs) for finetuning the encoder and for the other parameters. Specif- ically, we searched over learning rates [5e-6, 1e-5, 5e-5. 5e-2, [e-1] for the parameters in G G(q. f.m) which consist of x and y" and BT from batch normal- ization. The main grid search was conducted over the bolded values, although we found 5e-3 to be effective for G(g.f. m) for Amazon. We otherwise follow the default settings for both the optimizer (AdamW, W. dropout, etc.) and batch normalization (PyTorch 2.4.0). As mentioned, whether to apply batch normalization at all was also a hyperparameter searched over: we found it useful in the hybrid setting.
lc-4] for the encoder and [le-3, 5e-3, 1e-2,
17

---

Published as a conference paper at ICLR 2025
Our implementation uses Pytorch Lightning and serience-tzansforree 2.2.2 (Reimers & Gurevych 2019). We use fast, wython-basee implementation of BM25 as our lexical scorer (Lü, 2024 The best hyperparameters for each of models in this work are listed in Table 7 In the case where there is only a single field (last two sections), the adaptive query conditioning is not needed.
At inference, we retrieve the top- 10p results per field to form a candidate set, and we compute the full scores this candidate obtain our final ranking.
Table 7: The hyperparameters used for each of the runs in this work.
Model
Dataset
Mainly referenced in
Encoder LR
GQ LR
Batch norm?
Amazon
le-5
5e-3
MFAR
MAG
Table
most tables/figures
5c-5
le-2
yes
Prime
5c-5
le-2
yes
Amazon
Ic-5
5c-3
MFA R Dense
MAG
Table
Figure
5c-5
5e-2
no
Prime
lc-5
le-2
no
Amazon
lc-5
5e-3
yes
MFAR Lexical
MAG
Table
Figure
le-5
le-2
yes
Prime
5e-5
le-1
yes
Amazon
le-5
le-2
no
MFAR
MAG
Table Figure
5e-5
5e-3
yes
Prime
5e-5
5e-3
yes
Amazon
le-5
le-2
Densee&
MAG
Table (ppenddix only)
5e-5
Se-3
yes
Prime
lc-5
5e-3
yes
Amazon
5e-5
Contriever-FT
MAG
Table
le-5
n/a
n/a
Prime
5c-5
n/a
n/a
C DETAILED RESULTS ON STARK
We present comprehensive results the test split of Amazon, MAG, and Prime.
C.I FULL TEST RESULTS AND COMPARISON
Here, we report the same results in the main section, but we also include H@5, to be exhaustive with STaRK, in addition to our existing metrics. In Table g we show the test results with the included additional metric. In Table we also include the average
separate table. Here, we find that MFAR still does on average better than the other baselines on the semi-structured datasets, even against the strong lexical BM25 baseline.
C.2 MERGING FULL WITH PER-FIELL REPRESENTATION
We evaluate additional models that make combine both multi-field (IFI scorers) with a single-field (one concatenated document) document representation for either both lexical and dense retrievers. For example, BM25 could be interpreted as a lexical retriever over a (single-field) full-document text which concatenates all the fields. We extend our earlier experiments with additional experiments: MFAR MFAR1. Lexical++ is MFAR MFARLest Rexical with additional scorer (field) that scores the full document by using the single-document BM25 score; MFARDeese+ MFA R Dense with an additional scorer that embeds and scores the full document (using finetuned Contriever); MFARAIH22 is hybrid model, like MFARAn. which combines MFAR Reesicall and MFARDense4), i.c. this contains the most scorers.
https://Lightningg https://github: thub. com/xh xh /bm25s
18

---

Published as a conference paper at ICLR 2025
Table 8: Similar Table instead include H@5 and show the average a separate table, the test split. We include the same baselines and generally find that H@5 also follows the same trends. The average over all datasets can be seen in Table 9
Amazon
MAG
Prime
Model
H@S
R@
H@5
ada-002
0.392
0.627
0.533
0.291
0.496
0.484
0.386
0.126
0.315
0.360
0.214
multi-ada-022
0.401
0.650
0.551
0.516
0.259
0.504
0.508
0.369
0.151
0.336
0.381
0.235
Claude3
0.455
0.711
0.538
0.559
0.365
0.532
0.484
0.442
0.178
0.369
0.356
0.263
GPT4
0.448
0.712
0.554
0.557
0.409
0.582
0.486
0,490
0.183
0.373
0.341
0.266
BM25
0.483
0.721
0.584
0.589
0.471
0.693
0.689
0,572
0.167
0.355
0.410
0.255
Contriever-FF
0.325
AvaTaR (agent)
0.499
0.692
0.606
0.587
0.444
0.567
0.506
0.512
0.184
0.367
0.393
0.267
MFARLesical
0.332
0.569
0.491
0.443
0.429
0.634
0.657
0.522
0.257
0.455
0.500
0.347
MFARDene
0.390
0.639
0.555
0.512
0.467
0.678
0.669
0.564
0.375
0.620
0.698
0.485
MFAR:
0.574
0.814
0.663
0.717
0.721
0.603
0.227
0.439
0.495
0.327
MFARAI
0.412
0.700
0.585
0.542
0.490
0.696
0.717
0.582
0.409
0.628
0.683
0.512
MFARAD0
0.659
Table The averages for Table g
Averages
Model
Hθ!
R@
MRR
ada-
0.270
0.479
0.459
0.381
Claude3*
0.333
0.537
0.459
0.421
0.460
0.465
BM25
0.374
0.590
0.561
0.462
Contriever-FT
0.360
0.594
0,569
0.467
AvaTaR (agent)
0.376
0.542
0,502
0.455
ME'..essical
MFAR
0.411
0.652
0.641
0,520
MFAR;
0.435
0.656
0.626
0.537
MFARAI
0.437
0.675
0.662
0.545
MFAR A1122
0.496
0.729
0.710
0.602
Finally, MFARDenst&I is hybrid model that combines MFARDense with additional BM25 score over the full single-documee representation. We separately experiment with this combination due the promising results from the BM25 baseline (compared MFAR over certain datasets, the relative strength of MFAR Dense compared to MFAR All- and our qualitative analysis.
Each of these models re-used the hyperparameters from their corresponding base version. The one exception is MFAR Dense&1, for which we performed an additional grid search like the earlier models there is no clear corresponding base model.
Comparing these scores against the base model scores (from Table I we find that there is consid- erable benefit to including single-document representation in addition the multi-field one.
C.3 VARIANCE ACROSS RANDOM SEEDS
For each trained model in Table 1 we select 3 additional random seeds and retrain those models to establish the variation of the scores across runs. Note that none of these seeds are the same as the used in main experiments. Notably, MFARDense1 outperforms MFARAU despite having fewer scorers ([Fl 1 vs. 2(Fl)
D BM25F
BM25F is included as a point of comparison in our work, but there exists no public implementation in Python, we manually implement BM255 using existing codebase We do not exhaustively
https::/ thw.. com/ jommmjxxrriiio sttriis11/bboi s12/bm25_pt
19

---

Published as a conference paper at ICLR 2025
Table 10: Scores of models that consider combination of single-field and multi-field document representations.
Amazon
MAG
Prime
Model
H@1
H@5
R@
MRR
H@8
H@5
R@
MRR
H@1
H@B
R
MRR
MFAR
0.332
0.569
0.491
0.443
0.429
0.634
0.657
0.522
0.257
0.455
0.500
0.347
0,471
0.728
0.605
0.588
0.470
0.675
0.690
0.564
0.271
0.479
0.541
0.367
MFA MFARDene RDene
0.390
0.659
0.555
0.512
0.467
0.678
0.669
0.564
0.375
0.620
0.698
0.485
MFARRneee)
0.453
0.594
0.576
0.384
0.636
0.707
MFARAll
0.412
0.700
0.585
0.490
0.696
0.717
0.582
0.409
0.628
0.683
0.512
MFARDmsek1
0.530
0.785
0.663
0.643
0.559
0.742
0.741
0.643
0.400
0.659
0.726
0.520
MFARAI1+
0.562
0.808
0.672
0.674
0.507
0.721
0.717
0.605
0.342
0.615
0.669
0.464
Table 11: Standard Deviation of each metric for each dataset and model. These are typically between O to 0.015, which gives a sense of how significant differences between models are.
Amazon
MAG
Prime
Model
H@!
H@5
R@20
MRR
H@-
R@:20
MRR
H@1
H@5
R@20
MRR
MFABLasiaa
0.004
0.008
0.008
0.016
0.013
MFA Reese
0.007
0.004
0.004
0.003
0.008
0.005
0.001
0.001
0.005
0.003
0.004
0.002
MFARg
0.004
0.003
0.004
0.003
0.010
0.007
0.007
0.001
0.010
0.014
0.012
0.002
MPARAl RAIl
0.001
0.002
0.002
0.001
0.026
0.020
0.020
0.007
0.014
0.011
0.011
0.012
MFA AFFRDesss
0.001
0.002
0.001
0.001
0.011
0.009
0.009
0.002
0.004
0.002
0.002
0.003
MFARAR
0.005
search across weights to set, as it would require as many as 2| independent parameter searches (Zaragoza al., 2004). As result, an exhaustive BM25F baseline with optimal weights difficult without large amounts of compute") whereas MFAR does not require such grid search that scales the number fields of the dataset.
Below, we present BM25F with uniform weights for each field, setting all weights to 1, compared to regular BM25 and MFARAn- Though there may be more optimal settings, the weight selection entirely dataset-dependent.
Table 12: A uniformly-weighted BM255 against BM25 and MFARAI-
Amazon
MAG
Prime
Model
H@
2@2
MRR
H@
@020
MRR
H@
R
MRR
BM25
0.483
0.584
0.589
0.471
0.689
0.472
0.167
0.410
0.255
BM25F
0.183
0.332
0.264
0.451
0.671
0.551
0.142
0.244
0,214
MFARAUI
0.412
0.585
0.542
0.490
0.717
0.582
0.409
0.683
0.512
We find that using uniform weights, BM25F performs even worse than BM25. This highlights the importance of choosing appropriate weights, which is nontrivial. In contrast, the relative importance (or weights) assigned to each field is learned in MFAR
Thhere exist methods such RankLib (John Foley 2019). find that this still requires large amount compute
learning-to-rank algorithm, and coordinate
ascent, that can also used for searching weights, to fully realize. For each query and document pair, one must generate a set of features to be used for RankLib, These features scale with the number of documents, Therefore, if the combination of queries and documents is large, then generating all possible features may become intractable. Additionally, if one chooses add more samples, it nontrivial then use Ranklit Lib again (one would have search again from scratch).
20

---

Published as a conference paper at ICLR 2025
E FULL RESULTS FOR FIELD MASKING
We include full scores for masking each field and scorer for Amazon in Table [3] MAG in Table 14 and Prime in Table [5 The first row "-" is MFAR All without any masking and repeated three times as a reference. The final row "all" is the result of masking out all the lexical scores (or all the dense scores). It does not make sense mask out all scores, as that would result in no scorer.
Based on our findings in Table 14, all fields in MAG are generally useful, as all instances of zeroing the respect fields results in performance drop. Despite this finding with MAG, not all fields are as obviously important in other datasets. For Table L5| Prime has a notable number of fields that do not contribute the final ranking when both scorers are masked out. And for Amazon, Table [3] we surprisingly find that fields like "description" and "brand" have little effect. This is reflection on both the dataset (and any redundancies contained within) and on the distribution of queries and what they ask about.
Table 13: Test scores Amazon after masking out each field and scorer of the MFARAI at test-time.
Amazon
Jesicall =0
deseese
Both
Masked field
H@1
H@S
R@ 20
MRR
H@1
H@S
R@
MRR
H@1
H@S
R
MRR
0.412
0.700
0.586
0.542
0.412
0.700
0.586
0.542
0.412
0.700
0.586
0.542
also_buy
0,407
0.690
0J578
0.534
0.410
0.696
0.586
0.540
0.403
0.678
0.578
0.530
vvew
0.420
0.695
0.576
0.542
0.414
0.696
0.581
0.542
0.395
0.677
0.565
0.522
brand
0.410
0.699
0.585
0.540
0.397
0.692
0.575
0.528
0.400
0.686
0.570
0.526
description
0.417
0.699
0.587
0.542
0.410
0.692
0.580
0,540
0.413
0.680
0.576
0.535
feature
0.412
0.700
0.581
0.537
0.398
0.680
0.570
0.524
0.410
0.562
qa
0.412
0,700
0.586
0,542
0.412
0.700
0.586
0.542
0.381
0.636
0.545
0.499
review
0,410
0.696
0.583
0.541
0.398
0.680
0.575
0,526
0,384
0.666
0.548
0.510
tille
0.414
0.685
0.583
0.535
0.390
0.650
0.555
0.508
0.389
0.672
0.562
0.516
all
0.389
0.660
0.553
0.271
0.518
0.452
0.386
Table 14: Test scores on MAG after masking out each field and scorer of the MFARAI at test-time. Due to space, we truncate some field names, refer to Table 6 for the full names.
MAG
drme
Both
Masked field
H@1 1
H@S
R 20
MRR
H@1
H@B5
20
MRR
H@1
H@S
R
MRR
0,490
0.696
0.717
0.582
0.490
0.696
0.717
0,582
0.490
0.696
0.717
0.582
abstract
0.469
0.681
0.707
0.565
0.393
0.616
0.651
0.494
0.430
0.636
0.659
0.526
author aflil...
0,338
0.555
0.600
0.439
0.490
0.696
0.717
0,582
0.389
0.595
0.631
0.485
paper cites...
0.458
0.660
0.655
0.685
0.576
0.424
0.650
0.668
paper topic...
0.459
0.671
0.695
0.554
0.491
0.695
0.717
0,582
0,398
0.617
0.650
0.499
0.479
0.686
0.714
0.676
0.565
0.414
0.633
0.654
all
0.257
0.462
0.481
0.355
0.352
0.561
0.602
0.446
F TABLE RETRIEVAL
we demonstrate MFAR on table retrieval, specifically on NQ-Tables (Herzig et al. 2021 which consists of 170K tables along with their titles. We note the dataset has generally short inputs (with limited decomposition of fields into title, column headers, and table content), where fine-tuned full-context models may excel over MFAR. The dataset is sourced from Wikipedia, which contains knowledge seen in pretraining data. Finally, we did not sweep hyperparameters instead re-using those from earlier. We compare to DPR-table a model of similar size finetuned over tables (Wang) al. 2022). DPR-table outperforms MFAR by at large margin at Hit @ 1. However, we find that MFAR has improved recall over the table retrieval model when considering top- 10 or top-20 results. This shows that even in mismatched tasks (where there few fields and a competitive baseline designed for those fields), MFAR can show promise.
In Table 16
21

---

Published as a conference paper at ICLR 2025
Table 15: Test scores on Prime after masking out each field and scorer of the AF at test-time. Due to space, we shorten some field names, so refer to Table for the full names.
Prime
welsiical
=
wedeass 0
Both
Masked field
H@1
H@s
R@
MRR
H@1
H@5
R
MRR
H@1
H@S
R@
MRR
0.409
0.627
0.683
0.512
0.409
0.627
0.683
0,512
0.409
0.627
0.683
0.512
associated with
0.392
0.610
0.670
0.495
0.407
0.624
0.680
0,510
0.799
0.618
0.672
0.502
carrier
0.409
0.627
0,512
0.409
0.627
0.683
0.403
0.621
0.678
0,506
contraindication
0.409
0.627
0.683
0.512
0.409
0.627
0.683
0,512
0.380
0.587
0.652
0.479
details
0.386
0.606
0.670
0,488
0.363
0.569
0.619
0,458
0.388
0.601
0.661
0,489
enzyme
0.409
0.627
0.683
0.512
0.409
0.627
0.683
0.512
0.405
0.623
0.677
0,508
expression abs.
0.408
0.627
0.683
0
0.392
0.607
0.664
0.494
0.400
0.622
0.678
0,506
expression pres.
0.512
0.409
0.683
0,502
indication
0.407
0.627
0.682
0.511
0.398
0.613
0.663
0.498
0.392
0.611
0.665
0.495
interacts
0.403
0.624
0.681
0.507
0.406
0.626
0.682
0.510
0.403
0.622
0.674
0,506
linked
0.409
0.627
0,683
0.512
0.409
0.627
0.683
0.512
0.383
0.601
0.661
0.486
name
0.410
0.628
0.684
0.513
0.407
0.627
0.681
0.510
0.407
0.622
0.674
0.507
off label use
0.409
0.627
0.683
0.512
0.409
0.627
0.683
0.512
0.379
0.602
0.662
0,482
purent-child
0.385
0.619
0.680
0.494
0.391
0.613
0.663
0.495
0.386
0.601
0.663
0.487
phenotype abs.
0.511
0.409
0,477
phenotype
0.405
0.619
0.675
0,506
0.409
0.627
0.683
0,512
0.393
0.609
0.669
0.495
ppi
0.403
0.622
0.678
0,506
0.409
0.627
0.683
0.512
0.399
0.617
0.671
0,502
side effect
0.409
0.627
0.683
U.512
0.409
0.627
0.683
0.512
0.405
0.624
0.680
0.508
source
0.409
0.627
0.683
0.512
0.409
0.627
0.683
0.512
0.397
0.614
0.671
0.499
synergistic int.
0.408
0.627
0.682
0.511
0.409
0.627
0.683
0.512
0.381
0.597
0.659
0,483
urgel
0.407
0.627
0.683
0,511
0.394
0.613
0.662
0.497
0.397
0.617
0.671
0,501
transpotter
0.512
0.409
0.681
0.406
0.624
0.509
0.409
0.627
0.683
0,512
0.403
0.625
0.681
0,507
0.396
0.615
0.669
0,498
0.342
0.554
0.624
0.442
0.267
0.450
0.500
0.352
Table 16: Table retrieval results on NQ-Tables (Herzig et al. only one gold document per query.
2021
We report recall, since there
Model
R@1 1
R@5
R@ 10
R@20
MFAR,
0.497
0.812
0.878
0.930
MFARAI
0.498
0.829
0.900
0.949
DPR table 110M
0.679
0.849
0.889
0.906
22


Reviews:
[{"review_id": "hFW2zRBanG", "reviewer": "Reviewer_ZnRT", "summary": "The paper is concerned with the retrieval of documents which are multi-field, i.e, composed of multiple attributes such as title, authors and content. It proposes a method called MFAR which combines the use of different scoring methods, both lexical (word-based) and dense (embedding-based), for each field, allowing the model to adaptively predict the importance of each field for a given query. \n\nThe authors conduct experiments on three datasets (product reviews, scientific papers and biomedical studies) that demonstrate that MFAR outperforms existing retrieval methods, achieving state-of-the-art results in structured data information retrieval. The study explores the benefits of the multi-field approach and the hybrid use of scoring methods to improve retrieval accuracy, showing that, instead of choosing between dense or lexical-based scorers alone, one can combine them in a hybrid fashion. An adaptive weighting technique is provided to combine those scores given a query.", "questions": "Are the scoring dimensions really orthogonal, enabling summation as the summarization metric?\n\nAren´t there additional baselines and data sets that may be used in the experiments?", "limitations": "", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper tackles a relevant problem, although the proposal has some limitations discussed below. The paper is well written and easy to understand. Originality is rather limited, since the semi-structured retrieval has been researched for a long time. The significance of the results is also limited because the approach is rather simple, and the experimental evaluation focuses on a recently published benchmark. The paper quality is good to its purposes, despite some adhoc design decisions.", "weaknesses": "It is not clear to me whether the fields may be considered indepent, so that the summation of the field-related scores suffices for determining the overall score. That is, it seems intuitive that there are correlations among the field instances and they may bias the result, as was extensively researched in the information retrieval area.\n\nAnother field-related issue is regarding the process of selecting the fields that will be considered in the whole process.\n\nOverall, the proposal is simple and basically consists of combining scoring functions associated with fields, without considering their correlations and other characteristics that may either characterize the task or explain problems or failures. For instance, although the paper focuses on the information carried by the fields, it seems intuitive to mix the aggregated value of the fields with the remaining text, exploiting eventual information there. \n\nThe experimental result also needs to be improved, as detailed next. First of all, the two experimental hypothesis seem to be too simple, thus quite easy to demonstrate. The advantages of using document structure are expected, in particular considering the additional information given to the models. The expected gains of hybrid approaches are also quite predictable. In both cases, it would be interesting to somehow derive upper bounds on the gains, so that the results go beyond benchmark-based evidence. \n\nOn the other hand, it is also necessary to clear outline the limitations of such hybrid approaches and how they may be addressed. I really missed confidence intervals or similar statistical significance metrics on the results. For instance, on Table 2, the results may be too close and, despite the bold highlight, it is not clear how far the results are from the remaining of the baselines.\n\nOne terminology issue is regarding the difference between structured and semi-structured and where the paper fits. It seems to me that semi-structured is the proper jargon.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yqtDh5HEvx", "reviewer": "Reviewer_EjWS", "summary": "The paper presents a framework to improve document retrieval (Multi-Field Adaptive Retrieval, MFAR). Whereas document retrieval typically uses unstructured data for tasks such as retrieval augmented generation, MFAR is designed for structured data where documents have multiple fields (e.g., titles, abstracts, author information). MFAR first decomposes documents into individual fields and then learns a model that adaptively weights fields based on the input query. The framework uses both dense and lexical methods for retrieval, optimizing the combination of these representations per field to improve retrieval performance. Experimens show that MFAR outperforms previous retrieval models in complex document retrieval on the STaRK dataset.", "questions": "How much robust is the framework to variations in the number of fields, e.g., regarding field information overlap?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Rereach on structured document retrieval is highly relevant, especially for RAG approaches. The retrieval is well designed, using a hybrid and adpative query scoring mechanism, using both dense and lexical methods as well as a ranking strategy. The evaluation is thorough, and the paper is well-structured and generally well-written.", "weaknesses": "The fine-tuning approach makes the approach specific to a set of fields from a dataset. Information overlap in fields (see lines 416-424) might intrudice some redundancy to the retrieval process.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zbetcN8AJW", "reviewer": "Reviewer_JpGq", "summary": "This paper exploits the structure in various IR corpora by learning query, corpus, field dependent weights across vector retrieval and token retrieval. The paper compares the methods to a number of strong baselines and analyzes a thorough set of ablation experiments to identify difference makers across different benchmarks. This paper is well written, easy to read and has a comprehensive set of references.", "questions": "1. Are \"Dense only\" in Table 4 and \"mFAR_dense\" in Table 2 the same (the numbers are close but different). Were mFAR_dense and mFAR_lexical trained separately or trained together and one component ablated? \n\n2. See the weakness section, which I phrased as a question.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The paper is well motivated and well written. Many documents naturally have structure. Exploiting it should lead to better retrieval quality. However, doing so depends on the query, corpus and kind of retrieval. This paper comes up with an elegant and intuitive formulation to learn all of these weights during training. The baselines are well chosen, ablation experiments are extensive and the references are comprehensive.", "weaknesses": "The main weakness, and kudos to the authors for discussing this, is what they mention in Section 4 \"Multi-field vs Single-field\": \"A side-by-side comparison of the single field models against their multi-field counterparts shows mixed results\".  The difference in averages between mFAR_2 and mFAR_all doesn't appear statistically significant. The primary gains (looking at mFAR_2 vs mFAR_all) seem to be in the prime data set which has a lot of fields. Should the paper instead focus on adaptive hybrid retrieval, and choose hybrid retrieval baselines?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PvjKYKc7Iz", "reviewer": "Reviewer_xHpK", "summary": "This is a timely \"revisit\" of a multi-field retrieval problem with a proposal of multi-field adaptive retrieval that combines sparse-lexical and learned-dense-vector-similarity scores using field-weights, which are learnable and adaptive. Namely, each weight is a neural network incorporating a query representation and a field-specified learned weight vector.\n\nThe paper is very nicely written, it has a convincing set of results, as well as thorough literature review.", "questions": "The following questions were fully answered/addressed by the authors:\n1. What is exactly used for dense/vector retrieval?\n2. How did you decide on selection of top-100 records for each field (L885)?\n3. In Eq. 4, what is exactly q in G(q, f, m): I assume it should be a dense query encoder, but it's not clear from the text (at least for me).", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "* The paper revisits an important problem.\n* The paper is very well written\n* Experimental results are convincing and have quite a few baselines.\n* The method is simple and elegant yet effective", "weaknesses": "The only noticeable weakness is the need to compare to a better multi-field BM25 baseline, ideally, where field weights are learned using coordinate ascent (e.g., usling RankLib, or FastRank: https://github.com/jjfiv/fastrank). I do not think, however, that this should be a deal breaker. However, we encourage authors to list this as a limitation of their work.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7782nBTVbU", "reviewer": "Reviewer_vQrJ", "summary": "This paper presents Multi-Field Adaptive Retrieval (MFAR), a framework for retrieving structured documents by dynamically weighting document fields based on query relevance. By combining lexical and dense retrieval methods, MFAR improves retrieval accuracy on the STaRK dataset, surpassing state-of-the-art baselines and highlighting the potential of adaptive, hybrid approaches for structured data retrieval.", "questions": "Regarding Weakness 1 in the previous section, did you try to use other dense retrievers and do you have any insights about experiments with better retrievers?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper proposes a new approach that incorporates structured document retrieval, where documents contain multi-field information. The proposed method adaptively controls the information by using a weight adjustment strategy based on the query. \n2. The experimental results demonstrate that mFAR framework over various baselines on the STaRK dataset. \n3. The authors present detailed analysis demonstrating why their hybrid approach can outperform baselines through the experiments of both multi-field vs. single-field and hybrid vs. lexical/semantic similarity. \n4. The paper is well-structured, motivated, and written, thus easy to follow.", "weaknesses": "1. Though several recent retrievers [1] and [2] have been proposed, Contriever is used as a representative of dense retrievers without sufficient discussion. I am concerned that the conclusion may change if the authors use the recent retrievers. Especially, since FollowIR is instruction-tuned, it may be more robust against negation in the query, which the authors pointed out as a possible issue of the single-field method. \n2. The adaptive weighting mechanism in MFAR relies on training within specific domains and structured datasets, making it potentially sensitive to domain-specific data characteristics. This might lead to suboptimal field selection or scoring when the document structure is inconsistent across the corpus.  \n\n[1] Weller, Orion, et al. \"FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instructions.\" arXiv preprint arXiv:2403.15246 (2024).\n\n[2] Xiao, Shitao, et al. \"C-pack: Packaged resources to advance general chinese embedding.\" *arXiv preprint arXiv:2309.07597* (2023).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pA3Ji0LuxS", "reviewer": "Reviewer_Hpub", "summary": "This paper introduces a method called Multi-Field Adaptive Retrieval (MFAR), aimed at enhancing retrieval of structured or multi-field documents. The approach decomposes each document into multiple fields, each independently indexed and retrievable through different models. Each field is associated with a learnable embedding, and an adaptive function is trained to predict the importance of each field based on the query and scoring model. The authors validate MFAR's effectiveness through experiments across multiple datasets, comparing it against prior methods.", "questions": "n/a", "limitations": "", "rating": 6, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- S1: This study is the first to demonstrate that multi-field ranking outperforms single-field ranking within the context of dense retrieval. \n- S2: The experiments highlight that hybrid models enhance both single- and multi-field rankings, with certain multi-field scenarios benefitting more from hybrid approaches.\n- S3: A thorough ablation study demonstrates the significance of the query-conditioning mechanism in optimizing field weights. Additionally, the qualitative analysis shows the variability in optimal scorers across datasets, showing that there is no universally best approach.\n- S4: A detailed case study further illustrates the method’s effectiveness in practical applications.", "weaknesses": "- W1: Although the paper focuses on multi-field ranking, it does not include classic methods such as BM25F [1], its later extensions [2,3], or the mixture of language models [5], which are commonly applied in table retrieval [6], as part of the baselines. Incorporating at least BM25F or the mixture of language models would add valuable context and enable a more thorough comparison.\n- W2: Query-dependent field weighting has been previously explored, such as in table retrieval methods that incorporate both query-dependent and query-independent features [4]. Testing on table retrieval datasets could offer additional insights, as tables represent another structured, multi-field document type.\n- W3: The proposed method adaptively determines the importance of each field given a query and scorer; however, it does not select among scorers, instead requiring calculation of all scoring potentials, thereby increasing computational load.\n\n\n[1] Simple BM25 extension to multiple weighted fields, 2004\n\n[2] Field-Weighted XML Retrieval Based on BM25, 2006\n\n[3] Extending BM25 with Multiple Query Operators, 2012\n\n[4] Web Table Retrieval using Multimodal Deep Learning, 2020\n\n[5] Combining Document Representations for Known-Item Search, SIGIR 2003\n\n[6] Ad Hoc Table Retrieval using Semantic Similarity, WWW 2018", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Spotlight)