# An Efficient Document Retrieval for Korean Open-Domain Question Answering Based on ColBERT
## 2 Related Work  
The architecture of open domain question answering (ODQA) systems primarily combines information retrieval (IR) and reader modules. IR plays a crucial role in searching for evidence passages within a vast knowledge corpus to answer questions. Traditional non-neural models utilized in IR include TF-IDF [2] and BM25 [3]. These methods rely on lexical information, leading to significantly lower performance when there is a mismatch between the query and passage terminology. However, with advancements in deep learning and the introduction of powerful pre-trained language models such as BERT [4], methods surpassing lexical-level information have been proposed. These methods leverage semantic correlations between questions and passages for more effective document retrieval. The interaction-based retriever illustrated in Figure 1a inputs the query and context in the format [CLS] Query [SEP] Context [SEP] into BERT, enabling the computation of scores between queries and documents through derived representations [12; 13].  
Nevertheless, Transformer-based pre-trained language models (PLMs) tend to be computationally intensive and slow when dealing with longer input lengths. In response, dense passage retrieval (DPR) [3] independently encodes queries and passages for relevance score calculation, as shown in Figure 1b. Despite its efficiency, DPR does not consider the interaction between query and passage, which can limit its performance.  
To balance efficiency and accuracy, ColBERT [5] has been proposed. ColBERT, a representation-interaction retriever model, is depicted in Figure 1c. After encoding questions and documents separately using BERT-based encoders, it calculates token embedding scores for each question across all documents. Subsequently, all scores are summed to form the final relevance score between the question and document. ColBERT has demonstrated an effective balance between search performance and processing speed.  
Recent research efforts aim to develop efficient ODQA systems [6]. Techniques like brute search [15], hierarchical navigable small world graphs (HNSW) [16], approximate nearest neighbor (ANN) [17], locality-sensitive hashing (LSH) [8], and inverted file (IVF) [9] have been applied in various studies. Additionally, approaches such as removing the reader from the retrieval-reader structure [18] or directly generating answers from given questions using generative-only methods [19; 20; 21] have been proposed to enhance ODQA efficiency.  
This study aims to apply ColBERT, which shows a balanced performance in terms of search speed and accuracy, to the Korean language. In the ColBERT structure, BERT-based models are used to derive embeddings for queries and passages. BERT has significantly outperformed traditional methods in various tasks, including review classification [22]. Several versions of BERT exist in Korea [23], and open-source PLMs have been trained in different environments and datasets, necessitating efforts to find models suitable for  
Figure 1: Three types of dense retrievers [14]. Given a question (query) and a document as inputs, calculate the degree of relevance (s).  
specific tasks. Attempts to apply Korean BERT in document classification [24], comment analysis [25], and medical fields have been proposed [26].