Senti5: Sentinel Training with Function Names as Document IDs for Generative Code Search
Authors: Youngsoo Namgoong, Joonghyuk Hahn, Yo-Sub Han


Reviewer paper summaries: The paper introduces a novel approach to improving generative retrieval for code search. The authors propose an end-to-end generative retrieval method called Senti5, which is an improvement based on the DSI (Differentiable Search Index) and leverages sentinel tokens and function names as document identifiers.

In the vanilla DSI, the indexing task and retrieval task are employed in the training step using multi-task learning. In the vanilla setting, DocId is considered an integer number, which does not reflect the meaning of the code. The contribution of Senti5 is to use the function name as the DocId and add an auxiliary task that addresses the gap between code and natural language in code search by using a code summarization model.

Experimental results demonstrate that Senti5 significantly outperforms baseline DSI models on various datasets, showing strong generalization capabilities.
Summary of strengths: 1. Utilizes function names as doc IDs and sentinel tokens to align with pre-trained language models.
2. Outperforms baseline DSI models in generative code search across multiple datasets.
3. Integrates retrieval and indexing tasks into a single infilling task, reducing training complexity.
4. Employs a code summarization task to bridge the gap between natural language queries and code.
Summary of weaknesses: 1. Since Senti5 relies on the capabilities of pre-trained LMs, its effectiveness on large-scale datasets and its scalability compared to LLMs have not been explored.
2. The evaluation focuses on comparing with the DSI methods.
Comments, suggestions and typos: 1. I was wondering if the authors could add an experiment to compare with the traditional DPR-based methods.
2. Are there any hyperparameters to blend the proposed losses?
3. In the ablation study, it seems that code summarization is not that useful. Is it because of the quality of the summarization model?
4. Besides the function name, is the function description also helpful?
Overall assessment scores: 2.5