An Auxiliary Task Boosted Multi-task Learning Method for Service Account Retrieval with Limited Human Annotation (2023)

Abstract: Service accounts, including organizations’ official accounts and mini-programs, provide various convenient services for users, and have become crucial components of a number of applications. Therefore, retrieving service accounts quickly and accurately is vital. However, this task suffers from the problem of limited human annotation, i.e., manually assessing account functionality and assigning ratings based on user experience is both labor-intensive and time-consuming. To this end, this paper proposes a novel approach, the Auxiliary task Boosted Multi-Task Learning method (AuxBoost-MTL). Specifically, the proposed method introduces multiple auxiliary tasks, which is able to utilized the log data from our application as supervision, and enhance the performance of the main task, service account retrieval. Furthermore, we introduce an Adaptive Hierarchical Fusion Module (AHF module) into our approach. This module is designed to adaptively perform hierarchical fusion of embeddings from auxiliary tasks into the main task, thereby enhancing the model efficacy. Experiments on two real-world industrial datasets demonstrate the effectiveness of our proposed approach.

## introduction service account retrieval services, exemplified by functionalities such as mini-program and official account searches, are fundamentally changing the way users interact with digital platforms @xcite . these accounts offer an array of services without the conventional need for downloading, proving critical in today's rapidly evolving digital age. however, despite their transformative potential and distinctive characteristics, service account retrieval services have not been thoroughly explored in the research field. contrary to traditional webpage retrieval that primarily involves query-document text matching * corresponding author: zhao zhang @xcite , service account retrieval provides functional services that cater to user queries, rather than merely facilitating simple text matching between queries and documents as seen in traditional webpage retrieval. this complexity presents significant challenges, particularly when annotating ranking datasets, a process that requires substantial time and effort due to the handson evaluation of each account's functionalities. the enormity of this task is underscored by our ongoing efforts, resulting in the annotation of approximately one hundred thousand entries in the service account search ranking dataset to date. addressing the urgent challenge of limited retrieval datasets, we propose an innovative auxiliary task boosted multi-task learning method (auxboost-mtl) that incorporates one main ranking task and four auxiliary tasks. specifically, the proposed method is able to selectively fuse embeddings from the auxiliary tasks into the main task through an adaptive hierarchical fusion module (ahf module) to enhance training. our application holds a vast amount of log data, where each data point (a query-account pair -numerous such pairs exist within a single search session) comprises a series of self-supervised labels readily available for training @xcite . in particular, we judiciously select four auxiliary tasks to correspondingly predict the following user feedback labels: click-through @xcite , retention (reuse within seven days), exit click (the last click within a search session), and dwell time @xcite . the first three tasks represent binary classification tasks, while the last task is a regression task. auxboost-mtl offers two fundamental benefits for the main task of retrieval: 1) we take advantage of the rich log data from our application, which alleviate the issue of the dearth of annotated ranking datasets. particularly, the availability of supervised signals from log data allow the shared layers of the multi-task framework to be trained, consequently enhancing the efficiency of the main ranking task @xcite . 2) we introduce four auxiliary tasks to boost the main task. specifically, with ahf module, the task-specific layers of the auxiliary tasks can be selectively incorporated into the main task's layer. this strategic integration refines the main task's embedding. ahf module achieves this by leveraging a knowledge graph for accounts and introducing the concept of "query entropy" to modulate the specificity and generality of a query. we highlight the contributions as follows: ## problem definition we formalize our problem as a multi-task learning setting for service account retrieval. given a query-account pair @xmath0, we define @xmath1, y c , y u , y e , y t } as the set of targets we aim to predict, where • y @xmath2, 1, 2, 3, 4} is the relevance level between the query and the account. objective: the goal is to minimize the following loss function: in this equation, f i (f ) is the prediction function for task i, l i is the corresponding loss for the task, and λ i is the weight for the loss of task i. our goal is to promote the performance of the main task. ## conclusion this paper introduces a novel method, auxboost-mtl, for service account retrieval. auxboost-mtl addresses the challenge of sparse annotated data by utilizing user feedback in abundant log data, and trains auxiliary tasks through self-supervision. specifically, the proposed method consists of three modules. in particular, the first two modules learn task-specific embeddings, and the third module, the ahf module, is able to adaptively fuses embeddings from auxiliary tasks into the main ranking task. experimental results on real-world datasets demonstrate the effectiveness of auxboost-mtl, making it a promising solution for service account retrieval systems.