Contrastive Pre-training for Personalized Expert Finding
Authors: Qiyao Peng
Hongtao Liu
Zhepeng Lv
Qing Yang
Wenjun Wang
Venue: EMNLP 2023 Findings (2023)
Keywords: Expert Finding
Recommender Systems
Community Question Answering
Pre-training

Abstract: Expert finding could help route questions to potential suitable users to answer in Community Question Answering (CQA) platforms.
Hence it is essential to learn accurate representations of experts and questions according to the question text articles. Recently the pre-training and fine-tuning paradigms are powerful for natural language understanding, which has the potential for better question modeling and expert finding. Inspired by this, we propose a CQA-domain Contrastive Pre-training framework for Expert Finding, named CPEF, which could learn more comprehensive question representations. Specifically, considering that there is semantic complementation between question titles and bodies, during the domain pre-training phase, we propose a title-body contrastive learning task to enhance question representations, which directly treats the question title and the corresponding body as positive samples of each other, instead of designing extra data-augmentation strategies. Furthermore, a personalized tuning network is proposed to inject the personalized preferences of different experts during the fine-tuning phase. Extensive experimental results on six real-world datasets demonstrate that our method could achieve superior performance for expert finding.

Contrastive Pre-training for Personalized Expert Finding
Qiyao Peng', Hongtao Liu²; Zhepeng Lv², Qing Yang', Wenjun Wang
School of New Media Communication, Tianjin University, Tianjin, China
2 Du Xiaoman Financial, Beijing, China
Georgia Tech Shenzhen Institute, Tianjin University, Guangdong, China
4 Hainan Tropical Ocean University, Hainan, China
'[aypeng, wjwangJet.j.. edu. cn 2 (1iuhongtao01, lyzhepeng, yanggiigg)dduxiaammn. com
Abstract
2020). In order alleviate that, it needs to rec- ommend suitable experts provide satisfactory answers (Chang and Pal, 2013; Zhao et al., 2014) for these questions. Expert finding aims to effec- tively route questions to a suitable expert based on her/his historically answered questions, which has attracted considerable attention recently.
Expert finding could help route questions to po- tential suitable users to answer in Community Question Answering (CQA) platforms. Hence it is essential to learn accurate representations of experts and questions according to the ques- tion text articles. Recently the pre-traine pre-training and fine-tuning paradigms are powerful for natu- ral language understanding, which has the po- tential for better question modeling and ex- pert finding. Inspired by this, we propose CQA-domain Contrastive Pre-training frame- work for Expert Finding, named CPEF, which could learn more comprehensive question repre- sentations. Specifically, considering that there is semantic complementation between ques- tion titles and bodies, during the domain pre- training phase, we propose title-body con- trastive learning task to enhance question rep- resentations, which directly treats the question title and the corresponding body as positive samples of each other, instead of designing ex- tra mata-auumentttion strategies. Furthermore, a personalized tuning network is proposed to inject the personalized preferences of different experts during the fine-tunie fine-tuning phase. Extensive experimental results on six real-world datasets demonstrate that our method could achieve su- perior performance for expert finding.
Based on the powerful neural network (LeCun et al., 2015), many approaches (Li et al., 2019; Fu et al., 2020; Peng al., 2022a) have been shown effective improve the performance of expert find- ing. The core idea to first model the question semantically, and then learn the expert preference based on his/her historically answered questions. For example, Peng et al. (Peng et al., 2022a) pro- posed PMEF equipped with a multi-view ques- tion modeling paradigm aiming to model questions more comprehensively and then capture expert fea- tures. Hence, the capacity of the designed question modeling paradigm directly affects the expert find- ing performance.
Recently, Pre-trained Language Models (PLMs) have achieved great success in Natural Language Processing (NLP) due to their strong ability in text modeling (Devlin et al., 2018). Different from tra- ditional models that are usually directly trained with labeled data in specific tasks, PLMs are usu- ally pre-trained large unlabeled corpus via self-supervision encode universal semantic in- formation. However, existing PLMs are usually pre-trained on general corpus such as BookCor- pus and Wikipedia, which have some gaps with the CQA domain and the expert finding task. Di- rectly fine-tuning the existing PLMs with the expert finding task may be sub-optimal for CQA ques- tion understanding. Besides, such architecture does take into consideration the expert's 's person- alized characteristics, which could impact expert modeling and question-expert matching (i.e., ex- pert finding). At present, the exploration of the pre-training paradigm in expert finding is pre-
1 INTRODUCTION
Nowadays, Community Question Answering (CQA) platforms such as StackExchange BaiduZhidao 2 have been popular and extremely attracted millions of users (Fu et al., 2020), which raise their questions post answers for ques- tions they are interested in or good Due to the large participation, there are too many questions to wait for answers (Zhao al., 2017; Yuan et al.,
Hongtao Liu is the co-first author. Wenjun Wang is the corresponding author. 'ht httttsss//stackkccaanggge com chttps://zhidao baidu. com

---

liminary.
2015; Yuan et al., 2020: Liu al., 2022; Peng and Liu, 2022: Peng et al., 2022b) and it could help find capable experts for providing answers to questions. Early studies employed feature- engiiuerinngn (Cao et al., 2012; Pal et al., 2012) or topic-modeling technology (Guo et al., 2008; Riahi et al., 2012) to model questions and experts, then measured simi- larities between them for routing questions to suit- able experts. Subsequently, with the development of deep learning (LeCun et al., 2015), recent meth- ods employed neural networks learn expert and question representations, then matched them (Li et al., 2019; Fu et al., 2020; Ghasemi et al., 2021). For example, Peng et al. (Peng et al., 2022a) de- signed multi-view learning paradigm to learning question features, which could improve expert find- ing.
In this paper, we aim design novel ex- pert finding framework, which could learn more comprehensive question and expert representations with domain pre-training and personalized fine- tuning paradigm. For pre-training question rep- resentations, in addition to the domain pre-training Masked Language Model (MLM) task, we design contrastive task capture the inherent seman- tic information from the unlabeled CQA question corpus. Different from the vanilla contrastive learn- ing paradigm with data augmentation progress, we directly consider the question title and the ques- tion body as mutually augmented perspectives. In this way, the model could omit the tedious process of data augmentation while incorporating the two relevant types of information (i.e., question title and body) for question semantic modeling. For fine-tuning phase, we transfer the pre-trained ques- tion representations to model expert representations and the downstream expert finding task. Consid- ering different experts would have different tastes in the same question, hence we design a personal- ized tuning network, which could effectively inject personalized preference (i.e., expert ID) to learn more personalized expert representations for better question question-expert matching. In summary, the contributions of our method are:
Different from most existing methods learning question representations that rely supervised data, we design question title-body contrastive pre-training paradigm for modeling question. In this way, the model could capture more comprehen- sive question semantic representations, which are the basis of expert modeling and question-expert matching.
Furthermore, expert finding aims to recom- mend suitable experts for answering target ques- tions, which is similar to recommender system. Hence, we will introduce some recommendation researches with pre-training (Zhou et al., 2020; Wu et al., 2020; Cheng et al., 2021; Hou et al., 2022) in this part. For example, Cheng et al. (Cheng et al., 2021) proposed CLUE, which designed con- trastive pre-training tasks optimize transferable sequence-level user representations for recommen- dation. Compared with existing recommendation works under the pre-training paradigm, the expert finding has different characteristics, e.g., the tar- get question is usually cold-started and ID-based recommendation may not be feasible.
We propose domain pre-training and fine- tuning framework for expert finding on CQA platforms, which could learn comprehensive representations of questions and experts.
We design a novel question title-body con- trastive pre-training task, which could im- prove question modeling effectively. Further, we adopt personalized tuning network to learn personalized representations of experts.
We conduct extensive experiments on six real- world CQA datasets. Experimental results show that our method can achieve better per- formance than existing baselines and validate the effectiveness of our approach.
It is noted that the very recent method Ex- pertBert (Liu et al., 2022) designs preliminary expert-specific pre-training framework, which has achieved the best performance. Different from that, our approach focuses more comprehen- sive question semantic representations via integrat- ing the title-body contrastive learning task during pre-training. In addition, we design a personalized tuning network to tune the model and model expert according to different expert preferences.
2 RELATED WORK
In this section, briefly review the related work in two aspects, namely Expert Finding and Pre- training for Recommendation.
Expert finding has received much attention from both research and industry community (Liu et al.,

---

3 Problem definition
token [SEP] is added at the end of the input word sequence for recognizing the end of the sequence. Hence, the question information is denoted as:
Expert finding in CQA websites is to predict the most suitable expert to answer questions. Suppose that there is target question o' and candidate expert set E = {ei,--. ,eM) respectively, where M is the number of experts. Given candidate expert e E, she/he is associated with a set of her/his historical answered questions, which can be denoted as Q; {q1, ... .qn) where n the num- ber of historical questions. In addition to the title, each question contains the body information, which is a detailed description of the question; however, most existing methods ignore the question bodies, In fact, the question body could be complemen- tation of the question title. Hence, in this paper we would explore how to utilize the title and body together effectively.
Q [TITLE
[tk]], [t22)...
[BODY]:
title
(1)
[tki]. [tk2]. [tka]]
[SEP]]
body
Then, the input representation matrix R. is con- structed by summing its corresponding token, seg- ment, and position embedding:
R Rokeen Rugg + Raos
(2)
where Reoken is the embedding matrix of tokens derived from tokenizing Q (with BERT Tokenizer). and Rese and Rpos corresponding segment em- bedding and position embedding respectively fol- lowing the settings in BERT (Devlin et al., 2018). is noted that the question title segment ID is O and the question body segment id is I, which could help distinguish between these two types of infor- mation.
4 Proposed Method
In this section, we will introduce the training pro- cedure our method CPEF, which is composed the following stages.
In pre-training phase (question modelling), based the CQA domain unlabeled text cor- pus, propose a contrastive learning task between the question title and body and inte- grate it into the pre-training Masked Language Model (MLM) task jointly, to fully learn ques- tion semantic representations.
(2) Model Architecture In our method, we adopt the widely used BERT as our base backbone en- coder to further pre-train question representations in the CQA domain corpus. BERT architecture consists of multi-layer Transformer encoder layers. Each Transformer encoder layer has the following two major sub-layers, i.c., multi-head self-attention and position-wise feed-forward layer.
In fine-tuning phase (expert finding), we ini- tialize the parameters of the question represen- tation model with the pre-trained parameters and then utilize personalized tuning network to fine-tune the model with traditional expert finding supervised signals.
Multi Head Self-Artention. This sub-layer aims to capture the contextual representations for each word. The self-attention function is defined as:
Att Att K, V) = Smax (QK" T /VdVv, (3)
where Q K and V represent the query, key and value matrix correspondingly. Multi-head self- attention layer MH(-) will project the input to mul- tiple sub-spaces and capture the interaction infor- mation, which is denoted as:
4.1 Pre-training Framework
In this part, will present the question represen- tation pre-train mre-training framework, including the well- designed input layer and brief backbone architec- ture introduction.
MH(R) [head];- heada] W
(4)
(1) Input Layer For the model input, we concate- nate the words of the question title and the question body into whole sequence shown in Figure 1. We propose design special tokens (CTITLEJ and [BODY]) at the beginning of the title sequence and the body sequence respectively, to indicate the ti- tle text and the body text. Moreover, the special
head; = Att((XWy XW RW"),
(5)
where W7. W W E d and W Raxd are parameters. Via the multi-head self-attention, the representation matrix is transformed to E Rexd where n is the token number.

---

Title Reganseniaaion
Body Repremmtttioo
Tidle-Body C Contrastive Task
Postiive Sample
41
Negativve Sample
MLM Task
PLMs
41
ckg
[M]
tk;
tks
tkg
[BODY]
[k
tk
[M]
tkg
tkg
tit
tkg
[SEP]
Raith
92
[TTTLE]
6kg
c/2
[M]]
[BODY]
t/₃
C/2
[M]
tkty
t&g
9a
ck2
1kg
tkg
ckg
tk
[BODY]
CK2
Ck2
1/g
tkg
tkg
ck3
[M]
Question Title
Question Body
Figure 1: CPEF Question Modelling: Contrastive Pre-training Framework. We employ the title-body contrastive task and masked language model task to jointly learn more comprehensive question representations. th denotes the token and represents the question.
the question body could describe the question from two different perspectives, directly consider the question title and the question body positive samples mutually.
Predictor
Personalized Tuning
this way, the model could omit the tedious process of data augmentation and combine two types of information (i.e., question title and question body) by contrastive learning.
ML.Ms
PLMs
PLMS
[TTLE [BODY]
[TTTE] (RODY]...
MTL.E [BODY]
Q1 8
4n
Eq
Expert Answered Questions
Target
As shown in Figure 1, given the i-th question, its title representation qf and its body representation qf are positive samples of each other, while other in-batch questions considered negatives. The contrastive task can be denoted as:
Figure 2: CPEF Expert Modelling: Personalized Fine- tuning Network.
Position-wise feed-forward. For the input H, the calculation defined as:
Σ log exp(sim( a g/)/)) 1=1 Σ exp(sim(q g3)TT)
B
Lg -
FFN(H) RELU(HW/ + b// + +b
(6)
(7)
where of is i-th question title representation and qf is the question body representation of the neg- ative instance. is the temperature. Through this contrastive learning module, model could learn more precise and distinguished representations of questions.
where Wi, W and bf, by are learnable parame- ters. More details please refer to the BERT (Devlin al., 2018).
Then, we will next introduce our well-design pre-training tasks, which could help capture more comprehensive question representations.
Furthermore, for helping capture the CQA lan- guage knowledge, inspired by BERT (Devlin et al., 2018), we simply mask some percentage of the in- put tokens randomly and then predict those masked tokens. The formal definition is:
4.2 Pre-training task
Having established the input information, during the pre-training stage, we propose to employ two self-supervised tasks (including the contrastive task and mask language model task) for question pre- training.
N
Lm = Σ
logp(tk = th; th, [1,2, V]
Contrastive Task. This task is discriminate the representation of a particular question from other questions. Considering the question title and
(8)
where N the masked token set, V is the vocabu- lary set.

---

Model Pre-training. At the pre-training stage. we leverage a multi-task training strategy to jointly pre-train the model via the contrastive learning task and the MLM task, denoted as:
Datasets
questions
answerers
answers
Es
68,104
9,539
94,393
Tex
129,202
6,999
189,368
Unix
96,258
15,581
163,553
English
51,498
10,655
120,414
Physics
85,776
10,117
143,570
cp Lig + f.m
Electronics
78,954
7,402
143,522
4.3 Personalized fine-tuning
Table Statistical details of the datasets.
We have obtained the pre- rre-trained question represen- tation model based on the unlabeled CQA corpus. In this section, will tune the pre-trained model for evaluating the expert finding task. For realizing the task, we need model experts based on their historical answered questions.
denoted as:
exp(s) Ej= exp(s;)
K+1
Ltt
Σ
Blog(s)
KI
(12)
Despite modeling questions more comprehen- sively during the pre-training stage, such an archi- tecture is less capable in modeling personalized preferences for experts. Intuitively, an expert is likely to show personalized tastes when answer- ing questions. Hence, our solution is to design an expert-level personalized tuning mechanism for injecting personalized preference while modeling experts.
where is the ground truth label and is the malized probability predicted by the model.
Experiments
In this section, we first introduce the experiment settings and then present the results and analysis.
5.1 Datasets and Experimental Settings
Specifically, as illustrated in Figure 2, given the i-th expert historical answered questions (i.e., Q iqi.... ),,)), first employ the pre- trained question modeling weight obtain each historical question representations (q1 qn). Then, we design a personalized tuning mechanism to compute the attention weight a of j-th histor- ical answered question with respect to the expert, which is denoted as follows:
For pre-training question representations, we con- struct dataset containing 525,519 unlabeled ques- tion data (including question title and question body), which is from StackExchange". For fine- tuning and verifying the effect of the pre-trained model in specific domains, we select six different domains, i.e., Es, Tex, Unix, Physics, English and Electronies. Each dataset includes a question set, in which, each question is associated with its ti- de. body, and "accepted answer" among several answers provided by different answerers. The de- tailed statistical characteristics of the datasets are shown in Table 1.
exp(l,) T k= exp(4x) L7 = (o:)" @ qi (10)
where and qj representations of the i-th ex- pert ID embedding and the j-th historical answered question respectively. In this way, the model could capture the expert's personalized interests on dif- ferent questions.
We split each dataset into training set, a valida- tion set and a testing set, with the ratios 80%, 10%, 10% respectively in chronological order. Specifi- cally, we filter experts who did not answer twice for mitigating the cold start problem. We pad truncate the length of the question title and body as 10 and 30 respectively. For pre- mre-training, we set the ratio of masking tokens is 0.2 and the temperature is 0.1. For fine-tuning, following the previous works, we set the number of negative samples K is 19. For testing methods, we randomly assigned 80 experts to each question (including those who had answered the question) and the baseline methods were reproduced as such.
Finally, for obtaining the expert representation, we aggregate historical question features according to their weights:
e;
ajqh
(11)
where e; is the expert i final representation. For fine-tuning the model, we use the negative sampling technology (Huang et al., 2013) to sample K negatives and the cross-entropy loss, which is

---

Datesel
Physics
Tex
English
Metric
MRR
Pm3
NDCG@10
MRR
P03
NDCG@10
MRR
P83
NDCG@10
Method
CNTN
0.3088
0.3837
0,4036
0.3825
0.4513
0,4612
0,3015
0.3001
0,3653
NeRank
0.4801
0.5734
0.5608
0.5891
0.6734
0.6632
0.4086
0.4633
0.5089
TCQR
0.4025
0.4937
0.5087
0.4733
0.5553
0.5634
0.3525
0.3621
0.4053
RMRN
0.4720
0,5676
0,5522
0,5890
0.6786
0.6655
0.4023
0.4611
0.5099
UserEmb
0.3773
0.4656
0.4536
0.4256
0.5223
0.5333
0.3277
0.3362
0.3915
PMEF
0.4835
0.5723
0.5615
0.5966
0.7018
0.6711
0.4099
0.4732
0.5120
ExpertBert
0.4908
0.5803
0.5635
0.6042
0.7033
0.6735
0.4126
0,4812
0.5155
CPEF
0.5051
0.5909
0.5753
0.6209
0.7204
0.6896
0.4276
0.4936
0.5211
Dateset
Electronics
Unix
Es
Metric
MRR
P83
1D
MRR
P68
NDCG
MRR
P83
NDCGR10
Method
CNTN
0.3188
0.3537
0.3936
0.3529
0.4313
0,4416
0.2998
0.3121
0.3358
NeRank
0.5111
0,6012
0,6008
0.5628
0.6116
0.6423
0.4813
0.5386
0.5369
TCQR
0.4075
0.5037
0.5127
0.4836
0.5353
0.5734
0.3825
0.4621
0.4453
RMRN
0.5232
0.6111
0.6172
0.5723
0.6232
0.6521
0.4912
0,5393
0.5399
UserEmb
0.3623
0.4758
0.4699
0.4156
0.4823
0,5133
0.3077
0.3662
0,3715
PMEF
0.5423
0.6302
0.6282
0.5888
0.6412
0.6687
0.5093
0.5588
0.5501
ExpertBert
0.5433
0.6301
0.6199
0.5988
0.6521
0.6725
0,5052
0.5601
0.5575
CPEF
0.5579
0.6426
0.6305
0.6064
0.6711
0.6799
0.5192
0.5726
0.5735
Table 2: Expert finding results of different methods. The best performance baselines underlined. We perform t-test and the results show that CPEF outperforms other baselines at significance level p-value-<0.05.
We adopt the pre-trained weight bert-base- uncased as the base model including 110M pa- rameters. To alleviate the over-fitting problem, we utilize dropout technology (Srivastava et al., 2014) and set the dropout ratio as 0.2. We adopt Adam (Kingma and Ba, 2015) optimization strat- egy optimize
TCQR (Zhang et al., 2020) utilizes tempo- ral context-aware model in multiple temporal granularities to learn the temporal-aware ex- pert representations.
RMRN (Fu et al., 2020) employs recurrent memory reasoning network to explore implicit relevance between expert and question.
model and set the learning rate to 5e-5 in further pre-training and 5e-2 in fine- tuning. The batch of pre-training and fine-tuning is 8. We employ pytorch, transformers, sklearn, numpy, accelerator, etc to implement our code. We independently repeat each experiment 5 times and report the average results. All experiments use two 24GB-memory RTX 3090 GPU servers with Intel(R) Xeon(R) (2222))2. CPU.
UserEmb (Ghasemi et al., 2021) utilizes a node2vec to capture social features and uses a word2vec capture semantic features, then integrates them improve expert finding.
PMEF (Peng al., 2022a) designs a person- alized expert finding method under multi- view paradigm, which could comprehensively model expert and question.
5.2 Baselines and Evaluation metrics
We compare our method CPEF with recent com- petitive methods including:
ExpertBert (Liu et al., 2022) designs a expert- specific pre-training framework, towards pre- cisely modelling experts based on the histori- cal answered questions.
CNTN (Qiu and Huang, 2015) employs CNN to model questions and computes rel- evance scores between questions and experts.
The evaluation metrics include Mean Recip- rocal Rank (MRR) (Craswell, 2009). P@3 (i.c.. Precision@3) and Normalized Discounted Cumu- lative Gain (NDCG@10) 10) (Järvelin and Kekäläinen, 2002) to verify the expert ranking quality.
NeRank (Li et al., 2019) learns question, raiser and expert representations via HIN embedding algorithm and utilizes the CNN to match them.

---

Datesel
Electronics
Unix
Es
Metric
MRR
Pm3
NDCG@10
MRR
P03
NDCG@10
MRR
P83
NDCG@10
Method
w/o CT
0.5351
0.6111
0.6127
0.5722
0.6333
0.6339
0,4795
0.5423
0,5476
w/o Per
0.5401
0.6358
0.6266
0.5823
0.6576
0.6518
0.5005
0.5611
0.5628
Original BERT
0.5228
0.6016
0.5922
0.5619
0.6233
0.6176
0.4636
0.5328
0.5389
CPEF
0.5579
0.6426
0.6305
0.6064
0.6711
0.6799
0.5192
0.5726
0.5735
Table 3: The variants CPEF experiment results.
5.3 Performance Comparison
CPEF
0.5
PMEF
0.7
PMEF
We compare the proposed CPEF with the baselines on the six datasets. The experimental results of CPEF and other comparative methods are in Ta- ble 2. There are some observations from these results.
P8B
PGBS
0.3
0.4
20% 40% 60% 80% 100% Physies
20% 40% 60% 80% 100% (b) Tex
Some earlier methods (e.g., CNTN) obtain poorer results on almost datasets, the reason may be that they usually employ max or mean operation on histories to model expert, which omits different history importance. PMEF utilizes a multi-view question encoder to construct multi-view question features, which could help learn more comprehen- sive question representations, which is superior to other baseline methods.
Figure Impacts of Train Data Ratio.
sonalized fine-tune the model; (b) w/o Per, adopt the title-body contrastive learning and the MLM for pre-training but remove the personalized tuning mechanism during modeling experts; (c) Original BERT, directly adopt the original BERT weight for expert finding via fine-tuning way, without pre- pre-training and personalized tuning.
Furthermore, the PLMs-based methods (Expert- Bert, CPEF) achieve better performance. The rea- son is that the PLMS encoder could learn more precise semantic features of questions and perts compared with the traditional methods. Fi- nally, compared with all baselines, our approach CPEF performs consistently better than them on six datasets. Different from the ExpertBert which only conducts MLM pre-training, we make full use of question titles and question bodies and design an extra title-body contrastive pre-training task, which allows the model to learn question represen- tations more comprehensively in self-supervised way. Furthermore, utilize the personalized tun- ing mechanism to transfer and tune the pre-trained model according to different experts' personalized preferences. This result shows that our approach is effective to improve the performance of the expert finding.
As shown in Table 3, we can have the follow- ing observations: (1) Regardless of removing the comparative learning in the pre-training (i.e., w/o CT) or the personalized tuning network in the fine- tuning (i.e., w/o Per), the model performance will decline. (2) w/o Per outperforms w/o CT. The rea- son is that the pre-trained question representations are the basis for subsequent expert modeling and question-expert matching. And the results show that the importance of the contrastive learning task in pre-training greater than that of the person- alized tuning. (3) It not surprised that BERT obtains the worst performance. This is because the BERT is pre-trained on the general corpus, which causes the semantic gap with the CQA. Further- more, directly fine-tuning BERT could not take the personalized characteristics into account. (4) Our complete model CPEF obtains the best results. comprehensive The reason is it can pre-train question representations. Further, the personalized tuning network could capture expert personalized preference, which could yield better performance.
5.4 Ablation Study
To highlight the effectiveness of our designing ex- pert finding architecture, we design three model variants: (a) w/o CT, only adopt the corpus-level MLM to pre-train over CQA corpus and remove the title-body contrastive learning, and then per-
In all, the results of ablation studies meet our motivation and validate the effectiveness of our proposed method.

---

P08S
NDCGg10
PR8
NDCG@10
0.7
0.7
0.5
0.55
0.6
0.4
0.20
0.30
0.30
0.01
(4) Tex
(6) English
(x) Unix
(b)
Figure 4: Impacts of different mask ratio.
Figure 5: Model performance w.r.t.
5.5 Further Analysis
degrades. When the ratio is small, the BERT model could not capture adequate CQA language knowl- edge, which could reduce the performance of down- stream tasks. On the contrary, when the ratio is large, the [M] symbol appears in pre-training stage more frequently, which could intensify the mis- match between pre-training and fine-tuning, Hence, we set up the ratio of masking tokens to 0.2 during the pre-training stage.
this section, we further analyze the ability of the model to mitigate the impact of data sparsity and the impact two hyper-parameters on the model performance.
Effect of Train Data Ratio in Fine-tuning. In NLP. the PLMs could alleviate data sparsity prob- lems in the downstream task. In this part, we conduct experiments to analyze whether this phe- nomenon still cxists in expert finding, we adjust the training data ratio in the fine-tuning stage. We employ [20%, 40%, 60%, 80%, 100%] all data in Physics and Tex dataset as training data to fine- tune model, meanwhile, the ratios of validation data and testing remain the same (i.e., 10%) with the main experiments. And use the recent com- petitive method PMEF the baseline method.
Effect of Temperature т. In our method, we employ the contrastive task pre-train question representations, and hence, the temperature 7 plays a critical role during pre-training which could help mine hard negative samples. Figure shows the curves of model performance w.r.t. т. We can observe that: (1) Increasing the value of (e.g.. 2.2) would lead poorer performance. This is because increasing the would cause the model could not distinguish hard negatives from easy neg- atives. (2) In contrast, fixing
The results are shown in Figure 3. As we can see, the two model performance substantially drops when less training data is used. Nevertheless, find that there growing gaps between the results of CPEF and PMEF with the reduction of training data, which indicates the advantage of the pre-trained question representation model is larger when the training data is more scarce. This obser- vation implies that the CPEF shows the potential to alleviate the data sparsity issue via fully mining the knowledge from limited question information.
too small value (c.g., <0.05) would hurt the model performance. The reason may be the gradients of a few negatives dominate the optimization, which would lose bal- ance during model optimization. In nutshell, we suggest setting the temperature is 0.1.
Conclusion
In this paper, we propose a domain contrastive pre- training expert finding model to learn more effec- tive representations of questions and experts. The core of our method is that we design a contrastive framework between question bodies and titles to in- tegrate into the pre- pre-training phrase with the domain CQA corpus. In this way, our method could obtain precise features of questions well experts. In addition, the personalized tuning network could further enhance expert representations. Extensive experiments on several CQA datasets validate the effectiveness of our approach, which could outper- form many competitive baseline methods.
Effect of Masking Token Ratio. In pre- training, we employ the masked language model to capture the CQA knowledge, and hence the mask ratio is an important hyperparameter of CPEF, We have varied the ratio 10 0.15, 0.2, 0.25, 0.3] for exploring the model performance w.r.t different mask ratios.
The experimental results are shown in Figure We can find that all metric results increase at first as the masking ratio increases, reach the maximum value (i.e., the best model performance), and then

---

7 Limitations
ACM Transactions on Knowledge Discovery from Data, 15(4):1-16.
Our contrastive pre-training method has achieved better performance in expert findings, there are the following limitations that we would like to explore in the future yet. Firstly, although PLMs have been widely used in various fields, the large computation and resource costs are still issues for PL.M-asee models. Hence, we would consider using further distillation operations for inference efficiency. Sec- ondly, the experiments are conducted in different domains of Eaacanxxchnggecco and it would be more convincing if the datasets contains other CQA platforms such as Quora.com or Zhihu.com.
Jinwen Guo, Shengliang Xu, Shenghua Bao, and Yong Yu. 2008. Tapping on the potential of q&a commu- nity by recommending answer providers. In Pro- ceedings of the ACM Conference on Information and Knowledge Management, pages 921-930.
Yupeng Hou, Shanlei Mu, Wayne Xin Zhao, Yaliang Li, Bolin Ding, and Ji-Rong Wen. 2022. Towards universal sequence representation learning for rec- ommender systems. The 28th International Con- ference on Knowledge Discovery and Data Mining, pages 585-593 593359934445 ACM.
Po-Sen Huang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Acero, and Larry Heck. 2013. Learning deep structured semantic models for web search using click through data. In Proceedings of the Confer- ence on Information and Knowledge Management, pages 2333 2338.
A Acknowledgement
This work was supported by the Shenzhen Sustainable Development Project under Grant (KCXFF2000112211733:0000
Kalervo Järvelin and Jaana Kekäläinen. 2002. Cu- mulated gain-based evaluation of ir techniques. ACM Transactions on Information Systems (TOIS), 20(4):422 446.
References
Diederik P Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of the International Conference on Learning Repre- sentations.
Xin Cao, Gao Cong, Bin Cui, Christian Jensen, and Quan Yuan. 2012. Approaches to exploring cate- gory information for question retrieval in community question-answer archives. ACM Transactions on In- formation Systems (TOIS), 30(2):1-38.
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. Nature, 521:436-4444
Shuo Chang and Aditya Pal. 2013. Routing questions for collaborative answering in community question answering. In Proceedings of the International Con- ference on Advances in Social Networks Analysis and Mining, pages 494-501. IEEE.
Zeyu Li, Jyun- Yu Jiang, Yizhou Sun, and Wei Wang- 2019. Personalized question routing via heteroge- network embedding. In Proceedings of the International Conference on Artificial Intelligence. volume 33, pages 192-199.
Mingyue Cheng. Fajie Yuan, Qi Liu, Xin Xin, and En- hong Chen. 2021. Learning transferable user repre- sentations with sequential behaviors via contrastive pre-training. In IEEE International Conference on Data Mining, ICDM 2021, Auckland, New Zealand, December 7-10, 2021, pages 51-60.
Hongtao Liu, Zhepeng Lv, Qing Yang, Dongliang Xu, and Qiyao Peng. 2022. Experteett: Pretraining ex- pert finding. In Proceedings of the 31st ACM in- ternational Conference on Information and Knowl- edge Management. CIKM '22, page 4244-4248 New York. NY. USA.
Nick Craswell. 2009. Mean reciprocal rank. Encyclope- dia of database systems, 1703.
Xuebo Liu, Shuang Ye, Xin Li, Yonghao Luo, and Yanghui Rao. 2015. Zhihurank: A topic-sensitive expert finding algorithm in community question an- swering websites. In Proceedings of the Interna- tional Conference on Web Web-Based Learning, pages 165-173. Springer.
Jacob Devlin, Ming-Wei Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understand- ing. In Proceedings of the International Conference on North American Chapter of the Association for Computational Linguistics.
Aditya Pal. F Maxwell Harper, and Joseph A Konstan. 2012. Exploring question selection bias to identify experts and potential experts in community question answering. ACM Transactions on Information Sys- tems (TOIS), 30(2):1-28.
Jinlan Fu, Li, Qi Zhang, Qinzhuo Wu, Renfeng Ma, Xuanjing Huang, and Yu-Gang Jiang, 2020. Recur- rent memory reasoning network for expert finding in community question answering. In WSDM, pages 187-195.
Qiyao Peng and Hongtao Liu. 2022. Expertplm: Pre- training expert representation for expert finding, In Findings of the Association for Computational Lin- guistics: EMNLP 2022, pages 1043-1052, Abu Dhabi, Association for Computational Linguistics.
Negin Ghasemi, Ramin Fatourechi, and Sacedeh Mom- tazi. 2021. User embedding for expert finding in community question answering. Proceedings of the

---

Qiyao Peng, Hongtao Liu, Yinghui Wang. Hongyan Xu. Pengfei Jiao, Minglai Shao, and Wenjun Wang. 2022a. Towards multi-view attentive matching for personalized expert finding. In Proceedings of the ACM Web Conference 2022, pages 2131-2140.
Information and Knowledge Management, Virtual Event, Ireland, October 19-23, 2020, pages 1893 1902. ACM.
Qiyao Peng. Wenjun Wang, Hongtao Liu, Yinghui Wang, Hongyan Xu, and Minglai Shao. 2022b. To- wards comprehensive expert finding with hierarchi- cal matching network. Knowledge-Band Based Systems, 257:109933.
Xipeng Qiu and Xuanjing Huang. 2015. Convolutional neural tensor network architecture for community- based question answering. In Proceedings of International Joint Conference on Artificial Intelli- gence.
Fatemeh Riahi, Zainab Zolaktaf, Mahdi Shafiei, and Evangelos Milios. 2012. Finding expert users in community question answering. In Proceedings of the International Conference World Wide Web, pages 791-798.
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Πlya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: simple way prevent neural networks from overfitting. The journal of Machine Learning Research, 1511:1999--995.
Chuhan Wu, Fangzhao Wu, Tao Qi. Jianxun Lian, Yongfeng Huang, and Xing Xie. 2020. PTUM: Pre- training user model from unlabeled user behaviors via self-supervision. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1939-1944, Online. Association for Computational Linguistics.
Sha Yuan, Yu Zhang, Jie Tang, Wendy Hall, and Juan Bautista Cabotà. 2020. Expert finding in com- munity question answering: review, Artificial Intel- ligence Review, 53(2):843-8)
Xuchao Zhang, Wei Cheng, Bo Zong. Yuncong Chen, Jianwu Xu, Ding Li. and Haifeng Chen. 2020. Tem- poral context-aware representation learning for ques- tion routing. In WSDM, pages 753-761.
Zhou Zhao, Hanqing Lu, Vincent W. Zheng, Deng Cai, Xiaofei He, and Yueting Zhuang. 2017. Community- based question answering via asymmetric multi- faceted ranking network learning. In Proceedings of the Thirty-First Conference on Artificial Intelli- gence, February, pages 3532-3539. AAAI Press.
Zhou Zhao, Lijun Zhang, Xiaofei He, and Wilfred Ng. 2014. Expert finding for question answering via graph regularized matrix completion. IEEE Transactions Knowledge and Data Engineering, 1704).993-1004
Kun Zhou, Hui Wang. Wayne Xin Zhao, Yutao Zhu. Sirui Wang, Fuzheng Zhang, Zhongyuan Wang, and Ji-Rong Wen. 2020. S3-rec: Self-supervised learning for sequential recommendation with mutual informa- tion maximization. In International Conference on


Reviews:
[{"review_id": "2qROtg8shj", "reviewer": "Reviewer_QLNj", "summary": "", "questions": "Question A: Instead of BERT, do authors try different pre-trained language models and analyze how the selection of models would affect the performance?\n\nQuestion B: Could the authors provide some concrete examples about how the proposed method improves the performance?", "limitations": "", "rating": 4, "confidence": 5, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks a lot for all your positive and insightful comments.\n\n### R-1 (the technique itself is not novel in NLP)\n\nThanks for your comment.\n\nWe agree that the contrastive pre-training technology is not novel in NLP. However, there is no effective method to explore the effectiveness of contrastive learning in CQA or expert finding. \nIn fact, the expert finding task has the following unique characteristics:\n\n1) Existing general PLMs are usually pre-trained on the general corpus, such as BookCorpus, Wikipiedia, which have some gaps with the CQA domain and expert finding task. Additionally, we fine-tuned the original BERT model directly on the expert finding task and presented the experimental results in “Table 3”.\n2) The expert finding task is essentially a recommendation task based on the text modeling. Unlike traditional NLP tasks that solely focus on text modeling, the expert finding task needs to model question and expert simultaneously. Furthermore, different experts have different personalized characteristics. Hence it is non-trivial to directly employ the PLMs in expert finding.\n3) Unlike certain contrastive pre-training methods in NLP, we do not require the design of additional data augmentation techniques because the question body could serve as a naturally augmented view of the question title.\n\nConsequently, we propose a question-level pre-training and expert-level fine-tuning architecture that enables the learning of more precise question embeddings during pre-training and more personalized expert embeddings during fine-tuning.\n\nWe apologize for any confusion caused. In the final version, we will provide a more detailed explanation of the motivation to enhance the clarity and highlight the novelty of our paper for easier comprehension.\n\n\n### R-2 and Q-B (lack of concrete examples)\n\nThanks for your comment. We agree that there is indeed a lack of a case study to visualize the effectiveness of our approach more. \n\nHere we present the case study as follows. Given a target question:\n- title: \"Magnetic field on the surface of a solenoid\",\n- body: \"Rotating cylinder behaves like a solenoid, and we can thus compute that the external magnetic field is 0 and that the magnetic field inside it is a constant value  But what is the magnetic field at the surface?\". \n\nThe CPEF and ExpertBERT recommend different experts for answering the question. Expert A recommended by CPEF is the ground truth, while Expert B recommended by ExpertBERT is false. Partial historical answers for A and B are as follows:\n\n**Table 1.** Expert A.\n|Question|Title|Body|\n|------------|-------|------|\n|1|How can I find the force of a solenoid with a moving plunger|It seems like it should be a simple equation, until I realized that the core isn't magnetized until it is induced, then there is a dipole moment, and then as it moves the core of the solenoid gradually changes from air to the core material|\n|2|Force to extend a solenoid|Suppose a long solenoid has length l, area S and N turns. I want to know the force required to extend it when the current is I.|\n|3|Electric Field in an Infinite Alternating Current Solenoid|I stumbled upon an explanation I can't understand. It goes like this: There's an alternating current solenoid with radius a. We know the magnetic induction is|\n\n**Table 2.** Expert B.\n|Question|Title|\n|------------|-------|\n|1|Magnetic moment precession around magnetic field|\n|2|Moving conducting bar in (changing) magnetic field|\n|3|Condition for the magnetic field|\n|4|induced emf when a wire or coil travel through a magnetic field|\n\nFrom Tables 1 and 2, we can find Expert B has the potential to answer the target question because he answered \"magnetic field\" related questions. ExpertBERT recommends Expert B for the target question based on this fact. However, CPEF takes the question body as the augmented view of the question title, which could capture additional rich information for question embedding. In this way, the expert embedding learned by CPEF based on historical questions is more related to the \"solenoid magnetic field\". Hence, the CPEF model recommends Expert A not B to the target question \"Magnetic field on the surface of a solenoid\".\n\n\n\nWe will include a case study in the final version to better visualize how the proposed method improves the performance.\n\n\n\n### R-3 (presentation and writings)\n\nThanks for the suggestions about the presentation and writing in our paper. We will carefully do comprehensive proofreading and invite a native English speaker to help polish the writing.\n\n### Q-A (different pre-trained language models)\n\nThanks for your suggestions. We agree that employing a more advanced pre-trained model may improve the effectiveness of the model. However, the main focus of this paper is not the selection of pre-trained models, but rather the designing of the pre-training architecture in expert finding. Therefore, we adopt the widely used BERT as the “base” model.\n\nMoreover, we incorporate SimCSE and RoBERTa as alternative “base” models and subsequently assess their performance. The results are as follows:\n\n**Table 3.** Different Pre-trained Language Models (Physics dataset as an example).\n|              |MRR|P@3|NDCG@10|\n|------------|-------|------|---------------|\n|SimCSE|0.5052|0.5913|0.5750|\n|RoBERTa|0.5049|0.5908|0.5763|\n\nThe results indicate that replacing the pre-trained model has minimal influence on the performance since our pre-training strategy is conducting domain-specific pre-training in the CQA field. The existing pre-training model primarily offers improved parameter initialization, which can be viewed as the model \"base\". Our pre-training corpus and designed tasks would encourage the model to capture more domain-specific information.\n\nIn future work, we plan to investigate the application of more advanced pre-trained models for CQA and expert finding.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* Contrastive pre-training is novel for personalized expert finding.\n* Extensive experiments with comparisons to many SOTA methods.\n* Ablation study shows the effectiveness of each method.", "reasons_to_reject": "* The technique itself is not novel in NLP while the method is not tailored for the specific research question.\n* Lack of concrete examples to demonstrate how the proposed method improves over baseline methods.\n* Presentation and writings have room for improvement. ", "questions_to_authors": "Question A: Instead of BERT, do authors try different pre-trained language models and analyze how the selection of models would affect the performance?\n\nQuestion B: Could the authors provide some concrete examples about how the proposed method improves the performance?", "paper_topic_and_main_contributions": "In this paper, the authors propose a pre-training framework based on contrastive learning for personalized expert finding in the task of community question answering. Experiments are conducted on several StackExchange datasets. The experimental results show that the proposed method can outperform several conventional baseline methods. The main contribution of this paper is the novel usage of contrastive pre-training tasks for personalize expert finding. ", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "vtcMb3QzKi", "reviewer": "Reviewer_hASg", "summary": "", "questions": "1. It appears that the explanation for the necessity of personalized fine-tuning is insufficient.\n2. Shouldn't there be a comparison with existing data augmentation methods, even if it's limited to a small amount of data?\n3. It seems necessary to compare the proposed positive pair method with other unsupervised contrastive learning methods, such as SimCSE: Simple Contrastive Learning of Sentence Embeddings (EMNLP 2021).\n\n\n\n", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks a lot for all your meaningful comments.\n\n### R-1 and Q-A (explanation of personalized fine-tuning)\n\nThanks for your comment.\n\nGiven that experts often have distinct perspectives on the same question, it is impractical for different experts to learn identical representations for the same historical question (in the most extreme case, if two experts answer the same questions historically, the model will learn the same representation for the two experts). Furthermore, since our pre-training primarily concentrates on question modeling, it becomes essential to devise a personalized fine-tuning network to ensure that the learned embeddings are typically expert-specific.\n\n### R-2 and Q-B (comparison with existing data augmentation methods)\n\nThanks for your comment. \n\nIn fact, the data augmentation employed in our method does not involve manual data augmentation, since we can obtain the question titles and bodies naturally from the datasets. Then we take the question body as an augmented view of the question title. \n\nTo compare it with different augmentation methods, we **pre-train the model directly based on SimCSE and use SimCSE's data augmentation methods**.\nWe select two datasets to conduct the experiment, and the results are as follows,\n\n**Table 1.** Compared with Different Augmentation in SimCSE.\n|                 |MRR|P@3|NDCG@10|\n|--------------|-------|-------|---------------|\n|Electronics|0.5426|0.6303|0.6266|\n|Es|0.5078|0.5658|0.5601|\n\nWe can find that the performance is inferior to CPEF. We argue that the reason may be that although such enhancement methods have the potential to learn superior question embeddings, they are still not as effective as directly considering the question body as an enhanced representation of the question title, which is natural to introduce richer information.\n\nThese experimental results will be included in the final version of the paper.\n\n### R-3 and Q-C (compared with SimCSE)\n\nThanks for your comment.\t\n\nOur method concentrates more on question modeling while SimCSE focuses more on identifying different sentences. Furthermore, we argue that SimCSE, being a pre-trained model similar to BERT, could serve as the initialization weights for our task.\n\nWe **directly employ SimCSE** to evaluate expert finding performance. The experimental results are as follows:\n\n**Table 2.** SimCSE Experimental Results.\n|                 |MRR|P@3|NDCG@10|\n|--------------|-------|-------|---------------|\n|Electronics|0.5316|0.6078|0.5987|\n|Es|0.4701|0.5395|0.5453|\n\nBased on the results, we observe that the experimental results are similar to those obtained with the original BERT (Table 3 in Manuscript). This is attributed to both BERT and SimCSE could be considered as different methods of initialization pre-training weights. However, our method is a combination of question-level pre-training and expert-level personalized fine-tuning. Moreover, as SimCSE specifically emphasizes differentiating embeddings of different sentences during pre-training, it outperforms the original BERT.\n\nThese experimental results will be included in the final version of the paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. When comparing the experimental results of the proposed method with other baseline methods, the proposed method exhibits superior performance.\n2. The authors effectively performed question representation without manual tasks like labeling by introducing contrastive learning techniques to unlabeled data.", "reasons_to_reject": "1. It appears that the explanation for the necessity of personalized fine-tuning is insufficient.\n2. Shouldn't there be a comparison with existing data augmentation methods, even if it's limited to a small amount of data?\n3. It seems necessary to compare the proposed positive pair method with other unsupervised contrastive learning methods, such as SimCSE: Simple Contrastive Learning of Sentence Embeddings (EMNLP 2021).\n\n\n\n", "questions_to_authors": "1. It appears that the explanation for the necessity of personalized fine-tuning is insufficient.\n2. Shouldn't there be a comparison with existing data augmentation methods, even if it's limited to a small amount of data?\n3. It seems necessary to compare the proposed positive pair method with other unsupervised contrastive learning methods, such as SimCSE: Simple Contrastive Learning of Sentence Embeddings (EMNLP 2021).\n\n\n\n", "paper_topic_and_main_contributions": "This paper proposes techniques for improving question representation in the expert finding task through contrastive learning, and for finding experts that match personalized preferences by learning personalized expert representations.\n\nIn particular, it suggests viewing the question title and body as a positive pair, offering a solution to overcome the limitations of data augmentation methods, which could potentially be laborious work.\nFor personalized tuning, they proposed a mechanism grounded in attention-based processing of historically answered questions.\n\nThe authors assert the superiority and effectiveness of the proposed method based on the experimental results.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "Tianyu Gao, Xingcheng Yao, and  Danqi Chen, SimCSE: Simple Contrastive Learning of Sentence Embeddings, EMNLP 2021\n"}, "extra_text": {}}, {"review_id": "aRzR6yJvE2", "reviewer": "Reviewer_L924", "summary": "", "questions": "A. What exactly does \"expert ID\" refer to? Is it the username or something else? Furthermore, can the ID information truly represent the personal characteristics of the expert?\nB. Why are the lengths of the question body and question title set to 30 and 10 respectively, which are far from the maximum length of Bert(512 \ncharacters)? Is it possible to lose information if the length is too short? Is the length setting the same for different datasets?\nC. The experimental dataset is different from that mentioned in the comparison method paper, why can't choose the same batch of experimental data?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks a lot for all your positive and meaningful comments.\n\n### R-1 and Q-A (inadequate method details: the \"expert ID embedding\")\n\nThanks for your comment. \n\nThe expert ID embedding is obtained from the unique ID assigned to each expert. Due to the fact that experts may have different points of interest in the same question, it is unreasonable for different experts to learn the same representation for the same historical question (in the most extreme case, if two experts answer exactly the same question, the model will also learn the same representation for the two experts). Hence, it becomes essential to incorporate personalized information to ensure that the learned vectors are typically specific to each expert. One straightforward and effective approach utilizing the expert's unique ID is to generate a looking-up embedding table similar to word embedding in NLP.\n\nIn fact, similar operational methods have been employed in various fields like recommendation systems [1][2] and personalized text generation [3], providing evidence of the effectiveness of utilizing user-unique IDs to achieve more personalized and precise user modeling.\n\n- [1] Wu C, Wu F, An M, et al. NPA: neural news recommendation with personalized attention[C]//Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining. 2019: 2576-2584.\n- [2] Li L, Zhang Y, Chen L. Personalized prompt learning for explainable recommendation[J]. ACM Transactions on Information Systems, 2023, 41(4): 1-26.\n- [3] Xu H, Liu H, Lv Z, et al. Pre-trained Personalized Review Summarization with Effective Salience Estimation[C]//Findings of the Association for Computational Linguistics: ACL 2023. 2023: 10743-10754.\n\n### R-2 and Q-B (the length of the question body and title and the number of negative samples)\n\nThanks for your comment. \n\nThis is due to an oversight on our part that has led to your confusion. \n\n* For the length of the question title/body. \n\nIn our method, we consider the question title as the primary perspective of the question, and the question body as an augmented perspective. There is a long-tail distribution in the lengths of the question title and body. To maximize coverage of both titles and bodies while minimizing resource usage and training time, we set the title length to 10, which covers 80% of pre-processed titles, and the body length to 30, which covers 75% of pre-processed bodies.\nTo enable batch pre-training across various datasets, we standardize the lengths of the title/body in the different datasets.\n\n* For the negative samples.\n\nWe selected the same values as the previous paper PMEF [1], i.e., 1 positive sample and 19 negative samples, to ensure the fairness of the experimental comparison.\n- [1] Peng Q, Liu H, Wang Y, et al. Towards a multi-view attentive matching for personalized expert finding[C]//Proceedings of the ACM Web Conference 2022. 2022: 2131-2140.\n\nWe will revise the relevant statement in the final version to enhance the clarity of the paper.\n\n### R-3 (lack of fair comparison: compared with ExpertBERT)\n\nThanks for your meaningful comment. \n\nThe main purpose of this experimental setup is to demonstrate the effectiveness of pre-trained high-quality question representations.\n\nWe conduct similar experiments based on ExpertBERT to show the effectiveness of our method.\n\n**Table 1.** Experimental Results of ExpertBERT on the Physics Dataset.\n\n|Training Data Ratio|20%    |40%    |60%    |80%   |100%  |\n|-------------------------|---------|----------|---------|---------|---------|\n|P@3|0.3232|0.3908|0.4668|0.5366|0.5803|\n\nThe experimental results fall in between CPEF and PMEF. This is because ExpertBERT is a preliminary pre-trained model based on MLM, whereas CPEF incorporates the question body and employs contrastive learning tasks, which could help the model to learn enhanced question embeddings.\n\nThese experimental results will be covered in the final version.\n\n### Q-C (experimental dataset)\n\nThanks for your meaningful question.\n\nIn our method, a portion of our dataset overlaps with the dataset used in the previous paper (e.g., English, Electronics). StackExchange comprises a wide range of diverse datasets. The datasets compared in different papers lack consistency. We selected a subset of commonly used datasets based on existing papers, which may not encompass all available datasets. \n\nIf required, we can conduct additional experiments on relevant datasets.\n\n### Missing Reference\n\nThanks for the heads up.\n\nAs stated above, ExpertPLM is concerned with expert-level pre-training, whereas this paper is concerned with question-level pre-training.\n \nWe've evaluated the ExpertPLM performance and the experimental results are as follows:\n\n**Table 2.** ExpertPLM Experimental Results.\n|           |MRR|P@3|NDCG@10|\n|----------|-------|------|---------------|\n|English|0.4187|0.4833|0.5123|\n|Tex|0.6033|0.7023|0.6711|\n|Electronics|0.5435|0.6305|0.6252|\n\nBased on the results, we observe that ExpertPLM performs worse than CPEF. The possible reasons are as follows:\n1) Our model encompasses more comprehensive information, including the utilization of question body information, whereas ExpertPLM does not utilize the question body information.\n2) Precisely modeling questions is the foundation of precisely modeling experts, as expert embedding learning heavily relies on her/his historicalal answered questions.\n\nThese experimental results and analyses will be included in the final version.\n\nThank you for your suggestions regarding the presentation and writing of our paper. We will conduct a thorough proofreading and have invited a native English speaker to assist in polishing the writing.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1) The proposed  title-body contrastive pre-training task  and the personalized tuning network sound reasonable and innovative\n2) The performance comparison is well-executed as it involves multiple baselines compared across six datasets, making the results highly convincing.", "reasons_to_reject": "1) Inadequate method details: The \"expert ID embedding\" mentioned in Section 4.3 is somewhat confusing as it lacks specific clarification.  It remains unclear whether this ID refers to the registered name of the expert or some other form of identification.  If it simply represents the expert's registered name, its ability to capture the personalized characteristics of the expert is questionable.\n2) Insufficient experiments: The \"Further Analysis\" section of the paper and the experiments of \"ExpertPLM: Pre-training Expert Representation for Expert Finding (https://aclanthology.org/2022.findings-emnlp.74.pdf)\" share significant similarities. However, there is a lack of experimental analysis specific to certain parameters within the paper itself, such as the length of the question body and title and the number of negative samples (K value).\n3) Lack of fair comparison: In Figure 3, the authors compared CPEF with PMEF to demonstrate the advantages of the pre-trained question representation model under data scarcity conditions (line 529-lin534). However, emphasizing the advantages of CPEF through this comparison is unjust since PMEF lacks a pre-training module. To ensure fairness, it is recommended to compare CPEF with another pre-trained model, such as ExpertBert, to showcase the advantage of the innovative pre-training module design of CPEF.\n", "questions_to_authors": "A. What exactly does \"expert ID\" refer to? Is it the username or something else? Furthermore, can the ID information truly represent the personal characteristics of the expert?\nB. Why are the lengths of the question body and question title set to 30 and 10 respectively, which are far from the maximum length of Bert(512 \ncharacters)? Is it possible to lose information if the length is too short? Is the length setting the same for different datasets?\nC. The experimental dataset is different from that mentioned in the comparison method paper, why can't choose the same batch of experimental data?", "paper_topic_and_main_contributions": "Since existing PLMs are usually  pre-trained on general corpus, directly fine-tuning the existing PLMs with the Expert Finding task may not be the most optimal approach for Community Question Answering (CQA).  Furthermore, such architectures fail to account for the personalized characteristics of experts.\nTo solve the aforementioned problems, the paper designs a novel expert finding framework. The main contributions include: \n1) designing a novel question title-body contrastive pre-training task, which could improve question modeling effectively,\n2) adopting a personalized tuning network to learn personalized representations of experts.\n", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "ExpertPLM: Pre-training Expert Representation for Expert Finding (https://aclanthology.org/2022.findings-emnlp.74.pdf)"}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "1) It is suggested to replace \"comparative learning\" on line 487 with \"contrastive learning\", which is more in line with the academic expression.\n2) There are  inconsistent format settings for \"log\" in line 295 and line 307, it is suggested to unify after checking.\n"}}]

Author rebuttal:

Meta-review: This submission studies the Community Question Answering (CQA) problem and adopts the pre-training technique to enhance the representation of questions and answers. In particular, this submission adopts a contrastive learning with some specific design to inject personalized preferences. Nevertheless, the technical novelty is limited and lacks experiments beyond the StackExchange platform. In addition, the performance improvement in many cases is marginal. Besides, the writing of this submission should be improved.
Decision: Accept-Findings