Distantly Supervised Relation Extraction in Federated Settings
Authors: Dianbo Sui
Yubo Chen
Kang Liu
Jun Zhao
Venue:  (2021)
Keywords: Distant Supervision
Relation Extraction
Federated Learning

Abstract: Distant supervision is widely used in relation extraction in order to create a large-scale training dataset by aligning a knowledge base with unstructured text. Most existing studies in this field have assumed there is a great deal of centralized unstructured text. However, in practice, text may be distributed on different platforms and cannot be centralized due to privacy restrictions. Therefore, it is worthwhile to investigate distant supervision in the federated learning paradigm, which decouples the training of the model from the need for direct access to the raw text. However, overcoming label noise of distant supervision becomes more difficult in federated settings, because the sentences containing the same entity pair scatter around different platforms. In this paper, we propose a federated denoising framework to suppress label noise in federated settings. The core of this framework is a multiple instance learning based denoising method that is able to select reliable sentences via cross-platform collaboration. Various experimental results on New York Times dataset and miRNA gene regulation relation dataset demonstrate the effectiveness of the proposed method.

Under review as a conference paper at ICLR 2021
DISTANTLY SUPERVISED RELATION EXTRACTION IN FEDERATED SETTINGS
Anonymous authors Paper under double-blind review
ABSTRACT
Distant supervision is widely used in relation extraction in order create large- scale training dataset by aligning a knowledge base with unstructured text. Most existing studies in this field have assumed there is great deal of centralized un- structured text. However, in practice, text may be distributed on different plat- forms and cannot be centralized due to privacy restrictions. Therefore, it worth- while to investigate distant supervision in the federated learning paradigm, which decouples the training the model from the need for direct access to the raw text. However, overcoming label noise of distant supervision becomes more difficult in federated settings, because the sentences containing the same entity pair scatter around different platforms. In this paper, we propose federated denoising frame- work to suppress label noise in federated settings. The core of this framework is multiple instance learning based denoising method that is able to select reliable sentences via cross-platform collaboration. Various experimental results on New York Times dataset and miRNA gene regulation relation dataset demonstrate the effectiveness of the proposed method.
INTRODUCTION
Relation extraction (RE) aims to mine factual knowledge from free text by labeling relations between entity mentions, which is a crucial step in knowledge base (KB) construction. For example, given sentence "[Steve Jobs] and Wozniak co-founded [Apple] in 1967", a relation extractor should identify that "Sleve Jobs" and "Apple" are a "Founder" relationship.
Most existing supervised RE systems, such as Zeng et al. (2014); Zhang & Wang (2015); Wang et al. (2016); Zhou et al. (2016), rely on large-scale manually annotated training dataset, which is extremely expensive and cannot cover all walks of life. To ease the reliance annotated data, Mintz et al. (2009) proposed distant supervision to automatically generate training data by heuristically aligning a KB with unstructured text. The key assumption of distant supervision is that if two entities have a relation in the KB, then all sentences that mention these two entities will express this relation. Since then, there has been rich literature devoted to this topic, such as Riedel et al. (2010); Hoffmann et al. (2011); I); Zeng et al. (2015); Lin et al. (2016); Ye & Ling (2019); Yuan et al. (2019).
Though the progress is exciting, distant supervision approaches have so far been limited to the centralized learning paradigm, which assumes that great deal of text is easily accessible. However, in practice, text may distributed on different platforms and massively convoluted with sensitive personal information, especially in the healthcare and financial fields (Yang et al., 2019; Zerka et al., 2020: Chamikara al., 2020). Due privacy restrictions, is almost impossible or cost- to centralize text from multiple platforms. Recently, federated learning (McMahan et al., 2016) provides a compelling solution for learning model from decentralized and privacy-sensitic data. The main idea behind federated learning is that each platform trains local model based on its own local data and a master server coordinates massive platforms to collaboratively train a global model by aggregating these local model updates.
Unfortunately, directly applying federated learning to the decentralized distantly supervised data fails, because conventional federated learning requires the local data to come with labels without noise (Tuor et al., 2020), however, in distant supervision. automatic labeling inevitably accompanies with label noise (Riedel et al., 2010; Hoffmann et al., 2011; Zeng al., 2015; Lin et al., 2016),

---

Under review as a conference paper at ICLR 2021
which means not all sentences that mention entity pair can represent the relation between them. Training on such noisy data will substantially hinder the performance of the RE model.
Moreover, even involving previous denoising methods, such as Zeng et al. (2015); Lin et al. (2016); Ye & Ling (2019), cannot handle la- bel noise well in federated settings. This point can be illustrated by the example in Figure 1. Si and S2 contain the same entity pair ("Steve Jobs", "Apple") ") but are distributed on two platforms. S1 is true positive while S2 is false positive instance, which does not express the "founder" relation. In centralized training, there is no barrier between Platform and Plat- form 2; therefore, simultaneously considering S1 and S2 can easily filter out noise via only selecting S1 (Zeng et al., 2015) placing small weight on S2 (Lin et al., 2016; Ye Ling, 2019). However, raw data exchange between platforms is prohibited in federated settings. Due to the lack of comparison with S1, previous de- noising methods would mistakenly regard S2 as a true positive instance. As result. S2 is retained and then poisons the local model in platform 2, which would affect the global model in turn.
Steve- Johs
Apple
Fomnderr
Sj: Steve Jobs and Wozniak co-founded Apple 1967.
Sy: Steve lobs resigneid as chief executive from Apple
2011.
Platform
Platform 2
An example of the sentences that con- tain the same entity pair distributed on two plat- forms. The triple (Steve Jobs, Founder, Apple) is a fact the KB
Figure I:
To suppress label noise in federated settings, we propose a federated denoising framework in this paper. The core of this framework is multiple instance learning (MIL) (Dietterich et al., 1997; Maron & Lozano-Péree Pérez, 1998) based denoising algorithm, called Lazy MIL, which is only exe- cuted at the beginning of each communication round and then would rest until the next round. Since the sentences containing the same entity pair scatter around different platforms, Lazy MII. algo- rithm coordinates multiple platforms to jointly select reliable sentences. Once sentences have been selected, they would be used repeatedly to train local models until the end of this round.
In summary, the contributions of this paper are:
Considering data decentralization and privacy protection, we investigate distant supervision under the federated learning paradigm, which decouples the model training from the need for direct access to the raw data. To our best knowledge, combining federated learning with distant supervision is still an unexplored territory, which is the main focus of this paper.
Since the automatic labeling in distant supervision inevitably accompanies with label noise, we present a multiple instance learning based denoising method, which can select reliable instances via cross platform collaboration.
The proposed method yields promising results on two benchmarks datasets, and we perform various experiments to verify the effectiveness of the proposed method. The code will be released at http:/ http::://anoonae zed.
2 RELATED WORK
In this section, we will briefly review the recent progress in distant supervision and some existing studies in federated learning.
Distant supervision. Relation extraction is task of mining factual knowledge from free text by labeling relations between entity mentions. To alleviate the dependence of supervised methods on annotated data, Mintz et al. (2009) proposed distant supervision by using knowledge base to annotate a large-scale dataset automatically. However, automatic labeling inevitably accompanies with label noise. To deal with label noise, most distantly supervised approaches (Riedel et al., 2010; Hoffmann et al., 2011: I; Surdeanu et al., 2012; Zeng et al., 2015; Lin et al., 2016; Luo et al., 2017: Ye & Ling, 2019; Yuan et al., 2019) focus on reducing label noise at bag level prediction. These studies fall under multiple instance learning framework, which assumes that at least one sentence expresses the relation in bag. Another line of work aims reduce label noise sentence level prediction.
A set of sentences containing the same entity pair is called "bag"
2

---

Under review as a conference paper at ICLR 2021
These studies (Zeng et al., 2018; Feng et al., 2018; Qin et al., 2018a;b) use reinforcement learning or adversarial training to select trustable relation labels by matching the predicted labels with distantly supervised labels. Compared with previous studies, our work focuses reducing label noise at bag level prediction and extends distant supervision to federated settings.
Distant supervision has also been applied other natural language processing tasks, such named entity recognition (Ghaddar& & Langlais, 2018; Shang et al., 2018; Nooralahzadeh et al., 2019), event extraction (Chen et al., 2017), sentiment classification (Go et al., 2009) and question answer (Joshi et al., 2017; Lin et al., 2018; Cheng et al., 2020).
Federated Learning. Recently, federated learning (McMahan al., 2016; Konečný al., 201 6a;b; Bonawitz et al., 2017; Smith et al., 2017: Caldas et al.. 2018: Zhao et al.. 2018: Li et al., 2018; Jeong al., 2018: Peng al., 2019; Li et al., 2019; Wang et al., 2020; Rothchild et al., 2020; Yu al., 2020) has become rapidly developing topic in the research community. since it provides new communication-effiicen
way of learning a model over a collection of highly distributed platforms while still preserving data privacy. However, most of the previous studies require the data stored by the local platforms to come with ground-truth labels without noise. The problem of how to adapt federated learning to noisy environment is relatively ignored. In terms of overcoming noise in federated settings, Tuor et al. (2020) is most relevant to our work but require a clean benchmark dataset to train a benchmark model. Compared with Tuor et al. (2020), our work does not rely clean benchmark dataset, which does not exist in distant supervision.
FEDERATED DENOISING FRAMEWORK
3.1 TASK DEFINITION
In this paper, we focus on distant supervision in federated settings. Define K platforms (PI,...P'' with respective unlabeled corpora {D1....De Under the assumption of centralized training, each platform transfers or shares its local corpus to server, and the server will take the integrated corpus D = D1 U ... U Dx to conduct training, while the task of distant supervision requires platform P. does not expose its corpus D; to others (including the server). In distant su- pervision, a KB is required to automatically label these corpora. In this paper, we only focus on the data security of these unlabeled corpora and assume the KB is publicly available for all platforms. The issue of protecting the security of KB is beyond the scope of the current work.
federated settings
To solve this task, we propose federated denoising framework. The key components of this frame- work will be claborated in the following section. Concretely, we firstly introduce the basic relation extractor in Section 3.2, which is the network architecture shared by the global model and local models. Then, we present how to select reliable instances via cross-platform collaboration in Sec- tion 3.3. Next, we deseribe how use the selected instances to train the local model in Section 3.4. Finally, we present how to use the FedAvg algorithm to update the global model in Section 3.5.
3.2 RELATION EXTRACTOR
Following previous studies (Zeng et al., 2015), adopt the Piecewise Convolutional Neural Net- work (PCNN) as our relation extractor. Given a sentence s and two entities within this sentence, we first split the sentence into tokens, and then each token w; is mapped into a dense word embedding e, E Rdd.. To specify the entity pair, relative distances between the current token wi and the two entities are transformed into two positional features by looking up the position embedding matri- ces. Next, the token is represented as the concatenation of the word embedding and two positional features, and is fed into convolutional neural network. Then, piecewise max pooling (Zeng et al., 2015) is employed to extract the high-level sentence representation. In the piecewise max pooling, an input sentence is divided into three segments based on the two entities, and the maximum value of CNN outputs in each segment is returned. After that, we apply single fully connected layer to output the logit value o. Finally, the conditional probability of j-th relation is denoted as follows: exp(o;)
p(relg|s, θ) M
(1)
Σ exp(ox)
where θ is the model parameter and M is the total number of relation.
3

---

Under review as a conference paper at ICLR 2021
Algorithm Lazy Multiple Instance Learning
Input: global model parameters θ, the set of activated platforms A.
2: Define two dictionary on the server, named V and
Run on the master server
3: Distribute θ to each platform in
4: for each platform i E A in parallel do
> Run on the activated platforms
5: for each triple (h, r, 1) KB do
6: for each sentence sf. in the bag by do Compute pr(ss 1, ④)
7:
D According to Equation
8:
v², id' <- max; (p(rls", 0)), E
D v² is called uploaded value V[ r, t)]
9:
Upload [u*, åd', i] the server and append [o', idd,
10: for each key (h, r. () in V do
>> Run on the master server
Sort V[(h, r, in descending order according the uploaded values. 12: [[(,, r, f)] + V[(h, r, t)][O]
13:
Broadcast / to each platform in A
3.3
LAZY MULTIPLE INSTANCE LEARNING
To avoid the local relation extractor being poisoned by false positive instances, we propose lazy multiple instance learning (Lazy MIL), which can select reliable instances via cross-platform col- laboration. The overview of Lazy MIL. is illustrated in Algorithm
Suppose that there is triple (h, t) in the public KB, the set of sentences containing the head entity h and tail entity /. is represented i-th instance in the platform j. In the g-th communication round, assume that only platform and platform are activated. At the beginning of this round, the parameters of the global model O, are distributed to the activated platforms and for initializing local models, which ensures that all activated local models share the same parameters in Lazy MIL. In platform the sentences in the set (s1, s ..., sh,)) are fed into the local model to get conditional probabilities associated with the relation r according to Equation 1, where r is the predicate of the triple. The value v² and index id' of the instance with the maximum conditional probability associated with the relation are computed as follows:
{(1] s2..., 87, .... (s 8 .... s ). where sf indicates the
u¹, id' maxd=maxxplsssscco 0g)) 1525m.
After computation, platform uploads the value of and index id' to the master server. the same time, the same procedure is performed on platform j. and the value of and index id are also up- loaded to the server.
The master server decides which local instance can be selected among all activated platforms based on the uploaded values. v² > vi, then the id'-th sentence in platform is selected as the reliable sentence that expresses the triple (h f) in this round. This decision, called denoising information, broadcast to all activated platforms. Each activated platform selects reliable training instances from its local corpus according to this denoising information. Note that since only values and indices of conditional probabilities are uploaded to the master server, Lazy MIL almost does not leak the corpus information in each platform.
3.4 LOCAL MODEL TRAINING
After platform i selects reliable instances from its local corpus D., the selected reliable instance set D used for training the local relation extractor. We use the cross-entropy loss function to optimize parameters Og, which is defined as follows:
|
J(e,; = D
1
Σ
log p(rulls,, θ₂)
(3)
where s indicates the u-th sentence in the selected reliable instance set D After training E epochs on the selected reliable instance set, the trained parameters O 01+1 are uploaded to the master server, where the superscript indicates the parameters trained platform
4

---

Under review as a conference paper at ICLR 2021
Algorithm 2 Federated Denoising Framework
Hyperparameters: K is the total number of platforms: C is the fraction of platforms; B is the local minibatch size; E is the number local epochs; 17 is the learning rate.
2: Master server executes:
3: Initialize Ou
4: for communication round = 0, do
5:
< max(C' x K, 1)
Select activated platforms
6: Ag < (random set of m platforms)
7:
Execute lazy multiple instance learning algorithm for each platform Ag in parallel do
Defined in Algorithm
8:
9:
+ 9++1 f LocaalTrainin((. )
10:
[2; gq41 + EiE,, Σ,er, 1o 4+1
Defined in Equation
12: Function Local_ Training(i, +)
Run on platform
Generate denoised dataset D from Dj based on the denoising information / B + (split D into batches of size B) for each local epoch e from to E do for batch b E B do
14:
16:
((;
J is defined in Equation
18:
return O to the master server
3.5
GLOBAL MODEL UPDATE
Suppose A, is the set of activated platforms in the q-th communication round. After all activated platforms finish local training, the master server collects all trained parameters {or (0/ E A&) to update the global model. We define the goal of the global model as follows:
DD Σ D JEAy
min Σ By
J(9g: D1)
EEAr
where J(q D is the local loss function for the platform i. Follow previous studies (McMahan al, 2016), optimize this global objective function via taking the weighted average of all trained parameters, which is shown as follows:
DJ D2
1+ Σ
O
(5)
Σ
SEAr JEA,
where O - the optimal parameters obtained by minimizing the local loss function the local data of platform i. Since all trained parameters from different platforms are aggregated together, the corpus information of each platform is hard be inferred. Thus, corpora in platforms well- protected. Complete pseudo-code of this framework given Algorithm 2.
4 EXPERIMENTS
4.1 DATASETS AND EVALUATION METRICS
Since conducting experiments on non-public privacy-sensitte datasets is not reproducible, we choose public distantly supervised relation extraction datasets to investigate the effectiveness of the proposed framework.
(Riedel et al., 2010) is widely used dataset in distant supervision. It automatically generated by aligning the semantic triples in Freebase with the New York Times corpus. The training set contains 466,876 sentences. 251,928 entity pairs and 16,444 relational facts. The development set contains 55167 sentences, 28077 entity pairs and 1,808 relational facts. The test set contains
NYT 10
Phttps://git thub. wtt.tttt......ttttteeteetee www.//ttunn////ee/eeeeeee

---

Under review as a conference paper at ICLR 2021
172,448 sentences, 96,678 entity pairs and 1,950 relational facts. There are 52 actual relations and a special relation NA for representing no relation between two entities.
MIRGENE' (Li et al., 2017) is large-scale biomedical dataset. This dataset is generated by align- ing Tarbase and miRTarBase with the abstracts in Medline. An example is shown in the following: "MicroRNA-2 regulates FOXO1 expression and cell proliferation", where MicroRNA-22 miRNA and FOXO1 is a gene. There are 172727 sentences in the training set and 1239 sentences in the test set. We randomly sampled 10% of the data from the training set as the development set.
is
Data Partitioning. To study distant supervision in federated settings, we need to specify how to distribute the data across platforms. In this paper, focus the IID situation, where the training data are shuffled and then partitioned into K (the total number of platforms) platforms.
Evaluation Metries. We evaluate our approach and baseline methods on the held-out test set of these two datasets. Precision-recal (PR) curves, area under curve (AUC) values and Precision@)) (P@N) values are adopted as evaluation metrics in our experiments.
4.2 EXPERIMENTAL SETTINGS
For a fair comparison, we implement our method and all baselines in the same experi- mental settings. We divide the hyperparame- ters into three parts, i.e., fixed hyperparame- ters, unfixed hyperparameters and federated hy- perparameters. Fixed hyperparameters follow the hyperparameter settings in Lin et al. (2016), including the 50-dimensional pretrained word embeddings for NYT, the 5-dimensional po- sition embeddings, and CNN module that in- cludes 230 filters with a window size of 3. For MIRGENE, 200-dimensional word embed-
Hyperparameter
Search Space
Learning Rate (1) Learning Rate Decay
0.05, 0.08, 0.1,0.2 0.01, 0.05
Dropout Weight Decay
0.1, 0.2, 0.5
1e-5, 1e-6
Table 1: The search space of unfixed hyperparam- eter.
dings pretrained on PubMed and MIMIC-III are used. The optimal unfixed hyperparameters are determined by grid search based on the performance of the development set, and the search space of unfixed hyperparameters is shown in Table 1. Federated hyperparameters include the total number of platforms K, the fraction of platforms C, the local minibatch size B, the number of local epochs E. All of these control the amount of computation. In the end-to-end comparison, we fix the K to 100, B to 32, E to 3, and set the hyperparameter space C' as {0.1, 0.2, 0.5, 1} following McMahan et al. (2016). We stochastic gradient descent as the local training optimizer and all experiments can be done by using a single GeForce GTX 1080 Ti.
4.3 BASELINES
We compare our method with the following baselines in federated settings: (1) Directly applying FedAvg algorithm (McMahan et al., 2016) the automatically labeled data is the first baseline, which is called NONE. In this case, there is no denoising module this method. (2) Zeng et al. (2015) proposed to leverage multiple instance learning to choose the most reliable sentence as the bag representation, and we abbreviate this method as ONE; (3) ATT was proposed by Lin et al. (2016), which uses the attention mechanism to select reliable instances by placing soft weights on set of noisy sentences; (4) AVE (Lin et al., 2016) is naive version of ATT and represents each sentence set as the average vector sentences inside the set; (5) ATT RA (Ye & Ling, 2019) is variant of ATT, which calculates the bag representations in relation-aware way. The federated framework of these baselines is shown in Algorithm 3 in the appendix.
4.4 MAIN RESULTS
Figure 2 and Figure show the precision-recall curves on NYT dataset and MIRGENE datasets. At the appendix, we also present AUC values of these curves in Table 8 and detailed precision val- ues measured at different points along these curves in Table 9. To reduce randomness, we run 10
httpp:://// httpgtttc..../eeeeee/ee///ieeeee

---

Under review as a conference paper at ICLR 2021
Cm 6.1
C 0.2
C 0.5
C=
Thi york
115 woch
Thoewarc
0.2
NOVE DAE
DAE
KOME
ONT
KOME
011
AVC
0.7
0.3
Figure 2: Aggregate precision-recall curves NYT 10 dataset, where C is the fraction of platforms that are activated on each round.
C=1
work
wiR
The werk MONT DVE
MONT
NOM:
ATT
ATT AVE AT
AVE
AVE
ATT.AA
MT
2.85
0.80
0.6
6.1
0.2
53 Recal
9.4
05
0.6
0.0
02
6.3
3.6
0.5
60
0.1
3.2
6.3
0.4
355
06
0.0 0.1
3.2
6.3 04
0.5
0.6
local
Recal
Recall
Figure 3: Aggregate precision-recall curves on MIRGENE dataset, where C' the fraction of plat- forms that are activated on each round.
AUC
NONE
ONE
ATT
AVE
ATT_RA
Ours
C=0.1
0.128710.0034
0.171910.0000
0.16381000000
0.152110.0029
0.1664 10.0026
0.218911..0025
C=0,2
0.1255:1.0032
0.171010.0029
0.16301:0.0028
0.1517+0.0027
0.1642=0.0022
0.2085:1000023
C=0.5
0.1239=00.0045
0.1701=0.0020
0.1619=000025
0.1513=00.024
0.1630=0.0020
0,2420=0.000.
C=1.0
0.1223110.0337
0.168910.0021
0.160410.0022
0.149110.0015
0.1625-000022
0.244710.0001
Table 2: AUC values NYT 10 dataset. We run 10 models using random seeds with early stopping on the development set, and report mean and standard deviation of test AUC values for methods.
AUC
NONE
ONE
ATT
AVE
ATT RA
Ours
C=0.1
0.73164 0.0069
0.7665=20.0877
0.7535+: 0.0062
0.7499-0.0005
0.7514= 0.0053
0.7846:10.0066
C=0.2
0.7047.10.0047
0.7610.10.000 0.0092
0.7472.10.008
0.7428 0.0052
0.7431 10.0071
37897.10.000
C=0.5
0.7251110.054
0.7605110.000
0.7453100.005.
0.700910.0062
0.7423 10.0079
0.7915.10.006
C=1.0
0.7229 0.0059
0.755910.0080
0.7424 10.0067
0.73681200003
0.73951000002
0.7942-10.0060
Table 3: AUC values on MIRGENE dataset. We run 10 models using random seeds with early stopping on the development set, and report the mean and standard deviation of test AUC values for all methods.
models using random seeds with early stopping on the development set. Table 2 and Table 3 show the mean and standard deviation test AUC values for each method on NYT 10 dataset and MIR- GENE dataset, respectively. We find that: (1) Our method significantly outperforms all baselines in federated settings. We believe the reason is that our denoising method can use cross-plas tross-platform better
information to hinder false positive instances from poisoning local models, which leads performance of the global model. (2) Directly applying FedAvg algorithm (McMahan et al., 2016) to the automatically labeled data achieve the worst results in both datasets. The reason behind that
7

---

Under review as a conference paper at ICLR 2021
training the noisy data will substantially hinder the performance of the model. Therefore, it necessary to conduct denoise in federated distant supervision. (3) C' the fraction of platforms that activated each round, which controls the amount of multi-platform parallelism. With increasing platform parallelism, the performance of all haselines declines slightly while our method performs better. Intuitively, increasing platform parallelism is able lead to better results, since involving more platforms in training can increase the likelihood that all sentences with the same en- tity pair appear simultaneously. However, due lack of cross-platform collaboration, all baselines handle label noise only based on its own local data, which may hamper the performance. In con- trast, our method selects reliable instances among all activated platforms, which can effectively reap the benefits of increasing platform parallelism. (4) Leveraging attention mechanisms to denoise, an effective solution in centralized settings, seems not work federated settings. Compared with centralized training, the sentences in bag scatter around different platforms in federated settings, so the number of the sentences with the same entity pair on a platform is small, which may lead to placing large attention weights noisy sentences due to lack of inter-bag contrast.
4.5 INCREASING THE SIZE LOCAL DATASETS
In this section, increase the size of local datasets by setting K to 50. In such way, each local dataset is twice large it was (when K is set 100). For fair parison, we fix 4 show the results of AUC values. At the appendix, we also present corresponding precision-recall curves in Fig- and detailed precision values measured at different points along these curves in Table 10. From the results, we observe that: (1) Our proposed method significantly (2) Compared with setting K 100, the result of directly applying Fe- dAvg algorithm (McMahan et al., 2016) to the automati- cally labeled data remains almost unchanged when is set to 50. (3) As the size of local datasets increases, all denoising methods can achieve better results. The most likely reason is that compared with setting K to 100, setting K to 50 increases the probability that all sentences with the same entity pairs simultaneously exist in the same platform.
AUC
NYT
MIRGENE
0.1, B =
32 and E = 3. Figure
NONE
0.1325
0.7430
ONE
0.1856
0.7786
ATT
0.1806
0.7726
AVE
0.1687
0.7592
ATT ATT_RA
0.1842
0.7639
Ours
0.2285
0.7941
surpasses all baselines in both datasets.
Table 4: AUC values on NYT 10 dataset and MIRGENE dataset when K 50.
4.6 INCREASING THE NUMBER OF LOCAL UPDATES
THi work
ATT BA
2.35
5-22
5-23
0-22
3-30
3-30
D-30
2.08
Figure 4: AUC values vs. communication rounds NYT data with different E (the number of local epochs) and B (the local minibatch size).
In this section, we investigate the impact of varying the number of local updates in this section. The number of local updates is given by E |Dj| where |D; is the size of the denoised dataset in platform i at a round, B is the local minibatch size and is the number of local epochs. Increasing B, decreasing F, or both will reduce computation each round. We fix C to 0.1 and only B and
8

---

Under review as a conference paper at ICLR 2021
E are varied in this section The results are shown in Figure 4. We find that: (1) Compared with the other denoising baselines, our method converges faster to the optimal results. We conjecture that is due that the proposed denoising method can effectively filter out the noise, which makes the relation extractor less affected by false positive instances and converge faster. (2) When setting B to 64 and E to 1, method achieves the best AUC value. (3) Increasing the local minibatch B may improve extraction performance. (4) Increasing the local epoch E' can speed up converge, but may not make the global model converge a higher level of AUC value. These findings are in line with McMahan et al. (2016), which shows it may hurt performance when we over-optimize on the local dataset.
5 CONCLUSION
Considering data decentralization and privacy protection, we investigate distant supervision under the federated learning paradigm, which permits learning to be done while data stays in its local envi- ronment. To suppress label noise in federated settings, we propose federated denoising framework, which can select reliable instances via cross pooss-plttfomm collaboration. Extensive experiments on two datasets have demonstrated the effectiveness of our method.
Distant supervision in federated settings is far from being solved and this work is just the beginning. There are still many problems need to be solved, such noisy bag problem (Xu al., 2013; Liu et al., 2017) and shifted label problem (Ye et al., 2019). Noisy bag problem means that all sentences containing the same entity pair are incorrectly labeled, and shifted label problem means the label distribution of training set does not align with that of test set. In federated settings, how these problems affect the relation extractor still unknown. In our future work, we will devote solve these problems in federated settings.
REFERENCES
Keith Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, H Brendan McMahan, Sarvar Patel, Daniel Ramage, Aaron Segal, and Karn Seth. Practical secure aggregation for privacy- preserving machine learning. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, 2017.
Sebastian Caldas, Jakub Konečný, H Brendan McMahan, and Ameet Talwalkar. Expanding the reach of federated learning by reducing client resource requirements. arXiv preprint 2018
M. A. P. Chamikara, P. Bertok, I. Khalil, D. Liu, and S. Camtepe. Privacy preserving distributed machine learning with federated learning. 2020.
Yubo Chen, Shulin Liu, Xiang Zhang, Kang Liu, and Jun Zhao. Automatically labeled data genera- tion for large scale event extraction. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, 2017.
Hao Cheng. Ming- Wei Chang. Kenton Lee, and Kristina Toutanova. Probabilistic assumptions matter: Improved models for distantly-superviseed document-level question answering. In Pro- ceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint 3rXiv:1810.048/0 2018.
Thomas G Dietterich, Richard H Lathrop, and Tomás Lozano-Pérez. Solving the multiple instance problem with axis axis-parallel rectangles. Artificial intelligence, 1997.
Jun Feng. Minlie Huang, Li Zhao, Yang Yang, and Xiaoyan Zhu. Reinforcement learning for relation classification from noisy data. In Shirly-Second AAAI Conference on Artificial Intelligence, 2018 88.
"The Ir, Ir decay, weight decay and dropout are fix to is 0.1, 0.01, 1e-5 and 0.1 respectively, which are not the optimal hyperparameters for most experiments

---

Under review as a conference paper at ICLR 2021
Abbas Ghaddar and Phillippe Langlais. Robust lexical features for improved neural network named- entity recognition. In Proceedings of the 27th International Conference on Computational Lin- guistics, August 2018.
Alec Go, Richa Bhayani, and Lei Huang. Twitter sentiment classification using distant supervision. CS224N Project Report, Stanford, 2009.
Ian Goodfellow, Mehdi Mirza, Da Xiao, Aaron Courville, and Yoshua Bengio. An empiri- cal investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211, 2013.
Raphael Hoffmann, Congle Zhang, Xiao Ling, Luke Zettlemoyer, and Daniel S Weld. Knowledge- based weak supervision for information extraction of overlapping relations. In Proceedings of the 49th Anmual Meeting of the Association for Computational Linguistics: Human Language Technologies. 2011.
Eunjeong Jeong. Seungeun Oh, Hyesung Kim, Jihong Park, Mehdi Bennis, and Seong-Lyur Kim. Communication--fficienn on-device machine learning: Federated distillation and augmentation under non-iid private data. arXiv preprint arXiv:1811. 1479, 2018.
Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension. In Proceedings the 55th Annual Meet- ing of the Association for Computational Linguistics, 2017.
Jakub Konečný, H Brendan McMahan, Daniel Ramage, and Peter Richtárik. Federated optimization: Distributed machine learning for on-device intelligence. arXiv preprint arXiv: 2016a.
Jakub Konečný, H Brendan McMahan, Felix X Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon. Federated learning: Strategies for improving communication efficiency. arXiv preprint arXiv:/610.05497 2016b.
Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jae- woo Kang. BioBERT: pre-trained biomedical language representation model for biomedical text mining. Bioinformatics, 2019.
Gang Li, Cathy Wu, and K. Vijay-Shanker Noise reduction methods for distantly supervised biomedical relation extraction. In BioNLP 2017, 2017.
Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith.
Federated optimization in heterogeneous networks. arXiv preprint arXiv:1812.06127 2018
Tian Li, Maziar Sanjabi, Ahmad Beirami, and Virginia Smith. Fair resource allocation in federated learning. arXiv preprint 1rXiv: 2019.
Yankai Lin, Shiqi Shen, Zhiyuan Liu, Huanbo Luan, and Maosong Sun. Neural relation extrac- tion with selective attention over instances. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, 2016.
Yankai Lin, Haozhe Ji, Zhiyuan Liu, and Maosong Sun. Denoising distantly supervised open- domain question answering. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguisties, 2018.
Tianyu Liu, Kexiang Wang, Baobao Chang, and Zhifang Sui. A soft-label method for noise-tolerant distantly supervised relation extraction. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 2017.
Bingfeng Luo, Yansong Feng, Zheng Wang, Zhanxing Zhu, Songfang Huang, Rui Yan, and Dongyan Zhao. Learning with noise: Enhance distantly supervised relation extraction with dynamic tran- sition matrix. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, 2017.
Oded Maron and Tomás Lozano-Pére A framework for multiple-instance learning. In Advances in neural information processing systems, 1998.
10

---

Under review as a conference paper at ICLR 2021
Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, volume 24, pp. 109-165. Elsevier, 1989.
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, et al. Communication learning of deep networks from decentralized data. arXiv preprint arXiv:1602.05622, 2016.
Mike Mintz, Steven Bills, Rion Snow, and Dan Jurafsky. Distant supervision for relation extraction without labeled data. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, 2009.
Farhad Nooralahzadch. Jan Tore Lønning, and Lilja Øvrelid. Reinforcement-bart based denoising of distantly supervised NER with partial annotation. In Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resource NLP (DeepL.o 2019), 2019.
Xingchao Peng, Zijun Huang. Yizhe Zhu, and Kate Saenko. Federated adversarial domain adapta- tion. arXiv preprint arXiv: /91 I. 1.02054, 2019.
Pengda Qin. Weiran Xu, and William Yang Wang. Dsgan: Generative adversarial training for distant supervision relation extraction. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 2018a.
Pengda Qin, Weiran Xu, and William Yang Wang. Robust distant supervision relation extraction via deep reinforcement learning. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 2018b.
Sebastian Riedel, Limin Yao, and Andrew McCallum. Modeling relations and their mentions with- out labeled text. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 2010.
Daniel Rothchild, Ashwinee Panda, Enayat Ullah, Nikita Ivkin, Ion Stoica, Vladimir Braverman, Joseph Gonzalez, and Raman Arora. Fetchsgd: Communication-efficieent federated learning with sketching. arviv preprint arXiv:2007.07682, 2020.
Rico Sennrich, Barry Haddow, and Alexandra Birch. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp.
2016.
Jingbo Shang, Liyuan Liu, Xiaotao Gu, Xiang Ren, Teng Ren, and Jiawei Han. Learning named en- tity tagger using domain-specinf dictionary. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 2018.
Virginia Smith, Chao-Kai Chiang, Maziar Sanjabi, and Ameet S Talwalkar. Federated multi-task learning. Advances in Neural Information Processing Systems, pp. 4424 4434, 2017.
Livio Baldini Soares, Nicholas FitzGerald, Jeffrey Ling, and Tom Kwiatkowski. Matching the blanks: Distributional similarity for relation learning. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 2895-2905, 2019.
Mihai Surdeanu, Julie Tibshirani, Ramesh Nallapati, and Christopher D. Manning. Multi-instance multi-label learning for relation extraction. In Proceedings of the 2012 Joint Conference Em- pirical Methods in Natural Language Processing and Computational Natural Language Learn- ing, 2012.
Tiffany Tuor, Shiqiang Wang, Bong Jun Ko, Changchang Liu, and Kin K Leung. Data selection for federated learning with relevant and irrelevant data at clients. arXiv preprint arXiv:2001.08300 2020.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, 2017.

---

Under review as a conference paper at ICLR 2021
Hongyi Wang, Mikhail Yurochkin, Yuekai Sun, Dimitris Papailiopoulos, and Yasaman Khazaeni. Federated learning with matched averaging. arXiv preprint arXiv:2002.064400 2020.
Linlin Wang, Zhu Cao, Gerard De Melo, and Zhiyuan Liu. Relation classification via multi-level attention enns. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, 2016.
Wei Xu, Raphael Hoffmann, Le Zhao, and Ralph Grishman. Filling knowledge base gaps for distant supervision of relation extraction. In Proceedings of the 51st Annual Meeting of the Association for Compulational Linguistics, 2013.
Qiang Yang, Yang Liu, Tianjian Chen, and Yongxin Tong. Federated machine learning: Concept and applications. ACM Transactions on Intelligent Systems and Technology (TIST), 2019.
Qinyuan Ye. Liyuan Liu, Maosen Zhang, and Xiang Ren. Looking beyond label noise: Shifted label distribution matters in distantly supervised relation extraction. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2019..
Zhi-Xiu Ye and Zuen-Hua Ling. Distant supervision relation extraction with intra-bag and inter- bag attentions. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019.
Felix X Yu, Ankit Singh Rawat, Aditya Krishna Menon, and Sanjiv Kumar. Federated learning with only positive labels. arXiv preprint arXiv:2004. 10342, 2020.
Yujin Yuan, Liyuan Liu, Siliang Tang, Zhongfei Zhang, Yueting Zhuang, Shiliang Pu, Fei Wu, and relation extraction. In
Xiang Ren. Cross-relation cross-bag attention for Proceedings of the AAAI Conference on Artificial Intelligence, 2019.
Daojian Zeng, Kang Liu, Siwei Lai, Guangyou Zhou, and Jun Zhao. Relation classification via convolutional deep neural network. In Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, 2014.
Daojian Zeng, Kang Liu, Yubo Chen, and Jun Zhao. Distant supervision for relation extraction via piecewise convolutional neural networks. In Proceedings of the 2015 conference on empirical methods in natural language processing, 2015.
Xiangrong Zeng, Shizhu He, Kang Liu, and Jun Zhao. Large scaled relation extraction with rein- forcement learning. In Thirty-Second AAAI Conference on Artificial Intelligence, 2018.
Fadila Zerka, Samir Barakat, Sean Walsh, Marta Bogowicz, Ralph TH Leijenaar, Arthur Jochems, Benjamin Miraglio, David Townend, and Philippe Lambin. Systematic review of privacy- preserving distributed machine learning from federated databases in health care. JCO Clinical Cancer Informatics, 2020.
Dongxu Zhang and Dong Wang. Relation classification via recurrent neural network. arXiv preprint arXiv:1508.0/006 2015.
Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, and Vikas Chandra. Federated learning with non-iid data. arXiv preprint arXiv:1806. 00582, 2018.
Peng Zhou, Wei Shi, Jun Tian, Zhenyu Qi, Bingchen Li, Hongwei Hao, and Bo Xu. iitention-based bidirectional long short-term memory networks for relation classification. In Proceedings of the 54th annual meeting of the association for computational linguistics, 2016.
12

---

Under review as a conference paper at ICLR 2021
A
CASE STUDIES
Platform
Sentence
Type
NONE
ONE
ATT
ATT RA
This Work
Most Muslims in Montenegro, mindful of the Serbs' killings of Muslims in Bosnia, are expected to vote to end ties with Serbia, but villagers in Podgorica are worried about how their Serb neighbors would react to separation.
Fasle Positive
10
/
x
They have passed through Zagreh; Novi Sad: Belgrade; Pristina, in Kosovo; Skopje; Tirana, Albamia; and Podgorica, Montenegro, on their way to Sarajevo.
False Positive
7
x
This a great day for the citizens Montenegro to regain independence after 88 years, "said Ljubomir Djurkovic, a theater director from Centinje, a picturesque, pro-independence town to the west of Podgorica.
False Positive
56
x
The time has come, Montenegro's prime minister, Milo Djukanovic, said Thursday a jubilant final rally in Podgorica, the capital.
True Positive
26
Table 5: A case to illustrate the effectiveness of the proposed model. A fact in KB is (Podgor- ica, /locatioo/couuttt//cpaaaa Montenegro). Only the sentence in Platform 26 expresses the "/loca- tion/country/capital" relation, while the other sentences are all false positive.
Table 5 shows how different denoising methods select reliable instances in the training phase. In this case, a KB fact is (Podgorica, /lccation/couttty/capttllll Montenegro). Aligning this KB fact with decentralized raw text generates four training instances, which distributed in four different platforms. Only the sentence in Platform 26 correctly represents the "/occation/couuttty/aaaaall relation. The other sentences distributed in the other platforms are all false positive instances, which do not express the "/loaation/couttyy/aapaa""" relation.
From this applied to the automatically labeled data, it would face noisy environment where most sentences false positive. (2) Previous denoising methods, such ONE (Zeng et al., 2015), ATT (Lin al., 2016) and ATT-RA (Ye & Ling, 2019), all fail to filter out false positive instances. In the worst cases, these methods will lose their denoising function. (3) Our proposed method can remove all false positive instances and only keep the true positive instance to train local models.
find that: (1) If FedAvg algorithm (McMahan et al., 2016) was directly
13

---

Under review as a conference paper at ICLR 2021
B ADDITIONAL EXPERIMENTS
B.I CAN A STRONG EXTRACTOR MITIGATE THE LABEL NOISE IN FEDERATED SETTINGS?
In this section, we investigate the impact of involving a stronger extractor. More concretely, we replace the PCNN-based extractor with BERT-based extractor (Devlin et al., 2018). the BERT- based extractor, use the architecture of entity mention pooling (Soares et al., 2019) to represent relations with the Transformer model (Vaswani et al., 2017), which shown in Figure Given first segment the given sentence into tokens by
sentence and two entities within this sentence, the byte pair encoding (Sennrich et al., 2016) and feed these tokens into the BERT encoder. The output of the BERT encoder is the context-aware embeddings of tokens. After that, pooling on the context-aware embeddings that correspond to the word pieces in each entity mention, to get two vectors he1 and ha2 representing the two entity mentions. Finally, concatenate these two vectors to get the representation of relation.
Max sodling
Max pooling
Deep Transformer
[CLS] [F1]
Entty
Enily
(12)
(SEr)
Figure 5: The main architecture for BERT-based extractor.
For fair comparison, we fix C 0.1, = 32, K 100 and E For the BERT-based extractor, we set the Ir, Ir decay and weight decay to le-5, 0.01 and le-5, and we use the pretrained BioBERT (Lee et al., 2019) and cased base version BERT as the initialization parameters MIRGENE and NYT 10 dataset, respectively. The AUC values of PCCN--base based extractor and BERT- based extractor on NYT 10 dataset and MIRGENE dataset are shown in Table 5, From the result, we find that: (1) Involving stronger encoder improve the performance for all denoising methods. (2) Whether leveraging PCNN or BERT as the encoder, our method significantly outperforms all baselines.
Dataset
Extractor
NONE
ONE
ATT
AVE
ATT RR
Ours
BERT-based Extractor
0.1744
0.2217
0.2156
0.2120
0.2086
0.2526
NYT 10
PCNN-based Extractor
0.1287
0.1719
0.1638
0.1521
0. 0616
0.2189
BERT-based Extractor
0.7510
0.7773
0.7798
0.7650
0.7768
0.8103
MIRGENE
PCNN-hased Extractor
0.7316
0.7665
0.75335
0.7499
0.7514
0.7846
Table 6: The AUC values of PCNN-based extractor and BERT-basee extractor NYT 10 dataset and MIRGENE dataset when is set to 100 and C is set to 0.1.
B.2 ABLATION STUDY OF PARALLEL COMPUTATION
In this section, we conduct an ablation study of parallel computation in federated settings. Conven- tional federated learning approach (McMahan al., 2016) is based on master-slave topology In this topology, each platform (slave) trains a local model based on its own local data and a master server (master) coordinates massive platforms to collaboratively train global model by aggregating these local model updates.
'https://gitoe http://gtthhb.ccoooogoggogg htt.h//..cooooooeeeeeeeeeeeeeeee reweeerreeeeeeeeeeeeeeeeeeeeeeee
hhttps://en httpa/:://e..wiiiieeee www.widiaiiai..o......... http:////wwww//t////t/aaaaa/ /ttasttt/sasaaeeeeeeeeeee (technology)
14

---

Under review as a conference paper at ICLR 2021
Aclivated Platform
Synchroinze Parameters
Activated
Model Inttialization
Getting Parameters
Model Initialization
Getting Parameters
Figure 6: The main architecture for chain training.
To ablate parallel computation from the conventional federated learning topology, propose simple way, called chain learning, to handle decentralized data. In chain learning, we only train one local platform at time, and then synchronize model parameters the next platform for further training. We show the architecture of chain learning in Figure 6. Compared with federated learning, chain learning does not require master server and parallel training.
ONE
ATT
AVE
ATT.RA
Ours
Centralized Learning
0.2323
0.2345
0,2147
0.2365
Federated Learning
0.1719
0.1638
0.1521
0.1664
0.2189
Chain Learning
0.1574
0.1538
0.1466
0.1590
Table AUC values of different learning paradigms on NYT 10 dataset. In federated learning and chain learning, we fix C = 0.1, K =100 and E =3.
Table B.2 shows the AUC values of centralized learning, federated learning and chain learning NYT 10 dataset. For fair comparison, we set batch size to 32, set learning rate to 0.1, and set weight decay le-5 for all learning paradigms. From the result, find that: (1) Centralized learn- ing achieves the best result compared to federated learning and chain learning; (2) Compared to fed- erated learning, chain learning cannot achieve satisfactory results. We conjecture that catastrophic forgetting (McCloskey & Cohen, 1989; Goodfellow et al., 2013) may be the cause. In federated learning, averaging step for model integration is carried out in each epoch, which may ensures the generalization of the model. However, in chain learning, the model may forget the previous local data and overfit the latest local data, which hampers the performance. (3) Our approach narrows the gap between federated training and centralized training in terms of performance. Therefore, we can conclude that not chain learning but cross- ploss-olatform collaboration is the key to mitigate label noise in federated settings.
15

---

Under review as a conference paper at ICLR 2021
C
BASELINES
Algorithm Federated Denoising Bascline
1: Hyperparameters: K is the total number of platforms: is the fraction of platforms; B is the
local minibatch size; E is the number local epochs; 17 is the learning rate.
2: Master server executes:
3: Initialize Oo
4: for communication round 9 0,I... do
5:
max(C' K, 1) 6: Ag + (random set of m platforms)
Select activated platforms
7: for each platform i Ag parallel do
8:
O -raiin.raininggi. θ2)
9:
O+t EiEr, Σje,, D,
D.
θ 9+1
Defined in Equation
10:
11: Function Local. Training(i, θ):
Run on platform
12:
B 4 (split D; into batches of size B) for each local epoch e from to E do 14: for batch b E B do
> A batch is set of bag
13:
15:
Conduct the denoising method
> In NONE, we do not carry out this step
16:
Update θ based on the gradients of the loss function return é to the master server
17:
In Algorithm 3, we present the federated framework of denoising baseline. Compared with FedAvg algorithm (McMahan et al., 2016), only add one step in local training to denoise. Compared with the proposed federated denoising framework, local platforms in the baseline framework handle label noise only based its own local data.
16

---

Under review as a conference paper at ICLR 2021
D
SOME TABLES AND FIGURES MENTIONED IN THE MAIN TEXT
NYT
MIRGENE
NONE
ONE
ATT
AVE
ATT_RA
Ours
NONE
ONE
ATT
AVE
ATT_RA
Ours
0,1658
0.7696
0.7597
0.7893
C=0.2
0.1293
0.1747
0.1642
0.1531
0.1666
0.2315
0,7432
0.7649
0.7528
0.7491
0,7448
0.7923
C=0.5
0.1305
0.1725
0.1657
0.1527
0.1647
0.2448
0.7365
0.7626
0.7516
0.7431
0.7484
0.7946
C=1.0
0.1283
0.1715
0.1631
0.1503
0.1637
0.2465
0.7358
0.7570
0.7483
0.7432
0.7493
0.7966
Table 8: AUC values on NYT 10 dataset and MIRGENE dataset.
NYT
MIRGENE
P@N(%)
NONE
ONE
ATT
AVE
ATT.RA
Ours
NONE
ONE
ATT
ATT.RA
Ours
p 100
57.0
63.0
60.0
57.0
62.0
69.0
83.0
87.0
89.0
87.0
86.0
89.0
P@ 200
49.0
60.0
57.0
55.0
55.5
67.0
75.0
79.5
77.5
78.0
77.0
80.0
C=0.1
P0:300
44.7
54.7
52.7
53.0
63.0
69.0
71.3
69.3
70.7
71.3
70.7
Mean
50.2
59.2
56.6
56.9
75.7
79.3
78.6
78.1
79.9
p
56.0
66.0
59.0
59.0
85.0
87.0
20-
46.5
58.5
57.0
$1.5
54.0
70.5
78.0
79.5
78.0
76.0
76.5
80.5
C=0.2
P@ 300
42.3
55.0
52.7
50.7
$1.0
68.7
69.7
70.7
70.7
70.3
69.3
73.0
Mean
48.3
59.8
56.2
53.7
55.3
71.1
75.9
78.4"
78.6
77.1
75.3
81.5
p@ 100
47
65.0
63.0
58.0
60.0
77.0
79.0
87.0
87.0
84.0
83.0
92.0
C=0.5
200
47
59.0
57.5
80.0
75.0
77.0
P@000
44.3
55.0
53.3
52.7
50.3
71.7
70.3
70.7
70.0
70.3
74.0
Mean
46.1
59.7
57.9
$4.7
54.9
74.4
74.9
79.2
77.3
76.4
77.0
82.8
p@ 100
48.0
62.0
65.0
60.0
60.0
80.0
78.0
82.0
82.0
82.0
83.0
95.0
C=1.0
P@ 200
47.5
60.0
56.5
$4.0
54.5
75.5
75.0
78.5
76.0
77.0
76.0
82.0
P6:300
43.3
56.0
52.3
49.7
49.0
71.3
68.7
71.3
70.0
70.0
70.0
73.0
Mean
46.3
59.3
57.9
54.6
54.5
75.6
73.9
76.0
76.3
83.3
300 and the mean them for each model in held-out evaluation on NYT 10 dataset and MIRGENE dataset.
Table 9: P@ 100, P@-000, P@ 300
NYT
MIRGENE
P@N(%)
NONE
ONE
ATT
AVE
ATT_RA
Ours
NONE
ONE
ATT
AVE
ATT_RA
Ours
100
53.0
63.0
65.0
63.0
69.0
73.0
82.0
90.0
88.0
84.0
85.0
94.0
200
46.0
62.0
58.0
59.5
61.0
69.5
74.0
80.5
78.0
77.5
80.5
83.0
300
45.0
59.3
54.7
56.7
59.0
68.7
69.7
71.7
70.7
70.7
71.7
71.0
Mean
48.0
61.4
59.2
59.7
63.0
70.4
75.2
80.7
78.9
77.4
78.6
82.7
P0300 and the of them for each model in held-out evaluation NYT 10 dataset and MIRGENE dataset when is set to 50 and C' is set to 0.1.
Table 10: P@ 100, P@200,
17

---

Under review as a conference paper at ICLR 2021
NYT
MIRGENE
1.0
This work
1.00
This work
NONE
NONE
0.9
ONE
0.95
ONE
ATT
ATT
0.8
AVE
AVE
ATT_N RRA
0.90
ATT_RA
Precision
0.7
Precision
0.85
0.6
0.80
0.5
0.75
0.4
0.3
0.0
0.1
0.2
0.3
0.4
0.0
0.1
0.2
0.3
0.4
0.5
0.6
Recall
Recall
Figure 7: Aggregate precision-recall curves on NYT 10 dataset and MIRGENE dataset when K set to 50 and C is set 0.1.
18


Reviews:
[{"review_id": "mrStXnZp5u-", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\n\nThis work focuses on investigating distant supervision for relation extraction task within federated learning paradigm. The proposed Lazy Multi-instance Learning approach identifies reliable sentences for the same entity pairs across platforms to denoise distantly supervised data and perform relation extraction.  \n\nThe proposed approach has been applied on two relation extraction datasets and has reported gains. \n\n\nStrengths:\n- The core of distant supervision for relation extraction is to denoise sentences for building a supervised classifier. The problem is well investigated in NLP community. However, this work focuses on modeling the distant supervision problem in real-world environment addressing data decentralization, ownership and privacy. I buy the idea of applying the existing methods or extend the NLP applications in federated setup. \n- the paper is well written, clearly motivation and addressed real-world problem \n- interesting work: blend of distant supervision and federated learning for NLP task\n\nWeaknesses:\n- the paper lacks novelty in terms of methodology \n- Distant supervision for relation extraction (PCNN and MIL) and the federated learning methodologies used in this work have been inspired from existing works. This works combines the two.\n- the experimental setup needs more clarity \n\n\nQuestions:\n1. Assumption made: Sentences with same entity pairs must scatter across platforms. How would the Lazy MIL system bootstrap without such an assumption (no same entity pairs)?\n2. Assumption made: The number of output classes are same therefore the \\Theta parameter (including output-layer weights connecting softmax) is used across platforms. In real world, it is not the case. Each platform (or customer) may have different relation types. How would you aggregate/distribute the output layer parameters?\n3. How to obtain a minimum viable global \\Theta? How to init this without having the actual training data in Master server?  \n4. In equation 2, the v^i is not controlled by any threshold. Will it denoise if its value is too low (say <0.50) and it is still the highest among all the platforms? \n5. In section 4.1 in data partitioning, how do you ensure the number of relation types remains same across all platforms?\n\nAdditional comments: \n\n- Include an ablation study analyzing scores due to different values of K.\n\nExperimental setup  unclear:\n- how do to init \\Theta in global model?\n- What is the held-out data set for local and global model training?\n\nResults:\n- the experimental results show noticeable gains. It is surprising as the NYT dataset is well investigated in distant supervision settings. \n- As the training data is split across platforms, the overall system is decoupled into several local models. In essence, the overall performance in federated settings should deteriorate or remain competitive to the baseline models due to 'no joint' training on the overall corpora as well as due to (somewhat) lossy aggregation. Please provide a detailed reasoning about the noticeable gains achieved.   \n\nReproducibility\n- no code available", "extra_scores": {}, "extra_text": {}}, {"review_id": "j7K7HVGrnp", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper addresses relation extraction problem for distributed platforms for privacy concerns. The authors propose to leverage federated learning with denoising techinques for better performance. The relation extractor is set to the conventional piece-wise CNN and the main contributions focus on dealing noises in the distributed settings.\n\nPros:\n* A new method for relation extraction in the federated learning to address privacy concerns.\n* Best performance compared to baseline models.\n\nCons:\n* Lack of novelty. This method follows the federated learning paradigm with few improvements. The main contribution is limited to the proposed lazy multiple instance learning, which is not specific to relation extraction problem. If this is not specified for relation extraction, more tasks are expected to demonstrate it is a general algorithm.\n* Experiments are insufficient. First, it is doubtful whether the i.i.d. setting is a practical assumption for relation extraction, because in the real world the quality of corpus in different platforms may vary drastically. Second, given the i.i.d. setting, the authors can repeat their experiments to reduce randomness, because the proposed denoising strategy may only perform well in situations that useful sentences are allocated to only few platforms. The authors may provide more analysis on the distribution of contributions from platforms.\n* Writing could be improved. In section 3.3, what does the ``value v^i'' refers to? Also, if v^i > v^j, then the id^i-th sentence in the platform i is selected...''. Does it mean that at every round only one sentence from a platform is selected? What if for a certain relation, there are many promising candidate sentences from one platform?", "extra_scores": {}, "extra_text": {}}, {"review_id": "fEzvLpRrpZ", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper explores relation extraction with distant supervision in the federated setting and focuses on handling the label noise problem from automatic distant supervision by Multiple Instance Learning-based methods and proposes Lazy MIL. \n\nFor a specific triple (h, r, t) in KB, values (probability with relation r) for sentences containing h and t will be calculated locally in each platform, and such best value and index in each platform will be uploaded to the master server. A master server decides the most reliable sentence and broadcast such information to all activated platforms in this round.\n\nStrengths: \n\n+ This paper considers relation extraction in the federated setting, which is a new direction in this area;\n+ The mentioned label noise problem is a real problem in distant supervision, and it’s intuitive that this problem exists in the federated setting;\n+ Experiments show the effectiveness of the proposed method;\n\nWeaknesses:\n\n- The contribution of the paper is weak (in the context of the expectations of ICLR) \n- For baselines in experiments, it’s not clear how to use baselines in the federated setting. For example, for “keep the other modules unchanged and only replace the denoising module” in the last sentence of Sec. 4.3, it’s confusing for me that how to replace this part. Is the part for broadcasting denoising information? \n- Except for the experiment results, it’s not clear that if the proposed method is specific to the federated setting, and what are the federated-setting-specific designed compared with other baselines?\n\nQuestions:\n\nIn Sec. 4.4, “We believe the reason is that our denoising method can hinder false-positive instances from poisoning local models.” Does this mean that other baselines can’t hinder false-positive instances?", "extra_scores": {}, "extra_text": {}}, {"review_id": "eEEHZDHju4t", "reviewer": "AnonReviewer5", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary\n\nThe paper investigates intersection of federated learning and distant supervision of knowledge graphs from texts. The main innovation is a simple yet empirically effective denoising rule that selects only the sentences deemed to be the most reliable for learning in each round of training.\n\n\nStrong points\n* The method is very simple to implement therefore it might be widely adopted as a baseline.\n* The denoising step is specific only for federated learning and distant supervision. It can be applied to many other domains as well (and not just for relation extraction).\n\nWeak points\n* The paper would benefit from further proofreading. \n* Some uncertainty about experimental results, namely repeatability of the runs and methodology of hyperparam selection (more in questions section).\n\n\nRecommendation\n\n* Pre rebuttal: I like the simplistic approach however in current form I am leaning towards rejecting the paper due to my uncertainty in the empirical evaluation. However I am willing to change my evaluation if these questions are resolved.\n\n* Post rebuttal update:  Several of my concerns about clarity of the experimental section were addressed. Therefore I increased my score and now I am inclining towards accepting the paper.\n\nQuestions\n\n* Methodology of obtaining the empirical results isn't clear from the text. Section 4.1 mentions a single held out test set. However then it isn't clear how hyperparam sweep discussed in Sec 4.2 was done. If there is only a single test set were the hyperparams selected on the same test set that is later used to report results in Fig 2 and 3? I would consider that a flaw. Or is there a second validation set (that isn't mentioned in the text explicitly) used to tune hyperparams?\n\n* Curves in Fig 2 and 3 seem to be from just one run for each setting. Doing more runs with different random initializations and reporting mean (or top k results) + confidence intervals would increase confidence in these results.\n\n\nPossible improvements\n\n* \"Lazy MIL almost does not leak the corpus information in each platform\" --- can you make this statement more precise? \n\n* Baseline attention models are always forced to spread their attention among sentences in the local bag even if they are all irrelevant. What if the attention module is allowed to output \"all sentences are irrelevant\" option (this is sometimes called 'sentinel', https://arxiv.org/abs/1612.01887). This might work as learnable denoising that deals with the issue that some local bags don't include any relevant examples. \n\n* Add references for all datasets in section 4.1.\n* Alg 1: are lines 6 and 7 needed? they seem to be contained by line 8.\n* Alg 1, line 11: this piece of Python in pseudocode might not be readable by everyone, I would explain it in plain words.\n* Alg 2: Hyperparams --- \"E is the number\" (what number?)\n\nSome language issues:\n\n* leads to catastrophic repro- ducibility -> is not reproducible\n* is a large biomedical with -> biomedical dataset with?\n* KB is public available -> publicly available\n* the token representation is represented as -> the token is represented as?\n* The overall of Lazy MIL is illustrated in Algorithm 1. --- overall design of? overview of?\n* \"to extract the high-level sentence representation from three segments of CNN outputs, and the boundaries of segments are determined by the positions of the two entities.\" -> I would first define what segments are and only after that how they are used in NN, here the order is reversed and I find it more difficult to follow.\n* Konečný (different accentation in the last character)", "extra_scores": {}, "extra_text": {}}, {"review_id": "UFTUBSkvxFd", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\nThis paper introduced a new federated scenario for distantly supervised relation extraction where extractions come from multiple private resources. And therefore a global model cannot directly access all the data simultaneously. \n\nTo protect data privacy, this paper assigned each resource a local model for separate training. In order to apply the at-least-one sentence bag denoising technique, at the beginning of each round, local models will calculate scores of sentences in the local bag, compare scores with other resources and then generate the training data. After each round of parallel training, local models will be synchronized with the global model via a weighted average algorithm. By only sharing scores rather than the full text, data privacy is protected. Experiments over two datasets show promising results.\n\n####################\n\nReasons for score:\n\nI vote for marginally negative. Overall, this paper brought up an interesting challenge for this NLP sub-task. However, it is not clear to me the necessity of compromising accuracy while doing parallel computing, and model innovation is slightly weak. \n\n####################\n\nThere are two comments from a performance-driven perspective:\n\n1. Is there a specific reason for this task to do parallel computing? From my understanding (if correct), if parallel training is not required, we can avoid major information loss from delayed denoising and averaging step for model integration. In another word, if we only train one local platform at a time (and denoising frequently), and then synchronize model parameters to the next platform for further training, the performance should be close (or equal if using ONE model) to the performance upper bound when K=1.  If parallel computing is the key, then it is necessary to analyze the speed and performance trade-off.\n\n2. It is necessary to use pre-trained language models like MTB (matching the blank) paper as the encoder because improvement over the current baseline could possibly be mitigated by a stronger encoder.\n\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper studies Distantly Supervised Relation Extraction(DSRE) in distributed settings. Though DSRE has been studied in Centralized setting it has not been studied in distributed platform. This  paper leverages the federated learning setup for this problem and proposes to use Lazy MIL for this purpose.  The paper  identifies the main challenge as label noise but does not attempt to characterise the severity of the problem vis-a-vis the centralised setup.  Though intuitive but a formal approach would have helped in understanding the importance of the derived results better. 

Decision: Reject