CONFEDERATED MACHINE LEARNING ON HORIZONTALLY AND VERTICALLY SEPARATED MEDICAL DATA FOR LARGE-SCALE HEALTH SYSTEM INTELLIGENCE
Authors: Dianbo Liu
Tim Miller
Kenneth Mandl
Venue:  (2020)
Keywords: Confederated learning
siloed medical data
representation joining

Abstract: A patient’s health information is generally fragmented across silos. Though it is technically feasible to unite data for analysis in a manner that underpins a rapid learning healthcare system, privacy concerns and regulatory barriers limit data centralization. Machine learning can be conducted in a federated manner on patient datasets with the same set of variables, but separated across sites of care. But federated learning cannot handle the situation where different data types for a given
patient are separated vertically across different organizations. We call methods that enable machine learning model training on data separated by two or more degrees “confederated machine learning.” We built and evaluated a confederated machine
learning model to stratify the risk of accidental falls among the elderly.

Under review as a conference paper at ICLR 2020
CONFEDERATED MACHINE LEARNING ON HORIZON- TALLY AND VERTICALLY SEPARATED MEDICAL DATA FOR LARGE-SCALE HEALTH SYSTEM INTELLIGENCE
Anonymous authors Paper under double-bling review
ABSTRACT
A patient's health information is generally fragmented across silos. Though it is technically feasible to unite data for analysis in a manner that underpins rapid learning healtheare system, privacy concerns and regulatory barriers limit data centralization. Machine learning can be conducted in a federated manner on patient datasets with the same set of variables, but separated across sites of care. But federated learning cannot handle the situation where different data types for given patient are separated vertically across different organizations. enable machine learning model training data separated by two more degrees "confedcrated machine learning." We built and evaluated a confederated machine learning model stratify the risk of accidental falls among the elderly.
call methods that
INTRODUCTION
Significance. Access large amount of high quality data is possibly the most important factor for success in advancing medicine with machine learning and data science. However, valuable healthcare data are usually distributed across isolated silos, and there are complex operational and regulatory concerns. Data on patient populations are often horizontally separated,each other across different practices and health systems. In addition, individual patient data are often vertically separated, by data type, across her sites of care, service, and testing. We train confederated learning model in manner to stratify elderly patients by their risk of fall in the next two years, using diagnoses, medication claims data and clinical lab test records of patients. Traditionally, federated machine learning refers to distributed learning on horizontally separated data (Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, Vikas Chandra 2018; Cano, Ignacio, Markus Weimer, Dhruv Mahajan, Carlo Curino, and Giovanni Matteo Fumarola 2016; H. Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, Blaise Agüera y Arcas 2016; Anon n.d.). Algorithms sent different data silos (sometimes called data nodes) for training. Models obtained are aggregated for inference. Federated learning can reduce data duplication and costs associated with data transfer, while increasing security and shoring up institutional autonomy. (Geyer, R. C., Klein, T., Nabi, M. 2017; al. 2016), Zhao, Meng Li, Liangzhen Lai. Naveen Suda, Damon Civin, Vikas Chandra 2018; al. 2015e, R. C.. Klein, T., Nabi, M. 2017: al. 2016).
Notably, patient's vertically separated data may span data types-for example, diagnostie, pharmacy, laboratory, and social services. Machine learning on vertically separated data has used a split neuron network (Praneeth al. 2018) and homomorphic encryption (Praneeth et al. 2018; Stephen al. 2017). However, these new methods require either information communication at each computational cycle or state-of-art computational resource organization, which are usually impractical in many healthcare systems where support for data analysis is not the first priority, high speed synchronized computation resources are often not available, and data availability is inconsistent.
To accelerate scalable and collaborative rapid learning health system (Friedman et al. 2010; Mandl et al. 2014), we propose a confederated machine learning method that trains machine learning models on data both horizontally and vertically separated by jointly learning high level representation from data distributed across silios(Qi et al. 2017: Zhang Xiao 2015: Zhai et al. 2014). This method does not require frequent information exchange each training epoch nor ofate-of-the-art

---

Under review as a conference paper at ICLR 2020
distributed computing infrastructures. As such, it should be readily implementable, using existing health information infrastructure.
We demonstrate this approach by developing model of accidental falls among people at least 65 years, a problem which causes approximately 50.7 deaths per 100,000 the US annually (Anon 2019). Women and men from 65 to 74 years old had 12-month fall incident rate of 42.6 and 41.3
per 100, respectively; once over 74 years old, the incident rate climbed to 50.6 and 62.0 per 100 respectively, according the 2008 National Health Interview Survey in 2008 (Verma al. 2016). Nationally, the direct medical costs attributable to falls is 34 billion dollars (Kramarow et al. 2015;
Verma al. 2016; Heinrich al. 2010; Haasum Johnell 2017; Yang et al. 2016; Dollard al. 2012; Overstall 1985; Lord et al. 2007).
There are highly effective approaches to mitigating the risk of falls that could be selectively applied to individuals identified as being at high risk. These include medication adjustments, exercises, and home interventions (McMurdo al. 2000; Kosse et al. 2013). Multifactorial clinical assessment and management can reduce falls by more than 20
We train confederated learning model to stratify elderly patients by their risk of fall in the next two years, using horizontally and vertically separated diagnosis data, medication claims data and clinical lab test records of patients. The goal to compare confederated learning with both centralized learning and traditional federated learning, and specifically test whether a confederated learning approach can simultancously address horizontal and vertical separation.
2 METHODS
2.1 DATA SOURCE AND COHORT
The study uses claims data from major U.S. health plan. Elements include the insurance plan type and coverage periods, age, sex, medications, and diagnoses associated with billed medical services, from July 1 2014 to June 31 2017. The dataset contains indicator for insurance coverage by month. Only beneficiaries over the age of 65 by the beginning of the study period, and having full medical and pharmacy insurance coverage during the 36-month period were included. The study period is divided into a 12-month observational period, a week gap period and a follow-up period of 23 months and 3 weeks. Individuals not enrolled in the Medicare Advantage program ensure completeness of the private and public insurance data. In addition, members with fall-related diagnoses within observational or gap were excluded. For each individual, the predictive model uses claims from a 12-month observation period from the start of the study period. The outcomes (claims indicative of falls) are measured during the follow period.. The study cohort comprises 119,335 beneficiaries, with 56.6% female.
excluded to
The input features to the confederated machine learning model include age, gender, diagnoses as ICD 9 or ICD 10 codes, medications represented as National Drug Codes (NDC) and lab tests (encoded as LOINC codes). Lab test results were not available for this study. On average, each individual has 13.6 diagnoses, 6.9 prescriptions, and 7.4 LOINC codes during the 12 month observational period. 10,584 (8.9 (8.9% ) of beneficiaries the cohort had at least one fall in the 21-month testing period. The number of people and falls from each state are summarized in Supplementary Table 1.
2.2 STUDY OUTCOME
An online International Classification of Diseases, Ninth and tenth Revision (ICD-9 and ICD-10) lookup resource provided by the Centers for Medicare and Medicaid was used to obtain all codes directly associated with an accidental fall, for example, fall from sidewalk curb (E880.1). 1).. For each member, we marked with a binary outcome variable (0 or 1) of whether a person had any fall-related claims during the follow-up up period. A total of 690,295,554 2,5499 medical claim diagnoses were coded in ICD-9 and 900,713,944 were ICD-10. A total of 84 ICD-9 (Homer et al. 2017) and 330 ICD-10 (Hu Baker 2012) codes were used to identify falls.
2

---

Under review as a conference paper at ICLR 2020
A
Ug
Laboratory
Anelyzer
Untrained algonithms
algorithms
Pharmacy
Disgnosis Input
Diagnosis representation
-typertension ungasuum
Input
representation
Joint representation
Asotaminophen
basylate Purosemite
teat Input
Alantno Amiiottan
addivilyhahune)
[Massivolume] reective protess
Lab test representation
Figure 1: Confederated machine learning trains model across multiple dimensions of data separation. (A) Confederated machine learning utilizes data from clinics, pharmacies and labs simultancously without moving the data. (B) Confederated representation joining learns joint representation from different data types in separated silos.
2.3 CONFEDERATED MACHINE LEARNING
For specific state s 1,2, with S 34in this study Each individual has a, diagnosis vector x-digg a medication claim vector X from pharmacy, lab test vector X1sh from clinical lab and a binary label Yst where in each state 1,2, ....,,,, n² with n² being the number of beneficiaries in state
At high level, confederated learning can be intuitively understood as follows: universal machine learning model that has both representation learning and classification components is designed for all data sites. The representation learning component takes as input variety of input features drawn from different types of data, which cannot be presumed to all be present at the same time (Figure and 2). We took the following steps:
Step 1.CClimms for diagnoses, medications and lab tests during the observation period are the input features. The output of the classifier a binary variable indicating whether the beneficiary had fall during the follow up period. We simulate horizontal and vertical separation by separating the data for beneficiaries by U.S. state of residence. We simulate vertical separation by assuming that beneficiaries' diagnoses are only available in simulated clinics, medication claims data are only kept simulated pharmacies and lab data only in simulated labs. Data are presumed not shared among different organizations nor across state lines. In total, we simulated data distributed across 102 distinct nodes including 34 clinic nodes,34 pharmacy nodes and 34 lab nodes.
Step 2. For the distributed model training, each site delivered array of binary target labels (fall or no fall during the year follow-up period), linkable to the individual beneficiary.
Step 3. A neural network model is designed. Two different branches of input neurons (layer Çi ) and their directly linked connection to the next fully connected hidden layer (layer &2 were used learning subcomponents for each input data type (Figure I and 2, and see next section for details):
medications or lab tests. After the two branches merges, the second hidden serve as a high level representation learning and joining layer (layer Ça integrate representation learned from
3

---

Under review as a conference paper at ICLR 2020
different data types. The third layers is classifier layer (layer Ça ). it worth pointing out that the representation and classifying power may be not completely separated by each layer in the neural network models. The parameter eof f is randomly initialized.
Step 4. The confederated learning model is trained as shown in Figure and 2. Models with identical structures and parameters sent to all the 102 nodes together with the binary label of falls in the follow-up period. When the machine learning model is trained on a specific data type, such as medication claims, only the representation learning subcomponents for that data type are active and subcomponents for other data types are frozen. This implemented by sending a data availability indicator to each data source to indicate whether certain data type is available at each site. For example , Pharmacy = [1, Oj can be used to indicate that medication data is available at pharmacy Phormacy
but lab data and diagnoses are not, where w med used imput into layer Ç3-
1, w diag
Pharmacy
0 and wla
Pharmecy 0.wis
Step 5. We aggregated trained parameters from all sites by averaging the parameters to produce the updated model. By doing this, the data type specific components(C1 and {2 joint representation component (C3 and classification components (C4 were learned simultaneously from different data types and individuals from all 102 sites. Information sharing among vertical separated data is achieved by joining the the representation at {3-
Step 6. After model aggregation, the updated model was sent out all 102 sites again repeat the global training cycle.
3 CONFEDERATED LEARNING METHOD DETAILS
The key idea behind our approach is to jointly train a representation using different data types from different sources distributed manner with our moving or aggregated the original data. In this section, we formally describe data organization, confederated training and inference.
The goal of confederated training of the classification model to minimize the binary cross entropy, metric for binary classification error, without moving any data out from from the beneficiaries' state of residency nor their data silos (pharmacies or labs). The objective function to minimize for classification model f(Xding, " Xmmed X +) is:
n
L(Xdiag X Xlab Q) - -Σ
-YYsilo([[(X diig, X X, θ))
+(1 - Yxi)aal - - [(xding xmed xlab 0)))
Where θ is the parameter of model f.
As data were not allowed to be moved out from their silos, it is not possible to train f by minimiz- ing L(Xdiag, ed, Xl,,, θ) in centralized manner. Therefore, we randomly initialized the parameter è as Ou and sent model ff and parameter O₂ to pharmacies or clinical labs in each state E S.
In the clinic or hospital system of state we set the value of the pharmacy inputs to grned ,a zero vector, and value of the diagnoses inputs Odiag
The loss function is then calculated as:
L L(Xdiag gmed glab ediag)
Σ 1=1
(Y=zlog(f(Xddie gnmdd glab, ediag))
+(1 Yo f f(Xdia, gmed glab (diag)))
Using stochastic gradient descent to minimize the loss. parameter ddagg obtained. e 1, 2stadss for number of global loops which will be explained in detail just below.
4

---

Under review as a conference paper at ICLR 2020
In the pharmacy system of state s, we set the value of the lab inputs to 0fah and value of the diagnoses inputs to Odheg The loss function is then calculated as:
the
L(Odiag X.med gleb, Omed)
E
(f Xmmeed Olab, Omcd))
+(1 (1-YY,)oop Nog f(Odiag, Xmmed glab, Omed)))
As above, Ost is obtained using SGD.
In the lab system of state s, we set the value of the pharmacy inputs to Omnd ,and value of the diagnoses inputs to (ding
The loss of function in the lab system is calculated as:
The
L(Odiag, gmed Xlab Olab)
Σ
(f omed XXub, elab))
+(1 Ya log f(Odiag gmed, f(Xlabb Olab))
As above, Ostlab obtained using SGD.
After 0 .0med and @gab were trained locally in each single state, they were sent back to the analyzer for aggregation by weighted averaging: O, 35 Σ n" N N is the total number of beneficiaries included in the study from all states. O is then sent back to clinics. pharmacies and labs in each state to repeat the whole global cycle to obtain 0141. It is worth pointing out that in this study zero vectors were used as placeholders for data types that were not available because the performance vector and an all ones vector. It is absolutely possible that more sophisticated placeholders, such as vector estimated from sample distribution, would lead to better performance. However, this is not the focus of exploration in this study.
(O + O Olah) where
best validation
when comparing with a random [0,1]
Artificial neural networks are used as the primary machine learning model for predicting fallst. The model is constructed in Keras 2.0 environment using tensorflow 1.7 as backend. Adam is used as the optimizer method with default setting (Diederík Kingma 2014). The model consists of three branches, one for each data type (Figure and 2). Each branch consists of an input and fully connected hidden layer with 256 neurons. The two branches merged after the hidden layer and are connected to another fully connect hidden layer with 128 neurons before the output layer with 1 neuron. The activation function used for hidden layers is "ReLu" and for output layer is "Sigmoid". In the confederated learning, 10 epochs of training are conducted during each local training cycle and global cycles are stopped when performance did not improve for 3 cycles. All the model architecture and hyperparameters were determined by grid searching based on performance When training on clinical lab data, the model parameters corresponding to branch of medications was frozen and vice versa.
validation set.
20% of randomly chosen beneficiaries were reserved as test data, and not included in the training set, 20% were chosen as validation set to adjust hyperaarameters and 60% were used training set. When conducting federated or confederated learning, data of 20% of beneficiaries from each node were used as validation set. After hyperparameter tuning, both training set and validation used to train the model to test performance. When testing performance of each model, the test set has centralized data with both medication claims and lab tests. Outputs from ensemble learning models were averaged to give the combined prediction.
were
Performance evaluation included area under the receiver operating characteristic curve (AUCROC) and area under the precision recall curve (AUCPR), AUCPR was used because the data are imbalanced- there are many more people without falls than those with falls. Instead of following the common practice of choosing a threshold that sets the false positive rates to be equal to the false negative rate (equal error rate), we chose the threshold which is 5% quantile of the predicted score true fall, We sought to favor screening strategy and are willing to tolerate some false positives. Using this threshold, the positive predictive value (PPV) and negative predictive value (NPV) were calculated (Table 1) and used performance metrics in addition to AUCROC and AUCPR. Interpretation of
5

---

Under review as a conference paper at ICLR 2020
Model parameter eggregation at analyzer site
State
Slate
State 34
Data type 1 input
m
Data type input
Data type input
Availability indicator
medication
lab
Diagnosis
Figure 2: The confederated machine learning train model where an individual's data are vertically separated nodes.
the machine learning was conducted using DeepExplain package in Python2.7 using gradient input method.
EXPERIMENTAL RESULTS
We conducted experiments to compare the performance of predictive models trained the following settings (table 1): (I) centralized learning, where data were not separated at all; (2) federated learning, where data were only horizontally separated (3) model trained on each single data type, where data were only vertically separated; (4) confederated learning, where data were both horizontally and vertically separated.
The performance of all models in this study defined how accurately the model can predict the fall testing set of randomly selected individuals from the whole study cohort that were not included any model training, using features corresponding to the model inputs.
4.1 CENTRALIZED LEARNING
When we conducted the model training on aggregated data, where all types of data were centralized, the model achieved AUCROC of 0.70, AUCPR 0.21, PPV of 0.29 and NPV of 0.90. The centralized learning performance using each single data type was calculated (Table 1).
4.2 FEDERATED LEARNING
We note that when training data that is horizontally but not vertically separated, confederated learning is mathematically identical to traditional federated learning. When we conducted federated learning horizontally separated data, in which data were distributed in 34 states of residency but

---

Under review as a conference paper at ICLR 2020
Table I: Experimental results machine learning models for fall prediction under "separation of data types" and "separation of individuals"
AUCROC
AUCPR
PPV
NPV
Data with separation (centralized)
Learning on aggregated data
0.7
0.21
0.29
0,9
Data horizontally separated
Federated learning
0.68
0.2
0.31
0.9
Data vertically separated
Diagnosis data only
0.67
0.19
0.28
0.9
Medication claim data only
0.64
0.17
0.22
0.9
Lab test record only
0.59
0.14
0.15
0.9
Ensemble learning
0.64
0.17
0.22
0.9
Confederated Learning
0.68
0.2
0.29
0.9
Data horizontally and vertically separated
Diagnosis data only
0.67
0.19
0.28
0.9
Medication claim data only
0.63
0.17
0.24
0.9
Lab test record only
0.6
0.15
0.18
0.9
Ensemble learning
0.63
0.17
0.22
0.9
Confederated Learming
0.68
0.21
0.31
0.9
not vertically separated, the model achieved AUCROC of 0.68, AUCPR 0.20, PPV of 0.31 and NPV of 0.90.
4.3 AODD LEARNING
Using Confederated Representation Joining data that is vertically but not horizontally separated (single degree separation), the algorithm achieved an AUCROC of 0.68, AUCPR of 0.20, PPV of 0.29 and NPV of 0.90 on predicting fall in follow-up period.
Next, we conducted experiments to show that confederated learning is able to train a distributed model using distributed data with two degrees of separation, both horizontally and vertically. The confederated learning algorithm achieved an AUCROC of 0.68, AUCPR of 0.21, PPV of 0.31 and NPV of 0.90 predicting fall, which is comparable to centralized Iearning with all data aggregated and to federated learning where data were only horizontally separated. Performances of confederated representation joining in both vertical separation and vertical plus horizontal separation were better with ensemble learning where output from model trained on different data types were averaged.
In order understand behaviours of trained predictive models, the importance of each feature in the predictive models was calculated. The ten most important variables for the machine learning model trained centralized learning are shown in supplementary table The ten most important variables for the machine learning model trained in confederated learning are shown in supplementary table Hypertension, edema, and movement related medical conditions are found in both lists, which suggests the two models work in similar manners.
CONCLUSION AND DISCUSSION
Currently, the clinical screening process generally involves asking patients 65 years old and above questions about their previous falls and walking balance. People who give positive answers to the question can further assessed for their balance and gait (Panel on Prevention of Falls in Older Persons, American Geriatries Society and British Geriatrics Society 2011). Though the guidelines- based clinical assessment reduces falls, is costly and time consuming to conduct large seale screening. machine learning approach can stratify individuals by risk of fall, using their electronic health record data.
We demonstrate that health data distributed across silos can be used to train machine learning models without moving or aggregating data, even when data types vary across more than one degree of separation. Compared with other methods for model training horizontally and vertically separate
7

---

Under review as a conference paper at ICLR 2020
data, this confederated learning algorithm does not require sophisticated computational infrastructure such homomorphic encryption, nor frequent gradient exchange.
We anticipate that this confederated approach can be extended more degrees of separation. Other type of separation, such as separation by temporality separation by insurance plan, separation by healthcare provider can all be potentially be explored using confederated learning strategy. One such example of additional degree separation a patient's diagnosis might distributed with different healthcare providers his/her medication information with more than one pharmacy
Algorithm 1 Confederated Representation joining
Input:
Medications claims data (XXmed), lab tests records (X!ab) distributed S states and binary labels of fall (Y)
Parameter and hyperparameters of the neural networks θ Output: Whether an elderly will fall in follow-up period
Initialize neural network model with parameter On
for / E 1 to T in parallel do
// T is total number of global cycles
for State 8 E to S in parallel do
// Conducted in parallel across all S 34 states
In clinic node:
Oiing 0 // identical model parameters sent to each site Lab test branch of layers (i and frozen fit f
Obtain parameters of Odig In pharmacy node:
Omed < 0,-1 // identical model parameters sent to each site Lab test branch of layers Çi and (2 are frozen fit f
Obtain parameters of mmed
In lab node:
@lab + 0 identical model parameters sent to each site Medication branch of layers and &2 are frozen
Fit f
Obtain parameters of as Olab
end
Update model parameter θ 1 Σ9 ny (Odiag A Olab)
// Update model parameters using we weighted average from model parameters of each site
Set parameters of O
n, is the number of patients at state and N is the total population size across all states end
REFERENCES
Anon, 2019, Fatal Injury Data
WISQARS
Injury Center - CDC. Available at: (ccessed March 27, 2019].
8

---

Under review as a conference paper at ICLR 2020
Anon, Federated Learning: Collaborative Machine Learning without Centralized Training Data. Google AI Blog. Available at: http://hhogggoooogoooo/o///////e collaborativc.html [Accessed March 25, 2019].
Cano, Ignacio, Markus Weimer, Dhruv Mahajan, Carlo Curino, and Giovanni Matteo Fumarola, 2016. Towards geo-distributed machine learning. arXiv preprint. Diederik P. Kingma, J.B., 2014. Adam: A Method for Stochastic Optimization. arXiv.
Dollard, J. et al., 2012. Falls in old age: threat identity. Journal of Clinical Nursing, 21(17-18), pp.2617-2625. Available at: http://dx www.sooooooooog... ort.//r////// 10. 1/j. 1365-270..222111 1.03990.x. Friedman, C.P., Wong, A.K. Blumenthal, D., 2010. Achieving nationwide learning health system. Science translational medicine, 2(57), p.57cm29.
Geyer, R. C., Klein, T., Nabi, M., 2017. Differentially private federated learning: client level perspective. arXiv:
Haasum, Y. Johnell, K., 2017. Use antiepileptic drugs and risk of falls in old age: A systematic review. Epilepsy Research, 138, pp.98-104. 104. Available at: http://dx. 110/0.1116/eeeeepssyee
H. Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, Blaise Agüera y Arcas, 2016. Communication-EEfiicii
Learning of Deep Networks from Decentralized Data. arXiv. Heinrich, S. et al., 2010. Cost of falls in old age: a systematic review. Osteoporosis international: journal established as result of cooperation between the European Foundation for Osteoporosis and the National Osteoporosis Foundation of the USA. 21(6), pp.891-902.
Homer, M.L. et al., 2017. Predicting Falls in People Aged 65 Years and Older from Insurance Claims. The American journal of medicine, 130(6), pp.744.e17-744..22
Hopewell, S. et al., 2018. Multifactorial and multiple component interventions for preventing falls in older people living in the community. Cochrane database of systematic reviews. 7, p. CDO12221. Hu, G. Baker, S.P., 2012. An explanation for the recent increase in the fall death rate among older Americans: a subgroup analysis. Public health reports, 127(3), pp.275-281.
Kosse, N.M. al., 2013. Sensor technologies aiming at fall prevention in institutionalized old adults: a synthesis of current knowledge. International journal of medical informatics, 82(9), pp.743-752 Kramarow, F. al., 2015. Deaths from unintentional injury among adults aged 65 and over: United
States, 2000-2013, 3. NCHS data brief, (199), 199.
Lord, S.R. al., 2007. Falls Older People: Risk Factors and Strategies for Prevention, Cambridge University Press. Mandl, K.D. et al., 2014. Scalable Collaborative Infrastructure for Learning Healthcare System (SCILHS): architecture. Journal of the American Medical Informatics Association: JAMIA, 21(4), pp.615-620.
McMurdo, M.E.T., Millar, A.M. Daly, F., 2000. A Randomized Controlled Trial of Fall Prevention Strategies in Old Peoples' Homes. Gerontology, 46(2), pp.83-87. Available at: http://..doiirr/1111155///0
Moyer, V.A. U.S. Preventive Services Task Force, 2012. Prevention of falls community-dwelling older adults: U.S. Preventive Services Task Force recommendation statement. Annals of internal
medicine, 157(3), 19-
A APPENDIX

---

Under review as a conference paper at ICLR 2020
Beneficiaries in the insurance daim data base N=77,412,085
Beneficiaries age <65 at beginning of study period or without full insurance enrollment from July 2014 to June 31 2017 excluded N 77,252,338
Beneficiaries >65 at beginning of study period and with insurance enrollment from July 2014 to June 31 2017 159,701
Beneficiaries not enrolled Medicare Advantage program excluded 29,607
Beneficiaries enrolled in Medicare Advantage program N= 130,094
Beneficiaries with fall related diagnoses in observational and gap period excluded N=10,759
Beneficiaries without fall related diagnoses in observational and gap perlod N=119,335
Figure 3: Study cohort selection process from the health insurance claim database.
Table 2: Top 10 variables in model trained centralized data
varaible names
Mean Weights
variable type
401.1
0.0032
diag
729.5
0.0026
diag
110.1
0.002
diag
V76.12
0.002
diag
401.9
0.0017
diag
V70.0
0.0017
diag
781.2
0.0015
diag
530.81
0.0015
diag
782.3
0.0014
diag
atorvastatin
0.0014
med
Table 3: Top 10 variables in model trained in a confederated manner
varaible_name names
Mean_ Weights
variable_type
401.9
0.0058
diag
V70.0
0.0046
diag
729.5
0.0043
diag
V76.12
0.0036
diag
furosemide
0.0034
med
acetaminophen
0.0031
med
110.1
0.0029
diag
366.16
0.0028
diag
782.3
0.0028
diag
285.9
0.0028
diag
10

---

Under review as a conference paper at ICLR 2020
A
Lab tests
Medications
Medical diagnoses
Horizontal separation of patlents' data
Vertical separation patients' data
B
Observational period=12 months
Predictive features
Fall
Gap period=1 week
Follow-up period 24 months
Figure 4: (A)Two degrees of separation. Horizontal separation refers to fragmentation of an in- dividual's data across silos, for cxample across hospitals and climics. Vertical separation refers to differences in the domain, semantics and structure of the data, for example, data from pharmacies, clinies and labs, each in their own nodes. (B) Study period. Patient's data are divided into three periods. Observational period is 12 months, gap period week and follow-up period is 21 months. Diagnosis, medication and lab data from observational period used as predictive features for fall in follow-up period. The 1-week gap period is introduced to avoid complication of encounters happened directly before fall.


Reviews:
[{"review_id": "SkxvSC16tH", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper considers the problem of learning from medical data that is separated both horiontally (across different practices and centers) and vertically (by data type). The contribution is a \"confederated\" machine learning method that learns across these divides. The particular application considered here as means of illustration is that of fall prediction in the elderly. Specifically the authors investigate an ML approach to risk-stratifying elderly patients with respect their likelihood of falling in the next two years.\n\nThe basic challenge addressed here is learning in the setting in which different data elements are available only at specific sites, and it is assumed that they do not share data. In addition, it is assumed there are multiple distinct sites that have data corresponding to the respective elementes. However, it is assumed that labels (target vectors) are shared across all sites. This setup is simulated using available data. A simple distributed training scheme is outlined. \n\nThis is a potentially important problem worthy of study. I some major concerns with the present work, however. First, I do not think that ICLR is really the best venue for this work. The machine learning component of this is quite straightforward; basically SGD is performed iteratively on parameters associated with the data types \"owned\" by the respective (simulated) sites. Updates are then averaged over these parameter subsets. This is perfectly reasonable, but not terribly novel. The presentation of this is also much longer than it needs to be for the ICLR audience. I think this paper, in its current form, would be better suited for an audience more interested in clinical applications specifically (and I say this as someone quite appreciative of work on applied ML; it's just that the audience here will be more interested in methodological innovations.)\n\nWith respect to clinical utility: Do we really need ML to tell us about risk of falls? I mean, if we were to ask the MD who had seen these patients to perform a simple stratification (perhaps on an ordinal scale), would they not likely be able to do so reasonably well? The authors mention something like this, discussing the 'clinical screening process' which involves asking about prior falls. This seems like a really strong baseline. The authors argue that this is time-consuming, \n\nIn any case, is AUCPR an appropriate or useful metric here? In practice one would need to pick a threshold on which to act; perhaps a simulation that investigated doing so would provide a more meaningful evaluation. Although again a strong baseline here would probably be to ask physicians to risk stratify patients for interventions direclty (I appreciate that this would be a non-trivial experiment to run, but still).\n\nI also have a question regarding the simulation. I *think* the authors have randomly assigned patients to the respective simulated sites; is that right? This seems problematic because in practice patients would not be IID distributed in this way; sites would have their own patient populations which would affect the losses. This should be somehow taken into account in the simulation. \n\n\nOther comments\n---\n- I think I am missing something in the notation here. $Y_{si}$ is a 'binary label' but seems to vary across 'states' for an individual, is that right? Shouldn't this be constant for an individual? The paper states below that \"The output of the classifier is a binary variable indicating whether the beneficiary had a fall during the follow up period.\"\n\n- Labels were derived from ICD codes; was there any effort to spot check these? I am always a bit concerned about deriving labels from ICD and trusting them. \n\n- As far as I understand from 2.1, the authors have not included features extracted from notes in the patient history; is that right? Why not?\n\nSmaller issues\n--- \n- I would strongly suggest numbering your equations. Also, suggest using \\text while in mathmode for superscripts like `diag'. \n\n- \"Step 1..\" --> \"Step 1.\" (p3)\"\n\n- \"The parameter Θof f is randomly initialized\" --> missing space before \"of\"\n\n- On page 4: L(Xdiag, Xmed, Xlab, Θ) is written incorrectly.\n\n- page 4: \"Tstands\"  missing space.\n\n- page 5: \" fallst.\" --> \"falls.\"\n\n- Appendix Tables 2 and 3 both contain the typo \"varaible\" (should be \"variable\")\n\n- In Appendex Table 2, I suggest reporting results with a consistent amount of precision, e.g., 0.002 --> 0.0020 here.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "BylavgYCFB", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper describes a structure to approach the federated machine learning problem for hospitals. The approach does not seem very novel and it is hard to see what the representation learning challenges are. There is no open benchmark that the community can work on.\n\nI suggest that the paper focus on the method and not the private dataset used. If you cannot release a public dataset then maybe a synthetic dataset that presents known challenges you observe in private data. This can be used as a benchmark for the community to improve these methods. \n\nTypos:\n\n \"Step 1..Claims\"\n\nSome of the citations seem to be listing every author of the paper which is very hard to read the paper.\n", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "B1ggkv9ptr", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors propose a learning strategy to fit predictive models on data separated across nodes, and for which different set of features are available within each node.  \nThis concept is developed by introducing the concept of two degree separation across horizontal (nodes) and vertical (feature) axis. The proposed approach consists in an iterative scheme where i)  models at independentently trained at each site, and ii) models' parameters are subsequently averaged and redistributed for the next optimisation round.  \n\nThe problem tackled in this work is interesting, with an important application on medical records from > 100,000 individuals followed  over time. Unfortunately the paper is not clear in several aspects, and presents methodological issues. Here my main comments on this work:\n\n- The authors should definitely refer to the concept of meta-learning [1], which addresses modelling problems very close to the one presented in this work: training a meta-model by aggregating information from different learning tasks.  The paper should definitely compare the proposed methodology with respect to this paradigm. \n\n- The fact that the parameters can be averaged across nodes implies that they must be of same dimension. This is counterintuitive, as the dimension of the data represented at each site may significantly differ depending on the kind of considered feature. This aspect points to some methodological inconsistency.\n\n- There is no comparison with any other federated method, neither with any classification method besides a NN, at least with the aggregated data. Also it could have been possible to reduce the number of input features using simple dimensionality reduction previous to the NN, such as PCA. \n\n- Vertical separation importance: At the end it looks like diagnosis is the main driver for the classification, showing results that are comparable to the ones obtained with the aggregated data. It is therefore not clear whether the proposed application allows to clearly illustrate the benefit of using this method with regard to vertical separation.\n  \n- All in all, the paper appears in a draft form, and the text is often inconsistent. For example, there is often inconsistency in the number of branches, or types of data considered, figures are not self-explanatory and present notation and symbols not defined anywhere. The bibliography is given in a non-standard format. \n\n[1] Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. Finn, C., Abbeel, P., & Levine, S.  Proceedings of the 34th International Conference on Machine Learning-Volume 70 (pp. 1126-1135). ", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This manuscript proposes a strategy for fitting predictive models on data separated across nodes, with respect to both samples and features.

The reviewers and AC agree that the problem studied is timely and interesting, and were impressed by the size and scope of the evaluation dataset (particularly for a medical application). However, reviewers were unconvinced about the novelty and clarity of the conceptual and empirical results. On the conceptual end, the AC also suggests that the authors look into closely related work on split learning (https://splitlearning.github.io/) which has also been applied to medical data settings.
Decision: Reject