dataesr/scientific-paragraphs-categorization huggingface dataset:
Effect of number of PCA modes removed Obtaining accurate photometry for planets embedded in speckle noise requires balancing the number of PCA modes utilized with the acceptable level of planet self-subtraction. Utilizing a greater number of PCA modes will more fully remove the speckle noise, but at the expense of removing flux from the planet itself. We tested the effect of using different numbers of PCA modes with the 18 August 2018 dataset (which appears by eye to be the highest-quality and most stable of the datasets) by re-running the fine-grid planet photometry pipeline with 4, 6, 10, and 16 modes. We used 6 equally spaced parallactic angle temporal bins for both b and c (as opposed to 5 for c) so we could directly compare results for the same sets of input data frames. Lightcurves are plotted in Fig.
Simulated variability results To quantify our sensitivity to variability in the 2018-08-18 epoch (the best quality data of our search), we simulate variable and nonvariable lightcurves by inserting and retrieving a suite of simulated planets at a similar radius from the star as HR 8799bc, but offset in position angle from the true planet position angle. Thus, we sample a similar region of the speckle pattern, but place each simulated planet sufficiently far from its real counterpart that they do not interfere with each other during the PCA analysis. In the simulated lightcurves, we estimate systematic noise sources to first order (e. g. varying Strehl ratio, seeing, etc) expected in the data by scaling the simulated planet flux by the normalized satellite spot aperture photometry (averaged across all 4 satellite spots) after background subtraction.
(iii) In contrast to  Apai et al. (2016) , we do not find a nightby-night stable pattern of correlations / anti-correlations between satellite spots. In particular, for our best datasets (August 2018), we find low levels of correlation / anti-correlation between satellite spots, because the satellite spot photometry is very stable on those nights and do not cover a wide range of values (in other words, the satellite spot lightcurves are quite flat in these epochs, so do not appear very correlated at all). (iv) Ensuring 5×FWHM sky rotation at the exoplanet radius to optimize speckle suppression but minimize self-subtraction, our ∼4. 5 hour observations allow 6 equal parallactic angle temporal bins for HR 8799b, 5 such bins for HR 8799c, but only 3 for HR 8799de. (v) Planet photometry / astrometry for HR 8799bc was retrieved using negative simulated planet subtraction, inserting scaled unsaturated images of the star at the Δmag, position of each planet in each raw data frame, for 6 temporal bins for b and 5 temporal bins for c.
Depending on epoch, we rule out non-shared variability in Δ()  -Δ()  to the 10-20% level over 4-5 hours. (x) To fully quantify our sensitivity to variability in the 2018-08-18 epoch (the best quality data of our search), we simulate variable and non-variable lightcurves by inserting and retrieving a suite of simulated planets at a similar radius from the star as HR 8799bc, but offset in position angle from the true planet position angle. Simulated planets are assumed to have sinusoidal lightcurves, with noise estimated using the mean satellite spot lightcurve. We ran a complete grid of simulated planet lightcurves with periods from 2 to 20 hours, amplitudes from 3% to 30%, and phases from 0 • to 330 •. For HR 8799b, for periods <10 hours, we are generally quite sensitive to variability with amplitude > 5%.
Plotting satellite spots against each other then mostly highlights the inherent scatter in the data, as for the best datasets, we covered only a small range of different star spot values. This produces a smaller Spearman  value, even though the data is of better quality. On 2018-10-18, and 2018-08-20, we find the strongest anticorrelations between adjacent satellite spots 1-4 and 2-3, weaker anti-correlations between satellite spots 1-2 and 3-4, and no or slight positive correlation between the two opposite sets of satellite spots. On 2017-10-08, we find that adjacent satellite spots 1-4 and 2-3 show relatively strong anti-correlation (but not much correlation between other adjacent satellite spots). On 2017-10-12, we find anticorrelation between adjacent spots 1-2 and 3-4. On 2017-10-13, adjacent spots 1-2 and 3-4 are somewhat anti-correlated, as well as opposite spots 1-3 and 2-4.
ACKNOWLEDGEMENTS We thank the anonymous referee for useful suggestions which improved this work. B. B acknowledges funding by the  UK Science and Technology Facilities Council (STFC)  grant no. ST/M001229/1. J. M. V. acknowledges support by  NSF  Award Number  1614527  and  Spitzer Cycle   14 JPL Research Support Agreement 1627378. Support for this work was provided by  NASA  through the  NASA Hubble Fellowship  grant  HST-HF2-51472. 001-A  awarded by the  Space Telescope Science Institute , which is operated by the  Association of Universities for Research in Astronomy, Incorporated , under  NASA  contract  NAS5-26555. A. Z. acknowledges support from the  FONDECYT Iniciación en  investigación project number  11190837  Based on observations collected at the  European Organisation for Astronomical Research in the Southern Hemisphere  under  ESO  programmes  095.
DATA AVAILABILITY STATEMENT The raw data underlying this article were accessed from the ESO archive facility at http://archive. eso. org/cms. html and processed at the SPHERE Data Centre, jointly operated by OSUG/IPAG (Grenoble), PYTHEAS/LAM/CESAM (Marseille), OCA/Lagrange (Nice), Observatoire de Paris/LESIA (Paris), and Observatoire de Lyon. The derived data generated in this research will be shared on reasonable request to the corresponding author.
Epidemiologists model the dynamics of epidemics in order to propose control strategies based on pharmaceutical and non-pharmaceutical interventions (contact  limitation, lock down, vaccination, etc). Hand-designing such strategies is not trivial because of the number of possible interventions and the difficulty to predict long-term effects. This task can be cast as an optimization problem where state-of-the-art machine learning algorithms such as deep reinforcement learning, might bring significant value. However, the specificity of each domainepidemic modelling or solving optimization problem -requires strong collaborations between researchers from different fields of expertise. This is why we introduce EpidemiOptim, a Python toolbox that facilitates collaborations between researchers in epidemiology and optimization.
In other domains, they have proven efficient at finding robust control policies, especially in high-dimensional nonstationary environments with uncertainty and partial observation of the state of the system  (Deb et al. , 2007; Mnih et al. , 2015; Silver et al. , 2017; Haarnoja et al. , 2018; Kalashnikov et al. , 2018; Hafner et al. , 2019 ). Yet, researchers in epidemiology, in public-health, in economics, and in machine learning evolve in communities that rarely cross, and often use different tools, formalizations and terminologies. We believe that tackling the major societal challenge of epidemic mitigation requires interdisciplinary collaborations organized around operational scientific tools and goals, that can be used and contributed by researchers of these various disciplines. To this end, we introduce EpidemiOptim, a Python toolbox that provides a framework to facilitate collaborations between researchers in epidemiology, economics and machine learning.
TOOLBOX ORGANIZATION Overview. The EpidemiOptim toolbox is centered around two modules: the environment and the optimization algorithm, see Figure  1. To run an experiment, the user can define its own or select the ones contained in the library. Appendix Figure  4  presents the main interface of the EpidemiOptim library. The following sections delve into the environment and algorithm modules. Environment: epidemiological models. The Model module wraps around epidemiological models, and allows to sample new model parameters from a distribution of models, to sample initial conditions (ρ 0 ) and to run the model for n steps. Currently, the EpidemiOptim library contains a Python implementation of an extended Susceptible-Exposed-Infectious-Removed (SEIR) model fitted to French data, from  Prague et al.
(2020). Environment: cost functions and constraints. In the EpidemiOptim toolbox, each cost function is a separate class that can compute c i (s, a), normalize the cost to [0, 1] and generate constraints on its cumulative value. The Multi-Cost module integrates several cost functions. It computes the list of costs c i (s, a)| i∈[1. Nc] and an aggregated cost c parameterized by the mixing weights β i | i∈[1. Nc]. The current version of EpidemiOptim contains two costs: 1) a health cost computed as the death toll of the epidemic; 2) an economic cost computed as the opportunity loss of GDP resulting from either the epidemic itself (illness and death reduce the workforce), or the control strategies (lockdown also reduces the workforce). In addition, we define two constraints as maximal values that the cumulative measures of the two costs described above can take.
ACKNOWLEDGMENTS Cédric Colas is partly funded by the  French Ministère des Armées -Direction Générale de l'Armement. Numerical computations were in part carried out using the PlaFRIM experimental testbed, supported by  Inria ,  CNRS  (LABRI and IMB),  Université de Bordeaux, Bordeaux INP  and  Conseil Régional d'Aquitaine  (see https://www. plafrim. fr/). This work is supported in part by  Inria Mission   COVID19 , project  GESTEPID.
The hybrid methods to combine the numerical simulation and analytical model might also be a solution, which has been applied in single HE design  [9] [10]  [11] , but the application in the dynamic performance study of HEN has rarely been discussed. Papastratos et al. [12]  followed the simulation approach by utilizing the SpeedUp dynamic process simulator. Mathisen and Morari  [13]  proposed a HEN dynamic model based on the lumped model for single HE, with the main focus on the discussion about the model features for HE and HEN. As for the dynamic performance, their target is to avoid the control difficulty caused by the inverse response, and the dynamic result is obtained through MATLAB/Simulink. In their model, the number of cells describing the HE dynamic model determines the accuracy, and they provided a range to select this number.
049t) (-0. 060t) (-0. 037t) (-0. 036t) (-0. 031t) c out T t 495+6704e -981e -13369e +143034e -15132e +0. 266t t s = ≤ ≤. 1, ( ) , 20 (-0. 049t) (-0. 060t) (-0. 037t) (-0036t) (-0. 031t) c out T t 455+81. 6e -7. 82e -635e +623e -67. 7e t s = > (38) The comparison between the analytical model and the simulation in Dymola is depicted in Fig. 8  Without the pathway analysis and the presented technique of decomposition, it is possible to reach the Laplace function of the outlet temperature by applying directly the HE model on all the HEs which link from the inlets to the outlets. However, the obtained function is quite complex and a numerical method for Laplace transform inversion is required to get the function in the time domain. In practice, the common tools like Matlab or WolframAlpha usually fail to get the inverse Laplace transform, even for a smaller network compared to the one used in the example.
The increasing availability of electronic health record (EHR) systems has created enormous potential for translational research. However, it is difficult to know all the relevant codes related to a phenotype due to the large number of codes available. Traditional data mining approaches often require the use of patient-level data, which hinders the ability to share data across institutions. In this project, we demonstrate that multi-center large-scale code embeddings can be used to efficiently identify relevant features related to a disease of interest. We constructed large-scale code embeddings for a wide range of codified concepts from EHRs from two large medical centers. We developed knowledge extraction via sparse embedding regression (KESER) for feature selection and integrative network analysis.
We evaluated the quality of the code embeddings and assessed the performance of KESER in feature selection for eight diseases. Besides, we developed an integrated clinical knowledge map combining embedding data from both institutions. The features selected by KESER were comprehensive compared to lists of codified data generated by domain experts. Features identified via KESER resulted in comparable performance to those built upon features selected manually or with patientlevel data. The knowledge map created using an integrative analysis identified disease-disease and disease-drug pairs more accurately compared to those identified using single institution data. Analysis of code embeddings via KESER can effectively reveal clinical knowledge and infer relatedness among codified concepts. KESER bypasses the need for patient-level data in individual analyses providing a significant advance in enabling multi-center studies using EHR data.
INTRODUCTION The adoption of electronic health record (EHR) systems has simultaneously changed clinical practice and expanded the breadth of biomedical research. For clinical research studies, EHR data are used alone or integrated with other established data sources such as registries, genomic data from biobanks, and administrative databases  [1] [2] [3] [4] [5] [6] [7]. EHR clinical data typically includes diagnostic billing codes, laboratory orders and results, procedure codes, and medication prescriptions. These comprehensive longitudinal data allow for studies to examine a broad range of hypotheses. However, this wealth of data also raises challenges in selecting and creating EHR features among thousands of options relevant to the study or condition of interest. Most current studies manually select individual EHR features and map specific EHR codes to represent each feature, requiring input from clinical and informatics experts.
In addition to being susceptible to subjective bias, this manual, time-consuming process cannot be scaled for projects requiring multiple phenotypes. Moreover, sharing algorithms across institutions often require performing this manual process to identify institution-specific codes and coding patterns in collaborative or replication studies. One potential solution is to create large-scale clinical knowledge networks, providing information about the dependency structure across different EHR elements, thereby providing information about the relationship of conditions and codes at a particular institution as well as equivalent codes across institutions. These data would no longer be associated with individual patient data and could be readily shared, facilitating multi-center collaborations. Creating a clinical knowledge network using EHR data requires two major advancements.
Existing approaches for data mining require patient-level EHR data, posing significant administrative challenges for data sharing across research groups and institutions. To overcome these challenges, we propose to transform EHR data into embedding vectors  17  , thus uncoupling the data from the individual patient. The downstream machine learning tasks would use the embeddings vector as summary data rather than individual patient data. Our use of embedding in this study refers to projecting an EHR code into another representation space. In the past decade, embedding vectors have been successfully derived for clinical concepts with textual data and various sub-domains of codified EHR data  [18] [19] [20] [21] [22] [23] [24]. These embeddings were primarily derived for specific applications and not for the creation of knowledge networks.
In addition, most existing word embedding algorithms tuned the key hyper-parameters, e. g. , the appropriate dimension of the embedding vectors, to optimize a specific downstream task. For example, the Code2Vec  19  tuned the embedding dimension via clustering task, and the Med2Vec  21  chose the dimension via future code prediction. However, this approach may limit the applicability of the learned embedding vectors to other downstream tasks. This study aims to develop a knowledge extraction pipeline via sparse embedding regression (KESER) with EHR data from two large healthcare systems. We present methods to derive embedding vectors using multiple types of codified EHR data at scale. Here, we choose the hyperparameters to ensure the general quality of the embedding vectors and retain embedding vectors with higher dimensions to further enable users to fine-tune optimal dimensions for their specific tasks.
We also investigate to what extent the dimensions affect the performance of different tasks. With embedding vectors from both institutions, we fit graphical models via sparse regression to construct knowledge networks that encode relatedness among features. We then demonstrate how these knowledge networks can select potential features in the development of an algorithm to identify patients with specific phenotypes using EHR data. Furthermore, we demonstrate that the knowledge network trained via integrative analysis of embedding data from both institutions outperforms those trained with a single institution's data.
Data pre-processing We used EHR data from two large hospital systems, the VA Corporate Data Warehouse (CDW) and Mass General Brigham (MGB). After aggregating four codified data domains (i. e. , diagnosis, procedures, lab measurements, and medications) into PheCode, CCS, RxNorm, LOINC codes, and manual lab concepts, and applying frequency control, we finally obtained a total of 9211 codes at VA and 5507 at MGB.
Fig. 1  Overview of KESER procedure. The KESER procedure includes four steps: (i) data pre-processing; (ii) representation learning using cooccurrence data and pointwise mutual information; (iii) feature selection at a single site; (iv) building a knowledge network across multiple sites. We conducted additional sensitivity analyses using different window sizes and k to construct the co-occurrence matrices based on a total of about 70 K patients from MGB Biobank. When varying window sizes from 7, 30 up to 60 days and k from 1, 5, up to 10, we observed that the embedding quality is the best when k = 1 but is not sensitive to the choice of window size (Supplementary Table  2 ). Table  1  summarizes the overall accuracy of between-vector cosine similarities in detecting known similarity and relatedness relationships with embedding vectors derived from either SVD-SPPMI or GloVe  25.
Similar to language translation, we learned orthogonal transformation between embedding vectors across the two institutions to enable the mapping of a given VA code to the corresponding MGB code  26. As summarized in Supplementary Table  3 , the top-1 and top-5 accuracy of code mapping is around 38 and 67% for VA medication codes → RXNORM and around 42 and 74% for PheCode → PheCode using embeddings of dimension d auc. The code mapping accuracy is fairly comparable when using a larger d snr. The observed code mapping accuracy is comparable to the translation accuracy between different languages reported in the literature  26, 27.
Using codified EHR data from 68,213 MGB Biobank participants, we compared the performance of two supervised phenotype algorithms, the adaptive LASSO (aLASSO) and random forest (RF), trained with existing feature selection strategies to those trained with KESER-selected features. Those existing feature selection strategies included the main PheCode of the disease only (PheCode), all features (FULL), or informative features selected manually or extracted using unsupervised algorithms such as SAFE  15. The accuracies of the aLASSO phenotyping algorithms trained with different feature sets are summarized in Fig. 3  and more detailed comparisons including the RF results are given in Supplementary Fig. 10. Given the same feature set, the RF algorithms generally performed slightly worse than the aLASSO algorithms in part due to overfitting.
The relative performance of the RF algorithms trained with different feature sets is similar to those from aLASSO. The algorithms generally attained higher performance using embeddings from SVD-SPPMI than those from GloVe. The results are quite similar when using KESER INT versus KESER MGB and hence using MGB embedding information may be sufficient for phenotyping at MGB. Hence we focus our discussions below on the aLASSO algorithms and for KESER, we focus on KESER MGB with SVD-SPPMI embeddings for brevity. Across the 8 phenotypes, phenotyping algorithms trained via aLASSO with KESER MGB -selected features attained higher AUCs and F-scores than those based on PheCode alone or using FULL features, and similar AUCs as those trained with SAFE features. On average, the AUC of KESER MGB with SVD-SPPMI based algorithms was 0.
052, 0. 144, and 0. 007 higher than those based on PheCode, FULL, and SAFE features. The average F-score of KESER MGB based algorithms was 0. 173, 0. 157, and 0. 013 higher than those based on PheCode, FULL, and SAFE features. The 95% confidence intervals of the accuracies associated with algorithms trained with KESER-selected features are similar to the SAFE features, while those of the FULL features and main PheCode alone are substantially wider.
Knowledge mapping by performing node-wise KESER We summarized the clinical knowledge network, namely a knowledge mapping, by performing node-wise KESER across all PheCode and RxNorm (https://github. com/celehs/KESER). Figure  4  is a screenshot of the webAPI, given a specific target drug, RxNorm 214555 for etanercept. The node-wise knowledge extraction aims to find the neighborhood codes related to the target code etanercept. Figure  4A  displays codes connected to etanercept from KESER INT , which consists of 36 PheCodes, 49 RxNorm codes, 3 CCS codes, and 15 lab codes. Confirmed by domain experts, the results were largely consistent with clinical knowledge. For example, diseases commonly treated by etanercept, such as sacroiliitis, juvenile rheumatoid arthritis, RA, ankylosing spondylitis, were selected by the knowledge network.
Drugs, procedures and lab tests usually used together with etanercept, such as methotrexate, arthrocentesis, HLAB27, and CRP, were also selected. Figure  4B , C displays the local network based on KESER VA and KESER MGB. We observed four lab codes uniquely identified by VA and nine lab codes uniquely identified by MGB. The discrepancy of the local networks at VA and RPDR lies only in lab codes. This is expected because the majority of the lab codes are unique to the site, resulting in high cross-site heterogeneity in lab coding. By integrating data from both sites, KESER INT is able to achieve higher accuracy in reflecting clinical knowledge. These results demonstrate that KESER can successfully select informative and clinically meaningful features that can be used effectively for phenotyping and other downstream analyses.
DISCUSSION The KESER approach efficiently summarizes patient-level longitudinal EHR data into hospital-specific embedding data and enables the extraction of clinical knowledge based only on summary-level data. This summary data generated based solely on relationships between codes, and clusters related codes together, which provides ready information on features that may be important for identifying or studying different phenotypes. The KESER approach enables the assessment of conditional dependency between EHR features by performing sparse regression of embedding vectors without requiring additional patient-level data. In this paper, we demonstrate the advantage of integrative analyses across sites in detecting known associations. Ultimately, we believe this innovation provides a potential solution for barriers facing the much-needed multi-center collaborative studies using EHR data.