dataesr/scientific-paragraphs-categorization huggingface dataset:
We tried to make this point clearer in the revised version. The number of randomizations needed for a given dataset is not easy to set. Since increasing the number of randomizations a priori improves the modularity -and hopefully the accuracy -of the communities detected, we basically increased the number of randomization until obtaining no significant change in the communities detected. For the simulations, we selected 5 randomizations because the adjusted Rand index plot for 5 randomizations is confounded with that for 4 randomizations. For the Dream Challenge 1. for 5 randomizations is confounded with that for 4 randomizations. For the Dream Challenge dataset, we considered 10 randomizations because increasing this number was too heavy in computation time. Simulated networks with a community structure The method section has been extended to describe SBM network simulations more precisely.
Emre Guney Research Programme on Biomedical Informatics, the Hospital del Mar Medical Research Institute, Pompeu Fabra University, Barcelona, Spain Didier and colleagues present MolTi-DREAM, an update to their previous software for community detection in multiplex networks and its application to the disease module discovery using synthetic and DREAM challenge data. The new version of the software adds the possibility of edge and layer weights, the randomization of the underlying Louvain algorithm and partitioning of large modules into smaller modules based on user defined parameters. Based on their analysis of simulated and biological interaction data, they reaffirm that using multiple layers of networks and randomized repetition of module discovery could improve the accuracy. Overall the article is clearly written and the technical details are clearly explained with several exceptions I mention below: 1.
These tests show that the recursion approach provides overall a higher number of significant disease modules in the Dream Module Identification (DMI) challenge framework. Total number of modules obtained when running Molti with 10 randomizations on the 6 DMI challenge network for different values of the resolution parameter: Table  1  Number of significant Modules obtained with variations of Gamma: Table  2  Table  3  Table  4  Comparison with recursion (apply Molti in a recursive way to communities larger than 100): Table  5  For FDR= 0. 1 Figure  1  For FDR = 0. 05 Figure  2  For FDR = 0. 01 Figure  3  2. Randomization procedure is unclear. I believe the cost function used is the multiplex modularity given in Eq 1 (L) but the "move" leading to an increase of the modularity is not formally stated (sth like i, j: random_pick(argmax ci, cj L)).
Networks obtained from SBMs are generally not homogeneous, thus quite different from ER ones. The software performing the simulation of multiplex networks from SBMs, called "simul" is now available in the GitHub repository. We tried to display the standard deviation of the adjusted Rand index, but the result was quite confusing since the the error bars tend to overlap themselves (see below for the simulated sparse and dense networks). Figure 4 Figure  5  4. The coverage / precision of disease module discovery is missing. What is the number of all modules identified by the algorithm, how many disease modules are there (or could there be) in total? The latter question could be tricky to answer but I believe a lower and upper bound could be provided (e. g. , number of diseases and a number based on the GWAS hits for that disease divided by the min module size).
Indeed, the total number of discovered modules was not detailed in the manuscript. It was however used by the PASCAL tool to compute the number of significant disease modules, as the total number of submitted modules is used to correct for multiple testing. We added the total number of discovered valid modules (of size [7-100]) to the figure and table captions. Number of total modules detected by Molti for the results presented on: Figure  2. The numbers used in the Figure  2  have been added to the caption. Table  6  Figure  3 : Table  7  Table  2 : (nRandom = 10). These numbers have been added to the Table  2.
caption No Weights <-615 Weights <-585 The question of how many disease modules exist or could exist is indeed tricky to answer, and is partly discussed in the DMI challenge consortium paper. In this challenge manuscript, the full detail on the 180 GWAS disease/trait datasets, their classification, as well as the scoring method are detailed. Briefly, a module is considered significant if it is enriched in at least one GWAS-associated trait or disease according to the PASCAL approach. A sampling approach is also used to account for robustness in comparing the challengers' results. The figures 1, 3 and 4 of the consortium paper are in particular and attempt to dig in the reviewer's proposed direction. https://www. biorxiv. org/content/early/2018/02/15/265553 5. "combining biological networks in a multiplex generally increases the number of significant modules".
6. The choice of the for each biological network seems rather arbitrary. Could these weights be optimized (i. e. using leaderboard data) and their effect be tested on final data? We indeed considered different optimizations of the weights. This point was not clearly stated in the manuscript, but the weights were chosen based on the number of significant modules obtained in the application of Molti on the isolated monoplex networks. We rephrased the corresponding sentence to clarify this. However, we do not think that the results of the leaderboard can be used to choose the a posteriori best weights in the final data : the results obtained using the Leaderboard and the Final GWAS datasets (as well as using the union in the Total GWAS dataset) are not clearly correlated, making difficult to optimize the weights from the Leaderboard results.
The partition the most different is the one obtained from the cancer network. "following the PASCAL tool" identified using Changed "following" to "using" "Considering a greater number of layers. " lacks the main clause / verb OK, changed to "We observed that considering a greater …" "after more than four runs" upto four runs / repetitions OK, changed to "Increasing the number of randomization runs improves the results up to four runs. " "varies in a non-trivial way" non-trivial? OK, we changed to "is strongly dependent". This expression was related to the fact that we do not observe a clear correlation between the results obtained on the leaderboard and final GWAS datasets. No competing interests were disclosed.
https://doi. org/10. 5256/f1000research. 18430. r41029 Emre Guney Research Programme on Biomedical Informatics, the Hospital del Mar Medical Research Institute, Pompeu Fabra University, Barcelona, Spain I thank the authors for their rigorous amendments to the article and the detailed correspondence to the concerns raised by the reviewers. I only have the following minor comments: Gamma=1 could be added next to the Recursion label for clarity in Figures 1-3 in the response "We finished tied for second in the challenge. " => We tied for the second place "," is used instead of ". " to represent decimal in Tables 2-5 in the response No competing interests were disclosed. Competing Interests: I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.
Referee Report https://doi. org/10. 5256/f1000research. 16880. r36849 Arda Halu Channing Division of Network Medicine, Department of Medicine, Brigham and Women's Hospital (BWH), Harvard Medical School, Boston, MA, USA In their manuscript entitled "Identifying communities from multiplex biological networks by ," Didier et al. apply a network clustering method to identify randomized optimization of modularity communities that are significantly enriched in disease signatures. They build on their previously published community detection method, which is based on the greedy optimization (following the algorithm by Louvain et al. ) of multiplex modularity. The study is submitted as part of the DMI DREAM Challenge, which aims to evaluate different clustering algorithms by how well the resulting clusters are associated with GWAS-implicated disease genes.
Response 26 Oct 2018 Author Response 26 Oct 2018 , CNRS, France Anais Baudot In their manuscript entitled "Identifying communities from multiplex biological networks by randomized optimization of modularity," Didier et al. apply a network clustering method to identify communities that are significantly enriched in disease signatures. They build on their previously published community detection method, which is based on the greedy optimization (following the algorithm by Louvain et al. ) of multiplex modularity. The study is submitted as part of the DMI DREAM Challenge, which aims to evaluate different clustering algorithms by how well the resulting clusters are associated with GWAS-implicated disease genes. The authors test their approach on simulated datasets as well as DMI benchmark datasets.
They have three improvements on their original method: randomization, edge and layer weights, and recursive clustering for large clusters. The DMI DREAM Challenge itself is an important exercise addressing the non-trivial task of identifying biologically meaningful communities in molecular networks. This paper by Didier et al. takes on this challenge by treating the benchmark datasets as a multiplex network. While limited by the constraints of the challenge, I think the paper is an important contribution that offers an insight as to how multiplex methods fare on real-world biological networks of diverse and not necessarily related sources.
Kivela M, Arenas A, Barthelemy M, Gleeson JP, Moreno Y, Porter MA. 2014. Multilayer networks. Journal of Complex Networks 2:203-271 The authors use PASCAL to determine the disease-associated genes from GWAS, which are in turn used to determine the communities that are significantly enriched in disease signals. It is crucial at this point to convey to the reader very clearly what PASCAL does and how it's translated to "significant disease modules. " The authors should, with a couple of sentences, describe how PASCAL does the SNP-to-gene mapping, and then, more importantly, how the p-value weighted gene annotations from PASCAL are used in the enrichment to determine the significant disease modules. A good description of this would facilitate the interpretations of Figure 2-4.
The Methods section on controlling the size of modules could be more informative. "We tested different pre-filters (pruning leaves), parameters (resolution parameter, recursions, combination of graph weights for multiplexes) and post-filters (density, size, pruning leaves) in each leaderboard round. " This sentence contains a lot of information without really getting into details. The authors could elaborate on each item. For instance, the resolution parameter does not really seem to come into play (it is even omitted from the definition of multiplex modularity since it is set to 1). Recursion was used to limit the size of the clusters, and while the authors mention tests conducted in leaderboard rounds, no data here is shown as to how varying the resolution parameter changes the results. Perhaps the authors could either omit the section about the resolution parameter altogether or provide some supplementary figures on it.
Version 1 1 https://doi. org/10. 5256/f1000research. 18430. r41029 Emre Guney Research Programme on Biomedical Informatics, the Hospital del Mar Medical Research Institute, Pompeu Fabra University, Barcelona, Spain I thank the authors for their rigorous amendments to the article and the detailed correspondence to the concerns raised by the reviewers. I only have the following minor comments: Gamma=1 could be added next to the Recursion label for clarity in Figures 1-3 in the response "We finished tied for second in the challenge. " => We tied for the second place "," is used instead of ". " to represent decimal in Tables 2-5 in the response No competing interests were disclosed. Competing Interests: I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.
[ 1 ] 4 ) 14 Kivela M, Arenas A, Barthelemy M, Gleeson JP, Moreno Y, Porter MA. 2014. Multilayer networks. Journal of Complex Networks 2:203-271 The authors use PASCAL to determine the disease-associated genes from GWAS, which are in turn used to determine the communities that are significantly enriched in disease signals. It is crucial at this point to convey to the reader very clearly what PASCAL does and how it's translated to "significant disease modules. " The authors should, with a couple of sentences, describe how PASCAL does the SNP-to-gene mapping, and then, more importantly, how the p-value weighted gene annotations from PASCAL are used in the enrichment to determine the significant disease modules. A good description of this would facilitate the interpretations of Figure 2-4.
5 ) 5 The Methods section on controlling the size of modules could be more informative. "We tested different pre-filters (pruning leaves), parameters (resolution parameter, recursions, combination of graph weights for multiplexes) and post-filters (density, size, pruning leaves) in each leaderboard round. " This sentence contains a lot of information without really getting into details. The authors could elaborate on each item. For instance, the resolution parameter does not really seem to come into play (it is even omitted from the definition of multiplex modularity since it is set to 1). Recursion was used to limit the size of the clusters, and while the authors mention tests conducted in leaderboard rounds, no data here is shown as to how varying the resolution parameter changes the results.
These objectives are obviously coupled with each other, since the development of predictive models is the key building block of optimization procedures and, in turn, requires large datasets for assessing the model quality. On the modeling side, large databases are being developed throughout the world to assess from atomistic calculations the structure and physical properties of multi-constituent phases  [5] [6] [7] [8]. While this data is of great interest to feed larger-scale models, it is quite far from being able, from first principles, to describe the full multi-scale microstructure development in an alloy, even of a simple chemical composition. Therefore, the modeling effort needs to be carried out using a variety of scales and techniques, bridging them as much as possible to make it possible to reach the goal of numerically-driven alloy design.
On the experimental side, high-throughput methods can be classified into two broad categories. In the first, the traditional way of elaborating discrete sets of alloys is maintained, but procedures are developed to drastically increase the productivity of these methods. A compromise needs then to be found between the small size of the samples and the possibility to process them representatively and test their mechanical properties. In the second category, one finds the fabrication of compositionally graded alloys, and/or alloys with gradients of processing parameters. These materials enable to study continuously the material design space and, when associated with spatially resolved automated characterization techniques (such as X-ray diffraction), make it possible to acquire very large datasets coupling the chemical composition with the alloy's characteristics.
Even if massive parallel computing would reduce the overall calculation time, such a consideration clearly excludes time-demanding modeling approaches, like ab initio simulation, Monte Carlo simulation, molecular dynamics, phase field simulation, or the calculation of thermomechanical processes by finite elements. For this reason, the community of computational alloy design by combinatorial optimization has mainly adopted three categories of predictive tools: -those that can be calculated by simple equations, by solving elementary matrices or equation systems or by running through limited loops; this can include physical models -examples will be given later -but also, for instance, the cost of alloying elements; -thermodynamics using Calphad software; -data mining/machine learning tools, which will be presented later.
Complex numerical approaches, which are specific to each software, are then implemented to find the repartition of all elements in the alloy that gives the lowest possible value of G, i. e. the thermodynamic equilibrium. Possible outputs of computational thermodynamics are therefore the nature, fraction, and composition of phases, but also thermodynamic quantities like activity, enthalpy, entropy, free energy. Commercial and open-source software suites now exist, along with thermodynamic databases for most categories of engineering alloys. Many examples can be found of alloys designed using Calphad, and the method is now routinely used in the metallurgical industry, but we will here focus only on the use of combinatorial optimization relying on computational thermodynamics predictions. One of the first uses of Calphad predictions to guide the evolution of a genetic algorithm to design new alloys was published by Xu et al.
[15]. They questioned the Calphad software to obtain, for each composition, at the same time constraints on phase formation and equilibria as well as the driving force for nucleation of different types of secondary phases in the solid state. By maximizing the driving force, the authors intended to favor the formation of numerous nanosized strengthening precipitates by heat treatment, which was assessed by a simple nucleation rate model. Similarly, Calphad was coupled with a GA to design affordable superalloys in the system Ni-Fe-Cr-W-Al-Ti-Si-C-B  [16]. Iron was added to reduce cost but, observing that increasing the iron content in nickel-based superalloys would usually reduce the volume fraction of the γ reinforcing intermetallic phase Ni 3 (Al, Ti), it was tried to maximize both using a GA, while satisfying a set of constraints on microstructural stability.
Fabrication of compositionally and processing parameter gradient materials The design space of metallic alloys comprises dimensions of chemical composition and of processing parameters, such as temperature, time, stress, or strain. Each set of these parameters results in a given microstructure, which then translates in a specific set of properties. Although it is obviously impossible to explore the complete design space of a multi-constituent alloy, it is possible to obtain large datasets of microstructure by fabricating materials with continuous gradients of chemical composition and/or processing parameters. Other strategies for mapping the alloy design space involve the fabrication of many samples of different chemical composition using rapid alloy prototyping techniques; however, we will not describe them in details here and the reader is invited to read elsewhere the corresponding descriptions  [38, 39].
This extensive dataset obtained in one single experiment provides a very rich understanding of the effect of Mg on the clustering of Cu in this alloy system. Namely, it makes it possible to show that the effect of Mg on Cu clustering is monotonously increasing with Mg concentration, starting at the first measurable Mg content. The experimental dataset makes it possible to determine that above ∼0. 2%Mg, this effect saturates. Moreover, the influence of Mg concentration on the larger scale objects, despite being also monotonous, is shown to be quite different, since it is measurable only for Mg contents above ∼0. 1%Mg. Clearly, all this information would have been very difficult to obtain from a discrete set of alloy compositions. Following the microstructure development during natural ageing, we have performed a similar evaluation during artificial ageing at 155 • C  [49].
Combining this dataset with electron microscopy carried out on the two end compositions of the gradient, it was concluded that the two kinetics were representative of a different phase transformation path: T 1 phase for large Mg contents, and θ phase for low Mg contents, this transition being related to the solvus for the formation of a precursor phase on the dislocations that helps the T 1 phase to nucleate. Again, it would have been extremely difficult to characterize this sharp transition using discrete sets of alloy compositions.
Use of compositionally-resolved kinetics data for modeling As described above, characterizing continuously in compositional space the kinetics of microstructure evolution can improve our understanding of the effect of specific chemical elements on phase transformations and evidence bifurcations in kinetic paths followed during heat treatments. One other useful application of such databases, in more simple cases, where a single mechanism for microstructure evolution is active for a range of chemical compositions, is the use of the generated data for challenging the capability of the models describing this microstructure evolution. Indeed, microstructure models, e. g. , for precipitation kinetics, are usually compared to relatively scarce experimental data, generally on one alloy composition and a few heat treatment conditions.
Three experiments were performed at three different temperatures, and for each of them the precipitation kinetics was determined for 20 alloy compositions, thus generating a very large database for precipitation kinetics (actually the largest available at the present time to our knowledge). The choice of model to compare with this microstructure database was guided by the requirement that it should be sufficiently precise to capture effects of varying supersaturation and temperature changes on the different stages of precipitation, to describe the relevant parameters for comparison with the experimental data (average precipitate size, volume fraction and number density) and sufficiently fast so that it can be applied to the full experimental database. A natural choice was to use a Kampmann-Wagner class model describing the evolution of the precipitate size distribution discretized in size classes  [60].
This type of model being a mean field model, it describes the interaction of each size class with the average solid solution and is therefore well suited to the description of dilute precipitation as considered for our Cu-Co case. Fig. 14  shows the experimental database for precipitation kinetics, where the different alloy compositions are identified by the color intensity, and the model output. At each temperature, only three parameters were adjusted to describe the full range of alloy compositions: Co solubility in the Cu matrix, diffusion constant, and interfacial energy. The comparison between experimental data and modeling is overall rather satisfactory, except for the description of the nucleation stage, which points out that the classical nucleation theory is ill-adapted to describe this stage of precipitation in comparison with experimental data.
Concluding remarks The numerical and experimental approaches presented in this paper are still somewhat disconnected. On the numerical side, current fast models available for alloy optimization rely on relatively simple criteria such as maximizing the volume fraction of precipitates or the driving force for their formation. It is beyond the reach of currently available models to describe predictively the microstructure and related properties for a random set of alloy compositions and choice of processing parameters. We have shown how this lack of predictive models can be supplemented by data mining tools, capable of extracting from large scattered datasets enough relevant information for improving currently used alloys. However, if this strategy is now able to push alloy optimization towards its limits by finding the best possible trade-offs between characteristics, the approach is obviously limited to well chartered areas of alloy design space where a large amount of data is available, and will by construction not be able to design completely new alloy families, especially where undocumented mechanisms operate.