dataesr/scientific-paragraphs-categorization huggingface dataset:
Model assessment After the NMA model has been run, guidelines recommend that one assesses the convergence and fit of the model  [10] [11] [12]. In BUGSnet, convergence can be assessed using trace plots and other convergence diagnostics produced by the nma. diag() function. Lastly, the fit of the model and the identification of potential outliers can be carried out using the nma. fit() function which will produce a plot of the leverage values and also display the corresponding effective number of parameters, total residual deviance, and deviance information criterion (DIC). These latter values can be used to help determine or justify model choice when considering two or more competing models (e. g. between a fixed-or random-effects model) and to help identify data points that contribute heavily to the DIC and/or that are influential.
Results The following is a demonstration of some of the functions contained within BUGSnet (Table  1 ) and some of the possible outputs. To accomplish this task, we have recreated an analysis of a dichotomous outcome where studies had variable follow-up times described in the NICE-DSU technical support document 2 (referred to as "Data Example 3")  [17]. The BUGSnet code used to produce this analysis is available in the vignette titled survival in the BUGSnet documentation, and appended as a supplement to this article (see Additional file 1). Additional outputs are presented in the vignette as well as a more detailed description of how to conduct and report network meta-analysis, which is only presented here in brief. The evidence network used in this analysis consists of 22 randomized trials (including multi-arm trials) that examined the effects of six antihypertensive treatments on the risk of developing diabetes  [31].
The outcome for this data is the number of new diabetes cases observed during the trial period. The data is organized in the long format (i. e. one row per treatment arm), with variables indicating the study ID, the treatment ID, the number of patients, the number of events, and the mean age (and standard deviation) of participants for each treatment arm (see Table  4 ). The results of our package are concordant with those reported in the TSD as well results obtained with GeMTC (code and outputs provided as supplement to this article (see Additional files 2, 3, 4 and 5) and NetMetaXL.
Network meta-analysis We conducted an NMA on the Diabetes dataset by fitting a generalized linear model with a complementary log-log link function and binomial likelihood function to account for the dichotomous outcome and differing follow-up times between studies, which was specified through the use of nma. model(). To be consistent with the NICE-DSU technical support document, we specified a burn-in of 50,000 iterations followed by 100,000 iterations with 10,000 adaptations in the nma. run() function. We compared the fit of both a fixed-and random-effects model. According to a visual examination of the leverage plots and comparison of the DIC values produced by the nma. fit(), the random effects model would be preferred over the fixed effects model for this particular dataset because the DIC value is lower and because there are fewer outliers in the leverage plot (Fig.
Output We present results from the generalized linear model that we previously fit to the Diabetes dataset. As visualized in the SUCRA plot obtained from nma. rank(), the angiotensin-receptor blockers' (ARB) curve is consistently above the other treatments' curves suggesting that it is the most beneficial treatment with respect to the outcome among the treatments included in the Diabetes evidence network (Fig. 4 ). The effect estimates and credible intervals produced by the foregoing model are displayed in a league heat plot (Fig. 5 ) obtained using nma. league(). In Fig. 5 , one can see that the difference between ARB and other treatments are all statistically significant at the 95% level except for the ACE inhibitor and Placebo treatments.
Discussion BUGSnet is intended to be used by researchers when assessing the clinical efficacy of multiple treatments within the context of a submission to a journal or a health technology assessment agency. For conducting a contrast-based Bayesian NMA, the two main competing software packages that one may consider are GeMTC [15] and NetMetaXL  [16] , for which we have discussed limitations in the introduction. With BUGSnet, we  contrast, GeMTC can be used to conduct an NMA using entirely arm-level or entirely contrast-level data  [22]. Relative to GeMTC, another limitation of BUGSnet is that GeMTC currently provides a broader range of methods of assessing inconsistency such as the nodesplitting method and a broader range of meta-regression analyses such as subgroup meta-analysis. Since it is implemented within the R environment, some users may find BUGSnet more difficult to use relative to NetMe-taXL, which is implemented within Microsoft Excel.
At this point, arm-based models  [22]  have not been implemented in BUGSnet; the R package pcnetmeta allows such analyses, although it does not readily provide a complete suite of outputs like BUGSnet. We plan to address these shortcomings in future iterations of BUGSnet and interested users should check the previously mentioned URL for updates. Network meta-analysis is a rapidly evolving area of research with new methods constantly being developed  [32]. While the work presented within this paper provides the essential tools required to conduct an NMA in accordance with current guidelines, we plan to implement additional functions and features within this package, based on user feedback, to provide enhanced flexibility and to ensure relevance. Some of the preliminary requests for shortterm additions include: 1) additional functions for detecting inconsistency within the network such as the Bucher method  [27] ; 2) an option to allow the user to conduct an NMA using study-level effect estimates; 3) allowing for the relaxation of the proportional hazards assumption when analyzing time-to-event outcomes; 4) allowing for sub-group meta-regression and the inclusion of more than one covariate into the meta-regression model; 5) a function that will automatically generate a report or slide deck presentation of the results that could be saved as a pdf, html or Word.
Conclusions Here we present a new JAGS-based R package for conducting Bayesian NMA called BUGSnet. Relative to existing NMA software, BUGSnet provides an enhanced set of tools for conducting and reporting results according to published best-practice guidelines to help overcome the lack of quality identified within this body of literature. In addition to these features, we have attempted to provide ample documentation describing the use and implementation of BUGSnet to help promote the understanding and uptake of this software. Lastly, we plan to monitor the literature and to implement new features within BUGSnet based on the NMA analyst community to ensure that the package remains up-to-date with the latest advances in this rapidly developing area of research.
Acknowledgements We would like to thank the two reviewers for their feedback which helped improve this manuscript. We would also like to thank the attendees of our 2019 CADTH Symposium workshop and  UBC Network Meta-Analysis Workshop  for providing valuable feedback regarding the implementation and ease of use of BUGSnet. AB acknowledges the support of a start-up grant from the  University of Waterloo  and DBo acknowledges the support of the  Barrie I Strafford Doctoral Scholarship for Interdisciplinary Studies on Aging.
Availability and requirements Project name: BUGSnet Project home page: https://bugsnetsoftware. github. io/ Operating system(s): Windows 10 v1809 and Mac OS 10. 14 (may work on earlier versions but not tested) Programming language: R Other requirements: JAGS 4. 3. 0 Availability of data and materials All of the datasets and material contained within the manuscript can be accessed within the BUGSnet package via the BUGSnet homepage: https:// bugsnetsoftware. github. io/.
The Cherenkov Telescope Array (CTA), currently under construction, is the next-generation instrument in the field of very high energy gammaray astronomy. The first data are expected by the end of 2018, while the scientific operations will start in 2022 for a duration of about 30 years. In order to characterize the instrument response to the Cherenkov light emitted when cosmic ray showers develop in the atmosphere, detailed Monte Carlo simulations will be regularly performed in parallel to CTA operation. The estimated CPU time associated to these simulations is very high, of the order of 200 millions HS06 hours per year. Reducing the CPU time devoted to simulations would allow either to reduce infrastructure cost or to better cover the large phase space. In this paper, we focus on the main computing step (70% of the whole CPU time) implemented in the CORSIKA program, and specifically on the module responsible for the propagation of Cherenkov photons in the atmosphere.
In this context, the optimization of the simulation software has multiple purposes: faster production, higher statistics, better coverage of the phase space and eventually cost reductions when CTA is in production phase. Given the large amount of CPU consumed every year for CTA simulations, even a modest improvement of the code performances would allow to save several millions of CPU hours. In this paper we present our first performance optimization of the air shower simulation software used in CTA, called CORSIKA, which is described in Section 2. In Section 3, we present the results of the program profiling and in Section 4 the work on the optimization.
Corsika, an air shower simulation program CORSIKA  [3] , for COsmic Ray SImulations for KAscade, is a program for detailed simulation of extensive air showers initiated by high energy cosmic ray particles. Originally developed at KIT (Karlsruhe Institute for Technology) in the early 90's for the KASCADE experiment, CORSIKA has been rapidly adapted by other experiments for their specific use case. Nowadays, it is the reference air shower simulation software for all cosmic-ray, gammaray and neutrino astronomy experiments (e. g. Auger, CTA, IceCube, etc. ). For a given event, basically, the program tracks the initial particle until its first interaction in the atmosphere, then handles the development of the electromagnetic and hadronic cascades, particle energy losses, interaction processes and the production of the multiple secondary particles that are then propagated until they decay/interact or go below a given energy threshold, or reach the ground.
CORSIKA consists of a single main source code, written in Fortran 77, handling the description of the atmosphere, the stack of particles to propagate and the so-called particle transport. Physics interactions are handled in customized external packages. For the case of Cherenkov astronomy, an additional IACT/atmo package  [4] , written in C, implements the geometry of 3D arrays of Cherenkov telescopes, an improved description of the atmosphere (allowing the use of external atmospheric models) and the photon propagation through the atmosphere. In total CORSIKA consists of more than 10 5 lines of code. The description and response of the detectors are usually implemented in a separated software package.
Corsika profiling Before starting any optimization work on CORSIKA performances, we have performed a code profiling to identify which parts of the program are the most computationally intensive, using the Linux perf tool 1. The profiling was executed on a dedicated server (running Cen-tOS 7. 4. 1708 on a x86_64 Intel Xeon E5-2650 at 2. 20 GHz) using a set of standard input parameters, as those used in the official CTA productions. The program execution duration is controlled by varying the number of generated showers. After having checked the stability of the profiling results for different run durations, we have set to 1000 the number of simulated showers, which for the chosen set of input parameters takes about 5 minutes. Linux perf is an a event-based sampling tool. As sampling event, we have used the cycles event with a sampling frequency of 99 Hz.
In order to easily visualize the profiling results, we have processed the sampled data with the Flame Graph tool 2. The obtained profile is shown in Fig. 1 , where each box corresponds to a particular function or subroutine. The width of the boxes is proportional to the CPU spent in that function. The hierarchy of the function calls is also represented, each box having its parent function just below. Analysing the graph in Fig. 1 , we observe that 82% of the total CPU is consumed by the cerenk subroutine, of which about 50% is spent in the raybnd function, which we recall is responsible of the photon propagation through the atmosphere. These results confirm that photon propagation is one the most CPU intensive parts of CORSIKA. As explained in Section 2. 1, raybnd is called once per photon bunch.
Optimization strategy Given the size and the complexity of CORSIKA, it seems difficult to achieve a significant optimization by means of optimization options provided by the compiler 5. Even if compiler has been greatly improved in the last decade, whole program optimization is still a difficult task. Inter procedural analysis (IPA) is often limited by pointer aliasing and compiler heuristics. Nevertheless, in order to get eventual hints for further manual optimization, we have performed extensive tests, measuring the execution times for many different combinations of compilation options. The results of these tests are presented in Section 4. 1. The next step consisted in applying some manual transformations, focusing on the hot parts of the program identified by the profiling. As presented in the previous section, about 40% of the overall CPU time is consumed for the photon propagation through the atmosphere.
Since photon bunches propagate independently one from each other, the corresponding code might benefit from the application of vectorization techniques. The basic idea is to group photon bunches by 4 (or 8 according to the available instruction set on the target machine) into vectors and treat them all at once. In this context, we have also considered the use of highperformance (vectorized) mathematical libraries in specific parts of the program. Indeed, we have measured that 50% of the total CPU is consumed by calls to the libm. In Section 4. 2, we present the first transformations that we have performed in this sense. The vectorization approach is very promising, since it would bring a speed-up of about a factor 4 (or 8) for the concerned parts of the program, with relatively little code rewriting.
Automated optimization In order to explore the compiler optimization capabilities, we have prepared a test framework to easily compile and execute CORSIKA by varying the optimization options. First of all, we have run a reference version of the program, compiled with gcc 4. 8. 1. Secondly, we have chosen a list of compiler optimization options, subdivided in the following four categories: 1) standard optimization: O1, O2, O3; 2) loop optimization: ftree-loop-if-convert, ftree-loop-distribution, ftree-loop-distribute-patterns, ftree-loop-im, ftree-vectorize, funrollloops, funroll-all-loops, floop-nest-optimize; 3) advanced instruction sets enabling: mavx, mavx2; 4) link-time optimization: flto. Then, we have compiled CORSIKA with all possible combinations of these options, obtaining in total more than 3000 program versions.
Finally, each version was executed in exactly the same conditions, i. e. on the same host and with the same input parameters. The execution time has been measured with the Linux perf tool and compared to the reference version. Moreover, since CORSIKA is a stochastic simulation program, in order to ensure reproducible results, we have fixed the seed for the random number generators and checked that the output physics quantities (spatial coordinates and arrival time of the photon bunches) were identical to the reference version. The result of this test campaign is that we could achieve at most 1. 03 of speed-up for some specific set of options  [5]. Finally, we have also tried out the ffast-math flag, which optimizes the arithmetic expressions, without preserving however strict IEEE compliance.
Even in this case, we did not observe any significant improvement (maximum speed-up of 1. 06)  [5]. Moreover, as expected, the numerical values of the output physics quantities were slightly different from the reference version, so that their validity should be carefully checked. The whole test campaign took about 10 days of CPU, so that repeating the experience several times was not easily affordable. Nevertheless, by repeating our measurements several times for a few selected program versions, we have observed fluctuations of the order of a few percents. These results confirm that the optimization of a complex and large software, as CORSIKA, is beyond the compiler capabilities, while any eventual improvement necessarily go through manual transformations.
For this purpose, we have relied on external vectorized mathematical libraries. Among the various available, we have tested the SIMD vector libm  [6]  and the CERN's VDT  [7]  library. The main advantage of these two libraries with respect to others (i. e. Intel's SVML 6  , AMD's libm 7  ), is that they are open source, fully written in high-level languages (C and C++) and hence portable across systems. Another library we may consider in future is the libmvec 8  recently added to GNU glibc to support auto-vectorization in gcc. It is also open source, even if not portable on all architectures. The speed-up achievable with the two tested libraries depends on the considered function. In the case of exp, the SIMD vector libm announces a speed-up of a factor 4, while VDT library reports an interesting 77 speed-up factor (for SIMD vectors of length 4).
These differences may be explained by the fact that the SIMD vector libm certifies a certain level of accuracy, while there is no guarantee for VDT. Since performance and accuracy are intimately related, we plan to carry out a detailed study to determine the minimum required accuracy in air shower simulations. A similar study  [7]  has proven that typical High Energy Physics (HEP) applications do not require the high degree of accuracy of standard mathematical libraries which commonly provide correct rounding  [8]. The performance result obtained after our second transformation is an overall speed-up of 1. 16 for both the tested libraries. Regarding the accuracy, we did not observe any numerical difference on the output physics quantities with respect to the reference version. Encouraged by these results, we have then extended the vectorization to other mathematical functions within the IACT/atmo module.
This, however required a slightly larger code rewriting, such as the unrolling of the loop over the particle sub-steps (cf. Section 2. 1) in the cerenk subroutine. This third transformation allowed us to apply mathematical vector libraries also to trigonometric functions (sin, cos and asin) mainly used for the refraction angle calculation and the photon trajectory projection. The obtained speed-up thus increased to a value of 1. 20. More details about the three transformations just discussed can be found in  [5]. Finally, in order to apply vector operations also to arithmetic expressions, we have tested different high-level libraries, which provide an abstraction of low level SIMD instructions. Among them we mention: bSIMD  9  , Vc  [9] , UME (Unified Multicore Environment)  [10] , xsimd  10.
Perspectives In the optimization work presented in Section 4. 2, we have restricted ourselves to the Cherenkov propagation module, where we have applied vectorized mathematical functions only to a few selected instructions. The next step will consist in extending the vectorization to a number of arithmetic expressions we have already identified both in the Cherenkov propagation and in the Cherenkov production modules. Moreover, the implementation of some conditional expressions is non optimal with respect to compiler branch prediction. However, since the Cherenkov production module is written in Fortran (cerenk subroutine), it is not possible to rely on one of the above mentioned high-level libraries for vectorization (cf. Section 4. 2), so that we envisage to fully reimplement it as an external C++ module.
The fog system studied in this paper refers to FogAtlas, a platform designed to perform efficient deployment of fog computing applications according to the above guidelines. The tradeoff in this context is represented by edge resource occupation: compared to standard cloud technologies -based Fondazione Bruno Kessler, via Sommarive, 18 I-38123 Povo, Trento, Italy on overprovisioned datacenters -the business of edge infrastructure owners will not be able to rely on overprovisioning. Rather, they need to trade off premier service provision based on localized data processing and low round-trip time for storage, memory and processing capabilities of edge units  [6] ,  [7]. The paradigm of fog computing consists of a layered architecture, including a central cloud, a series of edge units, gateways to connect objects and, finally, objects which generate data and possibly actuate.
It handles microservice deployment and workload placement by managing a distributed fog infrastructure split into one cloud region and one or more fog regions. Actually, FogAtlas has a region-oriented architecture. In fact, existing OpenSource technologies such as OpenStack and Kubernetes handle well resources orchestration in traditional data centers where the cloud is centralized (optionally also spread across few large regions). However, they do not handle natively distributed and/or decentralized fog systems, where heterogeneous computing devices lay in diverse IoT regions and must be internetworked with a central cloud, often with bandwidth-limited and/or partially reliable connections. Ultimately, FogAtlas handles the orchestration among regions, while delegating intra-region orchestration to standard OpenStack or Kubernetes controllers.
The platform instantiates fog applications accouting for a set of optional deployment requirements. The application owner can specify requirements as constraints imposed to the deployment/execution of microservices in terms of requested resources and/or specific application needs. She is allowed to declare connections of IoT objects with a certain Microservice, see Fig 3. She can also require a specific target region for dispatching. In this context Microservice is a unit of software which plays a specific role as part of a larger fog application. But it can be deployed, upgraded or replaced independently from other microservices of same application. In FogAtlas it is distributed via Docker container images, which are stored in an Application Repository, in fact a Docker registry. FogAtlas adds above OpenStack and Kubernetes an Orchestrator, an Inventory, a Monitor and a set of RESTFul API together with some other components needed to operate the whole platform.
FogAtlas Implementation We provide hereafter a few technological details on Fo-gAtlas. In order to combine IaaS availability with flexible management of edge nodes, in FogAtlas the IaaS layer is provided by OpenStack while Kubernetes performs container orchestration. In particular, the OpenStack deployment adheres to the architecture proposed by th Edge Computing Group  [12]. Namely, the OpenStack controller lies in the cloud tier while compute nodes cover the edge devices. They are interconnected via "WANWide" links. A Kubernetes cluster is distributed on top of OpenStack virtual machines, covering both cloud and edge nodes. In case of small edge devices (with respect to available resources) OpenStack is not installed and Kubernetes workers are deployed directly on bare metal. Physical testbed and measurements: the FBK data center holds the cloud tier and the edge cloudlet tier.