dataesr/scientific-paragraphs-categorization huggingface dataset:
We can cite NAND FlashSim by Jung et al. [11]  providing a detailed disk model up to the plane level and SSDsim by Hu et al. [8]  allowing advanced command execution as copyback, multiplane and interleave commands. There are also the virtual devices simulators for Flash-based storage systems  [5]  [10] which implement very basic FTL functions, use analytical models for delays and propose very simple topologies without the device's internal detail. Thus, they are inappropriate for simulating accurately large and heterogeneous storage systems. Emulators are another kind of storage systems evaluation tools. They are more realistic tools but totally inflexible without any possibility to adjust hardware design parameters as OpenSSD  [1]  and BlueSSD  [13]. Also, there is an SSD simulator based on a virtual machine to allow the hardware evaluation in real time and uses a software on the top of the virtual machine to provide multiple design choice for the emulated SSD.
SIMULATOR DESIGN AND IMPLEMEN-TATION 3. 1 Global overview The proposed simulator is composed of several modules and submodules with a defined and specific role each. We have seven modules: the workload (WKLD), the hardware configuration (HWC), the pre-processing (PP), the volume driver (VDRV), the device driver (DDRV), the execution (EXEC) and finally the performance (PERF). OGSSim is developed in C++, for its speed and its modularity. This choice allows an easy expansion of any module if needed, for adding a new feature for example. Every module is handled by a thread, and all of them communicate via the push/pull Zero Message Queue (ZeroMQ), a socket API of message transmission developed by iMatix Corporation  [2]. A ZeroMQ is configured with a protocol, an IP address, a port and a queue type.
In our context, we used the API with the TCP protocol and only in local and we define for every queue, a port and an identifier. For example, the ZeroMQ between the workload module and the pre-processing interface has the identifier 5555. There are many types of ZeroMQ but we use only the push/pull ones, running as producer/consumer queues. A thread sends messages to a queue and another one receives them via the same queue. 1  shows the actual design of OGSSim and the information flow between the different modules. The simulator needs the trace and the hardware configuration files as inputs and generate the result file and related performance graphs at its output. The description of the different modules is given below:.
Workload The workload module (WKLD) constructs the request array structure from an input trace file. This one contains raw data describing request parameters. OGSSim supports three types of request format: • Basic: timestamp, type (read or write), address and size; • Advanced A: basic format and a group color or identifier; • Advanced B: basic format and both the generating host and the process identifiers. The WKLD module creates a request array, a shared memory structure and initialises it with the trace parameters. The request execution times and other intermediate values used by the simulator -such as the target device and the address on the device-are added to fill the rest of the structure during the simulation process. Because a request can be subdivided into subrequests, this array contains also a field for the parent index.
Hardware Configuration The hardware configuration module (HWC) creates the hardware structure in a shared memory from the XML input files. The first or global file describes the architecture and the bus configurations. Then, the device files indicated in the global one, describe the internal details of the hardware devices (disks, buses,. etc). OGSSim can evaluate the performance of a multi-layer system composed of tiers, volumes and devices. A tier is a bunch of volumes, each of which is a set of devices organised according to a specific topology as JBOD or a RAID (n+p). The created hardware structure holds the storage architecture configuration parameters, using four lists, one for every device type: buses, tiers, volumes and disks. These lists refer to each others according to the connectivity of the simulated system.
Volume Driver The volume driver module (VDRV) also has two roles. Its second role is to redirect the requests sent by the preprocessing module (PP) to the target device. To do so, first, the request index is sent to the layout model submodule. Second, it is decomposed into a set of subrequests taking into account that every subrequest is destinated to a unique disk. Third, the subrequest index is sent to the target disk device driver to be executed. The subrequests are stored in a secondary array located in the shared memory space. The size of this array is defined by the user in the configuration file of OGSSim. Each element of this array contains a boolean to indicate the subrequest status: True, for the living subrequest (to execute/execution in progress), False for the totally executed subrequest.
Device Driver The disk device driver module (DDRV) is composed of a set of maintenance submodules which are device-specific such as the garbage collection, the wear levelling or the defragmentation. Each submodule can be implemented with an algorithm defined by the user. Those submodules generate maintenance subrequests which are sent to the execution module. A new submodule can also easily be added to the simulator. Some technologies or data layout schemes necessitate synchronisation mechanisms to be achieved efficiently and guarantee that the simulation process will complete within a competitive time. The first synchronisation mechanism in OGSSim is related to the implementation of the RAID (n+p)  This mechanism uses special subrequests to indicate that a disk device needs to be synchronised with some others which are waiting for the new parity.
Performance The performance module (PERF) is composed of scripts written in python which generate visualisation tools (graphs, histograms,. etc) based on the collected data in the results files. These visualisation tools are plotted using a python plotting library named matplotlib  [9]. They can represent the workload per disk, the request decomposition or the mean service and response time of the requests, the disk utilisation during the simulation, the simulation time with the trace file size,. etc. Additional performance metrics can also be implemented, depending on the user needs.
The simulation duration Figure  4  shows the evolution of the simulation process duration or the execution time of OGSSim depending on both the number of requests in the workload trace (S trace 2 here) and the hardware configuration. The RAID (n+0) configuration needs a bit longer time, due to the request decomposition which necessitates more calculation because of the striping. The main clue here is the gap between the previous configurations and the RAID (n+p) with p different from zero. This gap is related to the synchronisation mechanism explained in subsection 3. 6 which generates a waiting time for all requested pre-read subrequests to join the execution module. Finally, the RAID (n+2) needs more time to be simulated than the RAID (n+1) because more subrequests are created for the second parity.
As shown on figure 6b and 6k, S trace2 presents higher mean service and waiting times on Archi4 because it contains more write requests than read ones so the write load is almost doubled. Figures  6c  and 6l  show the S trace3 behavior which is similar on both configurations. We can notice that, except for the highest step, the mean service times are close, but the mean waiting times on Archi4 are lower on Archi1 because of the load distribution and the dominance of reads. For the higher step, we can observe that both the mean service and waiting times are higher on Archi4. This is due to the combination of two factors: the big size of the requests and the fact that Archi4 has two data disks instead of four for Archi1. Large requests increase the service time, but there are less data disks in the configuration, so the load distribution across these disks is less efficient.
OGSSim spatial behavior This subsection aims to assess the spatial distribution of physical requests on the disks devices. Figure  7  shows the different device utilisation for the configurations Archi1, Archi3 and Archi4, submitted to OLTP and the three scientific traces. Thus, it allows to check the load balancing on the disks depending on the hardware configuration and the access profile. For the first configuration: Archi1 which is just a HDD-JBOD submitted to the sequential S trace1, a right behavior would be the utilisation of only one device as the entire trace does not cover a whole data disk or two disks (the end of one and the beginning of the next one). Also, the right and expected behavior for the random traces (S trace2 and S trace3) is a load balancing across the disks. The OGSSim results are shown on figure  7a  for the S trace1 and shows that only the first device is used, as it should be.
The CMS experiment at the CERN LHC has been designed with a 2-level trigger system. The Level 1 Trigger (L1) is implemented on custom-designed electronics. The High Level Trigger (HLT) is a streamlined version of the CMS offline reconstruction software running on a computer farm. Being able to identify b-quark jets (b-tagging) at trigger level will play a crucial role during the Run II data taking to ensure the top quark, beyond the Standard Model and Higgs boson physics program of the experiment. It will help to significantly reduce the trigger output rate which will increase due to the higher instantaneous luminosity and higher cross sections at 13 TeV. B-tagging algorithms based on the identification of tracks displaced from the primary proton-proton collision and/or on the reconstruction of secondary vertices have been successfully used during Run I.
b-jet identification: principles and algorithms Many channels studied in the CMS physics program involve b-jets. Top quark 1 , beyond the Standard Model 2 and Higgs boson 3 physics strongly rely on the capability to identify jets arising from the hadronization of b quarks. A good performance in terms of b-jet identification efficiency and rejection of non-b jets are essential to improve the purity and/or the significance of the selected events. Moreover, the use of b-tagging at trigger level is helpful if not mandory, to select efficiently the relevant events at trigger level while keeping the background sufficiently low to accommodate the online processing and bandwidth constraints. B-tagging algorithms exploit the properties of the B-hadrons such as their large decay lifetimes (cτ ≈ 450 µm) or the presence of leptons in the final state to separate them from light hadrons (made of u,d,s-quarks).
High Level Trigger The CMS trigger system is based on two consecutive levels. The Level 1 Trigger (L1) is an hardware level implemented on FPGA and custom-designed ASICs. The goal is to select events at maximum rate of about 100 kHz from the 20 MHz incoming rate delivered by the LHC. L1 treats in parallel information coming from the electromagnetic and hadronic calorimeters as well as from the muons chambers. A global decision is taken based on the presence of energy deposits compatible with the presence of photons, electrons, muons, jets or hadronically decaying tau leptons. The High Level Trigger (HLT) is a streamlined version of the CMS offline reconstruction software running on a cluster of commercial rack-mounted computers which consists of about 13000 processors. The goal is to reduce the rate to about 800 Hz for offline data storage.
flow Motion estimation SO 2 cameras are rapidly gaining popularity as a tool for monitoring SO 2 emissions from volcanoes. Several different SO 2 camera systems have been developed with varying patterns of image acquisition in space, time and wavelength. Despite this diversity, there are two steps common to the workflows of most of these systems; aligning images of different wavelengths to calculate apparent absorbance and estimating plume transport speeds, both of which can be achieved using motion estimation algorithms. Here we present two such algorithms, a Dual Tree Complex Wavelet Transform-based algorithm and the Farnebäck Optical Flow algorithm. We assess their accuracy using a synthetic dataset created using the numeric cloud-resolving model ATHAM, and then apply them to real world data from Villarrica volcano.
, 2009; Peters et al. , 2014a). MatLab code for this was kindly provided by Prof. Nick Kingsbury at the Engineering Department of the University of Cambridge. We used the implementation of the Farnebäck algorithm provided by the opensource computer-vision library OpenCV  (Bradski, 2000). Both of these implementations allow several parameters to be set by the user to control the operation of the algorithm. A description of the parameters along with the specific values used for this study is given in Appendix A. Broadly speaking the parameters set both the averaging level and the scale of features used by the algorithms. Selecting higher averaging and larger features will usually result in a more robust (less prone to noise, more tolerant of transient features) motion estimate, but at the expense of a less detailed motion field.
Villarrica dataset A ~20 min sequence of SO 2 camera images recorded at Villarrica volcano, Chile, on 8 February 2012 was used to test both algorithms' performance on "real" data. The images were captured using an Apogee Alta U47-SO 2 camera with an E2V CCD47-10, 1024 × 1024 pixel CCD and fitted with a 50 mm lens, giving a single pixel "footprint" of 1. 04 m 2 (distance to the plume was approximately 4 km). The camera used a filter-wheel to alternate between recording images with and without SO 2 absorbance. The images were calibrated into SO 2 column amounts using a co-located UV spectrometer, following the methodology of  Lübcke et al. (2012). Images were recorded every 4 s. As in the case of the ATHAM data, random noise was used to mask background and foreground regions prior to motion estimation.
Both algorithms produce a similar distribution of velocity vectors over the plume region. In general the DT-ℂWT technique produces a smoother velocity field. However, this is likely due to our choice of algorithm parameters rather than a fundamental difference between the algorithms themselves (for example a slightly higher level of averaging). Both algorithms produce a mean plume velocity of around 3. 5 m s -1 , which seems plausible for Villarrica. Closer inspection of the estimated motion shows, as demonstrated by Fig. 9 , that the magnitudes and directions of the velocity vectors from both algorithms match the motion of easily observable features. Fig. 10B  shows a comparison of SO 2 fluxes calculated for the Villarrica data set using the DT-ℂWT algorithm, the Farnebäck algorithm and the cross-correlation technique.
Discussion In general both algorithms performed similarly, with equal resultant errors when used for image registration, and comparable correlations with real flux values when applied to our synthetic data (Figs. 3  and 5 ). As previously mentioned, a wide range of motion estimation algorithms exist (e. g. (Geiger et al. , 2014) ) and it is not our intent to promote Fig. 7. Short time series comparing SO 2 fluxes calculated from synthetic images from the "open-top" ATHAM simulation using a simple cross-correlation method and the Farnebäck algorithm. The true fluxes (calculated using the ATHAM velocity field) are also shown. This time period was chosen deliberately as an example of poor performance of the cross-correlation technique. Fig. 8. An SO 2 camera image of Villarrica captured on 8 February 2012 using a 310 nm bandpass filter, with motion vectors estimated using the DT-ℂWT algorithm (left image) and the Farnebäck algorithm (right image).
The images have not been calibrated, and the colour scale is arbitrary. Normalised histograms showing the distribution of magnitudes of the velocity vectors produced by each method are also plotted. Motion vectors from the outside 10 pixels of the image have been excluded from the histograms, as have the zero velocity vectors. the DT-ℂWT or Farnebäck algorithms in particular as being superior for use with SO 2 camera images. That said, the results produced by both when applied to the Villarrica and ATHAM datasets were encouraging, and suggest that, even if not optimal, either algorithm would be a reasonable choice for this application. For real-time processing applications however, the Farnebäck algorithm is more suitable than the DT-ℂWT algorithm as it is ~6 times faster. Using a PC with a 2.
In the future it may be possible to combine the motion estimates and the apparent absorbance data into a single coherent plume model which could then be inverted for SO 2 flux taking into account continuity constraints and therefore minimising the errors from both the motion and the calibration. However, this is beyond the scope of this investigation. Synthetic plume images, such as those created using ATHAM, are clearly a useful validation tool for SO 2 flux calculation techniques as they enable quantitative error calculations to be made. In addition to validation, they also facilitate analysis and tuning of motion estimation algorithms which is not otherwise possible. For example, the choice of input parameters for an algorithm could be optimised by systematically varying its input parameters, applying it to a synthetic image sequence, and minimising the mean-squared error with respect to the known velocities (e.
It would, however, facilitate very useful studies into the effects of treating a real volcanic plume as a two-dimensional object when calculating transport speeds. It is important to remember that each pixel in a SO 2 camera image represents an integrated amount of SO 2 in the plume along the optical path of that pixel. Hence, estimating motion vectors from such an image can only ever yield a coarse approximation to the true plume velocity. Threedimensional plume model data would allow the errors introduced by estimating transport speeds from a two-dimensional projection of the plume to be constrained. Although both algorithms were able to accurately compute the transformation required to correct our distorted image, in reality the problem is more complex. The image pairs to be registered from SO 2 camera systems employing two camera units are captured at different wavebands and features visible in one image may not be present in the other.
The two algorithms presented here were similar in performance and would both be suitable for SO 2 camera applications, however, the Farnebäck algorithm is considerably faster, making it a better choice for real-time processing. A more thorough comparison of different algorithms is needed to determine the optimal algorithm for SO 2 camera applications. Using plume models, such as ATHAM, to produce test datasets for motion estimation algorithms provides a simple way to evaluate and compare their performance. Further research is required to ensure that the data produced are an accurate representation of real volcanic plumes, and extension of the model setup into three-dimensions promises to provide valuable insights into the errors associated with flux measurements. Supplementary data to this article can be found online at http://dx.
Acknowledgments This work was supported by the  UK National Centre for Earth Observation "Dynamic Earth and Geohazards" theme  ( NERC NE/F001487/1 : http://comet. nerc. ac. uk/). The Villarrica dataset was collected with help from  Kelby Hicks ,  Yves Moussallam ,  Kayla Iacovino  and  Tehnuka Ilanko  during fieldwork supported by the  Royal Geographical Society  (with IBG) with a  Geographical Fieldwork Grant. Additional financial support for the fieldwork was given by Antofagasta plc via the  University of Cambridge Centre for Latin American Studies. We are extremely grateful to  Fernando Gil Cruz  and OVDAS for their logistical support and help in the field. We also thank  Nick Kingsbury  for providing the implementation of the DT-ℂWT algorithm. We are grateful to the two anonymous referees of the manuscript for their beneficial comments.
This feature has not (yet) been incorporated into Duchamp, as its focus is primarily on threedimensional, spectral-line source-finding. Selavy represents the adaption of the Duchamp software for continuum sourcefinding and parameterisation. Selavy is the prototype ASKAPsoft source-finder that is under development and has been continually refined since the Data Challenge was run. Development has focused principally on improving the background and noise estimation, using a sliding box approach to measure the local noise, corresponding to the Selavy (box) mode used here albeit improved in reliability, and on improving the determination of the initial conditions for the Gaussian fit. This has benefited from input from the EMU source-finding group, in particular the approaches used for Aegean described in  Hancock et al.
A. 4 CuTEx CuTEx (Curvature Threshold Extractor,  Molinari et al. 2011 ) is an IDL-package that was developed (and is extensively used) within the framework of the Open Time Key Project on the Herschel satellite called Hi-GAL  (Molinari et al. 2010). This program gathered data in 5 bands  (70, 160, 250, 350, 500 μm)  of the entire Galactic plane, with the aim of studying the early stages of the formation of (high-mass) stars across the Galaxy. CuTEx was designed to enhance compact sources (sizes not larger than three times the instrumental point spread function) in the presence of an intense and highly variable background such as that seen in Herschel observations of the Galactic Plane. The CuTEx package is divided into two parts, a detection element that identifies sources and a photometry extraction element that measures their sizes and fluxes.
The photometry extraction part uses this list of candidates to determine the integrated flux and the background values on the original image (in our case the restored image) by fitting elliptical Gaussians, and measures the peak flux as well as the FWHM in two orthogonal directions and position angle (PA) of the fitted Gaussian. The fitting engine used is the Markwardt MPFIT package and strong constrains on the large number of parameters for each sources are applied to ensure convergence of the fit.
A. 6 PyBDSM PyBDSM 8  ('Python Blob Detection and Source Measurement', a Python source-finding software package written by Niruj Mohan Ramanujam, Alexander Usov and David Rafferty; Mohan & Rafferty 2015) calculates rms and mean images and then identifies islands of contiguous significant emission, computed either by a hard threshold or by using the False Detection Rate algorithm  (Hopkins et al. 2002). PyBDSM allows fitting of one or multiple Gaussians to each island and grouping of nearby Gaussians within an island into 'physical' sources. A modified fitting routine can also handle extremely extended sources. It can also decompose islands into shapelet coefficients. In addition, a PyBDSM module is available to decompose the residual image resulting from the normal fitting of Gaussians into wavelet images of various scales, and building these back into sources using the pyramidal morphological transform.
A. 7 PySE PySE 9  was developed within the LOFAR Transients Key Science Project  (van Haarlem et al. , 2013; Fender et al. 2007 ) as part of its real-time transient search pipeline. On the assumption that (relatively) fast radio transients are unresolved, the software is optimised for the detection of point-like sources. PySE processing fundamentally involves the following steps: 1. The image is divided into rectangular cells, and the pixel values in each cell are iteratively σ -clipped around the median; 22 Hopkins et al. 2. Bilinear interpolation of the mean across cells is used to derive a background map, which is subtracted from the data; 3. Bilinear interpolation of the standard deviation across cells is used to calculate an rms noise map; 4. Groups of contiguous pixels at some detection threshold over the rms noise are selected as potential source peaks; 5.