dataesr/scientific-paragraphs-categorization huggingface dataset:
The statements at lines 2 and 4 of Algorithm 1 enqueue x into the appropriate queue, and Algorithm 2 dequeues them at line 1, instead of reading them from disk. After the k-mers are distributed, buckets from a group are compacted in parallel. The CompactBucket routines are independent of each other, and hence we run CompactBucket(i) in parallel using all available processors. After a BCALM 2 finishes processing a group, it moves on to the next group. To reduce memory of the UF data structure, we created a minimal perfect hash function (MPHF)  (Cormen, 2009)  of all distinct k-mer extremities in the Reunite file (denote their number as d). The UF structure is therefore implemented as a vector v of MPHF indices, of total size dlogd. The UF class of a given k-mer is therefore v½x, where x is the MPHF index of the k-mer.
The BCALM 2 algorithm takes as input a set of distinct k-mers. However, in our implementation, BCALM 2 is developed using the GATB library  (Drezen et al. , 2014) , allowing it to seamlessly integrate GATB's k-mer counter. Therefore, the BCALM 2 software takes reads as input, and executes this k-mer counter prior to compaction. This is a disk-based algorithm inspired by KMC2  (Deorowicz et al. , 2014)  and DSK  (Rizk et al. , 2013). In this k-mer counter, k-mers are divided into partitions according to their minimizer, then each partition is counted independently. We modified the GATB k-mer counting algorithm so that partition files correspond exactly to bucket groups. We obtained further optimizations by representing strings using two bits per character.
Experimental results We evaluated the scalability of BCALM 2, and how it compares to other tools for compacting the de Bruijn graph. Experiments were run on a single machine equipped with an Intel Xeon CPU with 32 cores clocked at 2. 76 GHz and 512 GB of memory. We used two human sequencing datasets from the GAGE benchmark  (Salzberg et al. , 2011)  and from and two larger datasets from the spruce and pine sequencing projects  (Birol et al. , 2013; Zimin et al. , 2014).
Human datasets The first dataset is Illumina reads from a human chromosome 14 (36 million, 155 bp each, 2. 9 GB compressed FASTQ). The second dataset is Illumina reads from the whole human genome NA18507 (1. 4 billion, 100 bp each, 54 GB compressed FASTQ, SRA SRX016231). We first evaluate how BCALM 2 is affected by changes in the parameters k (k-mer size) and ' (minimizer length). Figure  2 (a) shows that BCALM 2 has nearly identical running times for 6 ' 10, and across all tested k values. Shorter minimizers sizes such as ' ¼ 4 create fewer buckets, hence limit parallel speedups. Second, we evaluate how well BCALM 2 scales with multiple processors. Figure  2(b)  shows that compaction and Reunite steps scale almost linearly with the number of threads. There remains overheads related to disk I/O.
, 2011; Meng et al. , 2014). For BCALM, the datasets were first processed using the DSK kmer counting software  (Rizk et al. , 2013)  to generate the set of kmers. In addition to the results shown in Table  1 , Minia took 27 h and 7 GB of memory on the whole human dataset (using identical k and abundance cutoff as in the table). For ABySS-P, the shown numbers include the k-mer counting step, which could not be extricated from the software. For the purposes of comparison, the k-mer counting step to generate the input for BCALM 2 completed in 46 mins and 2 GB of memory for the whole human dataset. Table  1  shows that BCALM 2 outperforms existing techniques in terms of running time. Since multiple graph compactions are done in parallel, BCALM 2 requires more memory than BCALM, however it is more memory-efficient than Meraculous 2.
Pine and spruce datasets We further evaluated BCALM 2 on two very large sequencing datasets: Illumina reads from the 20 Gbp Picea glauca genome (8. 5 billion reads, 152-300 bp each, 1. 1 TB compressed FASTQ, SRA056234)  (Birol et al. , 2013) , and Illumina paired-end reads from the 22 Gbp Pinus taeda genome (9. 4 billion reads, 128-154 bp each, 1. 2 TB compressed FASTQ, SRX016231). The k-mer counting step took around a day and <40 GB of memory for each dataset. Table  2  shows the performance of BCALM 2 on these two datasets, as well as unitigs statistics. Graph construction of the spruce dataset previously required 4. 3 TB of memory and 2 days on a 1380-core cluster  (Birol et al. , 2013) , while the assembly of the pine dataset previously required 800 GB of memory and 3 months on a single machine  (Zimin et al.
, 2014). Another execution of BCALM2 on the same datasets using a value of k¼61 shows similar performance, see Supplemental Table  1. Although we used the same sequencing datasets, several parameters differ between these previous reports and our results (e. g. k value, abundance cutoff, and whether reads were error-corrected). Hence run time, memory usage, and unitigs statistics cannot be directly compared. However, it seems reasonable to infer that BCALM 2 would remain 1-2 orders of magnitude more efficient in time and memory. In addition, we tested the robustness of BCALM 2 to an even larger number of erroneous k-mers by reducing the k-mer abundance cutoff to 2. The k-mer counting and compactions steps completed also within 2 days and 40 GB of memory. The resulting unitig file was much larger (resp.
Discussion In this paper, we present BCALM 2, an open-source parallel and lowmemory tool for the compaction of de Bruijn graphs. BCALM 2 constructed the compacted de Bruijn graph of a human genome sequencing dataset in 76 mins and 3 GB of memory. Furthermore, k-mer counting and graph compaction using BCALM 2 of the 20 Gbp white spruce and the 22 Gbp loblolly pine sequencing datasets required only 2 days and 40 GB of memory each. BCALM 2 is different from previous approaches in several regards. First, it is a separate module for compaction, with the goal that it can be used as part of any other tools that build the de Bruijn graph. While parallel genome assemblers offer impressive performance, there are many situations where differences in data require the development of a new assembler, and hence it is desirable to build For BCALM 2 and BCALM we used k ¼ 55, and ' ¼ 8 and ' ¼ 10, respectively; abundance cutoffs were set to 5 for Chr 14 and 3 for whole human.
Notably, they highlight that our local decomposition and particle handling is, for many applications, superior to classic SFC decompositions where the domain decomposition is known globally as well as to approaches where particles are handed around cyclically or sorted from a coarse regular mesh into the fine AMR structure  [4]. A comparison to a classic linked-cell code that does not support tunneling finally reveals the price we have to pay to enable tunneling. Throughout the present paper, we rely on our open source AMR software framework Peano  [30, 31]. It allows us to rapidely implement and evaluate the present algorithmic ideas. raPIDT was fed back into this framework and now is available as black-box particle handling scheme. Yet, none of the present algorithms is tied to that particular framework.
If a child of b, a is contained within b and is derived from b by k cuts through b along each coordinate axis. We continue recursively while we decide for each cell whether to refine further or not. The resulting graph is a kspacetree given by child of , a set of cells T and a distinguished root. For k = 2 the scheme mirrors the traditional octree/quadtree concept  [1, 15, 20, 21]. Our present code relies on the software Peano  [30, 31]  and thus uses k = 3. However, all algorithmic ideas work for any k ≥ 2. The distance from the root cell to any other cell is the cell's level. All cells of one level represent geometric primitives of exactly the same size, are aligned and do not overlap. The k-spacetree consequently yields a cascade of ragged Cartesian grids. The union of all these grids is an adaptive Cartesian grid.
Results Our case studies were executed on SuperMUC at the Leibniz Supercomputing Centre and the N8 Polaris system at Leeds. SuperMUC hosts Sandy Bridge E5-2680 processors clocked to 2. 3 GHz, Polaris hosts Sandy Bridge E5-2670 processors at 2. 6 GHz plus Turbo Boost. Both have two eight-core processors per node. SuperMUC runs Intel MPI 4. 1, and every 8192 cores (512 two-processor nodes) are called an island and are connected via a fully non-blocking Infiniband network. Beyond that core count, SuperMUC relies on a 4:1 blocking network. Polaris runs Open MPI 1. 6. 1, and every 192 cores are fully non-blocking connected to one Mellanox QDR InfiniBand switch. Beyond that core count, Polaris realises 2:1 blocking. Our code relies on good, i. e. close to optimal, placement, i. e. it uses as few switches as possible.
ppc ≈ 1000 here yields the best throughput. Polaris and SuperMUC results do coincide if scaled with the clock rate, i. e. the duplication of throughput rates on Polaris cannot be observed again. Hardware counter measurements with Likwid  [26]  reveal that cache misses for both approaches are negligible. They resemble exactly the results reported in  [1, 29, 32]  and references therein for other application areas. Our algorithms' AMR code base Peano  [31]  relies on a depth-first alike tree traversal  [30]  that picks up Hölder continuity properties of the Peano space-filling curve [32]. This implies advantageous spatial and temporal memory access characteristics. However, we expect a reimplementation with another code base to yield advantageous properties as well, as all PIC and PIDT ingredients follow a strict element-wise/local formulation and as the memory subsystem in above measurements is underutilised.
Detailed comparisons to the present Peano approach used in our PIT and raPIDT implementation can be found in  [30]. If the cuts along the SFC are known, it is possible due to the SFC code  [30]  to compute per particle the preimage of their position, i. e. an index along the SFC. This immediately yields the information which rank holds a particle. One thus can send any particle directly to the right rank after the move. Though our implementation base relies on a space-filling curve as well  [22, 29, 31, 32] , it does neither exhibit curve information to the application nor does it hold information about the global partitioning on any rank. However, we can manually expose this information to the ranks and thus reconstruct a SFC-based implementation. This allows us to compare an SFC-based code directly to the present alternatives: Our implementation using SFC cuts runs through the grid and moves the particles as for PIT.
In a second variant, we use the local SFC information to sort the received particles directly into the correct cells. Besides the alternative variants, we also compare PIT, PIDT and raPIDT to a plain linked-cell algorithm. For the latter, we use a modified PIDT code. The velocity profile is chosen such that no particle may tunnel. As a consequence, we manually switch off all multiscale checks, and we, in particular, skip any master-worker or worker-master communication. Such a test is thus a valid  setup to quantify the overhead introduced by the tunneling for the present software independent of the quality of implementation. No software ingredient is particular tuned and all codes rely on the same ingredients where possible. All setups rely on exactly the same domain decomposition, i. e. balancing differences are eliminated.
This analysis labels regions where no particle is moved in-between certain grid levels in the subsequent time steps. This allows us to eliminate reductions within the tree locally. Comparisons of the straightforward PIT with the dual grid strategy PIDT reveal that no scheme is superior to the other schemes by default. For small problem setups with slowly moving particles, PIT yields higher performance than PIDT. For big problem setups or fast particles, there is a sweet spot where the higher code complexity of PIDT pays off. Particular interesting is the fact that raPIDT picks up advantages of linked cell strategies, i. e. asynchronity of the ranks and exchange of data in the background, while arbitrary tunneling still is enabled. This might make it a promising candidate for the upcoming massively parallel age.
Acknowledgements The authors gratefully acknowledge the  Gauss Centre for Supercomputing e. V. (www. gauss-centre. eu) for funding this project by providing computing time on the  GCS Supercomputer SuperMUC at Leibniz Supercomputing Centre  (LRZ, www. lrz. de), as well for the support of the LRZ. This work also made use of the facilities of N8  HPC  provided and funded by the  N8 consortium and EPSRC  (Grant No. N8HPC DUR TW PEANO ). The Centre is co-ordinated by the  Universities of Leeds and Manchester. Special thanks are due to  Bram Reps  and  Kristof Unterweger  for their support on linear algebra and implementation details. Kristof also contributed to make the numerical experiments run. Robert Glas  and  Stefan Wallner  made a major contribution to the maturity and performance of the presented implementations as they used it for their astrophysics code and thus pushed the development.
Results UVB irradiation triggers mitochondrial fragmentation in NHEK. NHEK were irradiated with increasing doses of UVB (100-500 mJ/cm 2 ) and mitochondria stained with MitoTracker Green (6 h after irradiation) were visualized by live-cell confocal microscopy. UVB irradiation induced mitochondrial fragmentation as evidenced by the appearance of small round structures in cells throughout the field (Fig. 1a ). Two mitochondrial shape descriptors were then measured using Fiji/ImageJ software: aspect ratio (ratio between the major and minor axes of the analyzed particles) and circularity (ratio between their area and perimeter) to estimate the length and degree of branching of mitochondria. As shown in Fig. 1b , UVB irradiation significantly increased mitochondrial circularity in a dose-dependent manner with a concomitant decrease in aspect ratio.
Implementation of an automatic process for quantification of mitochondrial fragmentation. Although our semi-manual method offered accurate and reproducible measurement of changes in mitochondrial morphology, data capture was too slow and labor-intensive. Thus, we decided to implement a quantitative analysis pipeline (Mitoshape) to calculate the previously defined shape factors (circularity and aspect ratio) in an automated fashion. Staining with the vital fluorescent dyes CellMask Deep Red Plasma membrane Stain and Hoechst 33342 along with MitoTracker Green was used to determine cell shape and nucleus location in addition to mitochondrial morphology, which allowed computing the mitochondrial descriptors on a per-cell basis. Data were processed along the workflow outlined in Supplementary Figure  S3.
Inhibition of DRP1 expression attenuates UVB-induced mitochondrial fragmentation. To determine whether DRP1 is important for UVB-induced mitochondrial fragmentation, DRP1 expression was knocked down by transient transfection with siRNA-DRP1. After transfection for 48 h, high knock-down efficiency was observed for siRNA-DRP1 by western blot analysis of total cell lysates, whereas a control siRNA against luciferase had no effect (Fig. 4a ). Loss of the fission protein Dpr1 caused mitochondrial hyperfusion, which was easily detected by visual inspection of mitochondrial morphology (Fig. 4b ) and confirmed using the Mitoshape program (Fig. 4c ). Enhancement of mitochondrial fusion at baseline was induced by siRNA-DRP1 but not the control siRNA targeting the luciferase gene. Moreover, UVB-induced mitochondrial fragmentation was attenuated in siRNA-DRP1-transfected NHEK, with a slight increase in circularity and a minor decrease in aspect ratio (Fig.
2 × 10 5 cells per well) 48 h prior to staining with TMRM. Cells were incubated with 600 nM TMRM diluted in culture medium for 30 min at 37 °C in a humidified atmosphere containing 5% CO 2. Cells were then rinsed in DPBS and incubated in culture medium containing 150 nM TMRM. All treatments were performed in the presence of 150 nM TMRM. Following treatments, cells were trypsinized, rinsed in DPBS and then kept on ice. Flow cytometric measurements were performed with a BD LSRII flow cytometer (BD Biosciences). Data were analyzed using FlowJo software. Annexin V/Propidium Iodide assay. Apoptosis assay was performed by flow cytometric analysis of cells double stained with Annexin-V-FITC and Propidium iodide (PI). Cells were plated in 6-well plates (at 3 × 10 5 cells per well) 48 h prior treatment.
After being loaded, cells were washed twice with PBS and replaced at 37 °C before live-cell imaging. Immediately before imaging, cells were pulsed with 5 μ g/mL CellMask deep red plasma membrane stain (Life technologies) for 5 min at 37 °C. Confocal fluorescence microscopy and image analysis. Images were acquired on a Zeiss LSM710 confocal inverted microscope with a 63 × 1. 42 NA objective. Hardware and image acquisition were controlled by Zen 2012 software. Quantitative analysis of mitochondrial morphology was performed on confocal images using two different protocols. A semi-manual method was based on the manual delimitation of the cells using Fiji/ ImageJ software. An automatic method used Matlab software to create a mask for each cell after Hoechst 33342 (for nuclei) and CellMask deep red plasma membrane stain (for cytoplasm) stainings.
Acknowledgements This work was supported by grants from  SILAB S. A. R. J. is a recipient of a  CIFRE doctoral fellowship  from  ANRT. We thank SILAB efficacy platform for technical support and critical reading of the manuscript. We thank  Platim  ( UMS 3444 Biosciences Gerland-Lyon Sud ) for help with imaging and especially  Claire Lionnet  and  Christophe Chamot. This is publication ISE-M  2016-202.
Automation of the mounting of cryocooled samples is now a feature of the majority of beamlines dedicated to macromolecular crystallography (MX). Robotic sample changers have been developed over many years, with the latest designs increasing capacity, reliability and speed. Here, the development of a new sample changer deployed at the ESRF beamline MASSIF-1 (ID30A-1), based on an industrial six-axis robot, is described. The device, named RoboDiff, includes a high-capacity dewar, acts as both a sample changer and a highaccuracy goniometer, and has been designed for completely unattended sample mounting and diffraction data collection. This aim has been achieved using a high level of diagnostics at all steps of the process from mounting and characterization to data collection. The RoboDiff has been in service on the fully automated endstation MASSIF-1 at the ESRF since September 2014 and, at the time of writing, has processed more than 20 000 samples completely automatically.
The high-capacity dewar The RoboDiff mounts and unmounts samples to and from an HCD made from polyurethane foam and which has a maximum capacity of 240 SPINE standard sample holders in 24 pucks (Figs. 1a  and 3 ). The HCD consumes around 1. 5 l h À1 of LN 2 and its interior is maintained at slightly above atmospheric pressure. Ambient humidity is therefore kept outside the HCD, dramatically reducing ice formation. The LN 2 level in the HCD is maintained around the top of the pucks. The vials in which the samples are contained are thus always full of LN 2 when in the HCD (Fig. 3b ). Two independent access ports allow the filling of the HCD with samples without interfering with sample transfer, while a third port is dedicated to the barcode reader cameras necessary for the full tracking of samples during an experiment (Figs.
Software integration The functions of the RoboDiff are programmed within the CS8C controller. Routines are written in VAL3 using the Sta ¨ubli Robotics Suite v. 7. 3 (Sta ¨ubli Faverges SCA, Faverges, France). Trajectories and logic to be applied during robot movements are defined as tasks that can be launched by the StaubCom (EMBLEM Technology Transfer GmbH, Heidelberg, Germany; http://software. embl-em. de) generic VAL3 socket server on the controller side. The StaubCom library communicates through an ethernet connection and uses two independent ports dedicated to command and update services. The command service is used to receive keep-alive messages from the client, to launch or abort tasks and to modify VAL3 global variables on the fly. The update service sends asynchronous notification messages to the client machine including the status of the robot, input/output states and log or exception messages (Fig.
5 ). A Python library (R-lib) acts as the client of the StaubCom server, linking it to the beamline-control GUI MXCuBE  (Gabadinho et al. , 2010). All VAL3 routines are made available to user as Python functions (Python Control). Other ancillary devices and motors in the sample environment such as the dewar, actuators or additional motors are integrated in the standard beamline-control software. The library provides high-level Python objects both for convenience and for integration purposes. The main RoboDiff functions are wrapped into distinct objects depending on their role. Sample identification (data-matrix reading) and handling functions (mounting and unmounting) can be called as part of a generic Sample Changer object in MXCuBE 2. 0. Oscillation and helical scans are provided by a Diffractometer object.
1a  and 3a ) are used to check for the absence of a vial at the unloading position. The empty vial held at the Robot-Assistant park position is filled with LN 2 , moved to the sample position and replaced on the pin. In case of icing, a stream of pressurized gas under the vial helps in this procedure. At the entrance of the HCD the presence of both the pin and vial is checked. If the vial is missing, the sample is dropped for later recovery. If not, the sample is replaced in its original position. The entire cycle of loading/unloading (see Supplementary Movie S1) takes an average of 50 s, not including optical loop centring and the movement of beamline components. During loading, once the robot arm is at = 0 , the optical centre of rotation of the sample, as determined by the MXCuBE utility LUCID (https://github.
com/mxcube/lucid), is brought to the beam position by moving the robot motors. Movements during data collection, including line and mesh scans, are then performed using the air-bearing spindle and the high-accuracy YZ translation stage beneath the robot (Fig. 1b ). For this, a MUSST synchronization card is used to gather the signals of the ! axis, vertical and horizontal encoder positions, the Pilatus3 2M detector command and state, the shutter command and state, the beam intensity before and after opening the fast shutter, and the motor position of the robot centring table  (Nurizzo et al. , 2006). These data also serve as a diagnostic tool and link motor position and image number in order to find the best diffracting position of the crystal during X-ray centring protocols  (Svensson et al.
Tables  2  and 3  show the data-collection and refinement statistics from the three systems. The data-quality indicators and final electron-density maps (Fig. 6 ) for all crystals are excellent, demonstrating that the RoboDiff performs to the highest levels expected from a modern goniometer. In keeping with the ESRF data policy, the raw diffraction images for these data collections are available at Zenodo (http://zenodo. org/).
, 2016) , and 20 structures have been solved de novo to date (see, for example,  Kharde et al. , 2015; Muir et al. , 2016). In combination with software routines that locate, centre and collect data  (Svensson et al. , 2015) , the RoboDiff has proved to not only be the cornerstone of complete automation but is also an instrument that can collect high-quality data from samples.