dataesr/scientific-paragraphs-categorization huggingface dataset:
Denoising applied to spectroscopies -part II: decreasing computation time Spectroscopies are of fundamental importance but can suffer from low sensitivity. Singular Value Decomposition (SVD) is a highly interesting mathematical tool, which can be conjugated with low-rank approximation to denoise spectra and increase sensitivity. SVD is also involved in data mining with Principal Component Analysis (PCA). In this paper, we focussed on the optimisation of SVD duration, which is a time-consuming computation. Both Intel processors (CPU) and Nvidia graphic cards (GPU) were benchmarked. A 100 times gain was achieved when combining divide and conquer algorithm, Intel Math Kernel Library (MKL), SSE3 (Streaming SIMD Extensions) hardware instructions and single precision. In such case, the CPU can outperform the GPU driven by CUDA technology.
It is thus essential to optimise the computation procedure. Different approaches have been chosen in literature: specialised processors  (33) , wavelet transformation before performing the SVD  (34) , divide and conquer method  (35)  or sparse matrices  (36, 37). Another approach is General Purpose computing on Graphics Processing Unit (GPGPU)  (38). SVD has been applied multiple times using GPGPU  (39) (40) (41) (42). Two programming languages are available: CUDA (Compute Unified Device Architecture) for Nvidia graphic cards  (43)  and OpenCL (Open Computing Language) for all graphic cards (Graphic Processing Unit, GPU) and processors (Central Processing Unit, CPU)  (44). In addition, Dongarra et al. developed very efficient algorithms combining CPU and GPU  (45, 46) , what is called heterogeneous computing  (47).
Recently Man et al proposed a Java implementation of SVD for NMR (48) using CPU  (49)  or Nvidia GPU  (50). Pending questions can be raised on these Java applications: (i) how powerful does the computer need to be? (ii) how long does computation take? (iii) how large can be the data set? (iv) is the used algorithm efficient? (v) will other programming languages give better results? In this second part (II), after providing some experimental details in section "Materials and methods", we benchmarked SVD using Java, on various CPU and Nvidia GPU ranging over 10 and 6 years, respectively. We then focussed on algorithms and precision under Matlab. In a further step, we tried to decorrelate software libraries from hardware capabilities (Single Instruction Multiple Data, SIMD)  (51). Finally, we compared Java, Matlab and Python to reach the fastest possible denoising computation.
Measurement of SVD computation times Special care has been taken to validate measured times: the computer was checked to be idle, without any update running and without an active internet browser window, which would use the graphic card through Adobe Flash module. Computations were systematically repeated and results were highly reproducible. Nvidia drivers ranging from 331. 113 to 352. 30 were used, either under Linux Ubuntu 14. 04, Centos 5, Fedora 21, 22 or Windows XP, 7 SP1 or 8. 1. Under Windows it was necessary to modify the TdrLevel registry key to 0 in order to avoid graphics driver failure  (57). Under Java (Oracle Corporation, Redwood Shores, CA, USA) and Matlab (The MathWorks, Inc. , Natick, MA, USA), 29 Si FID was used up to 4028 points. Above this value, 87 Sr FID was used. The corresponding FID was truncated if needed to the desired data length.
Under Python  (58) , data set was a simple list of increasing values with the corresponding length. Unless otherwise stated, k = 25 singular values were kept for low-rank approximation. This value corresponded to the major spikelets observed on 29 Si spectrum (Figure  1a ) and was not changed for coherence along the series. The measured delays are the sum of decomposition and low-rank approximation steps, including all processor to graphic card latencies, if relevant.
Java Two applications are available online: one for CPU  (49)  and the other one for Nvidia GPU  (48, 50). While CPU version calls JAMPACK library  (59) , GPU version calls CULA R15  (60). The CPU 32 bits version failed above a matrix size of 1025 × 1024 = 1. 0e6 points, whatever the CPU used. The same problem was observed with GPU 32 bits version above a matrix size of 3005 × 1024 = 3. 1e6 points. The reason is that Java heap space is limited to around 1. 5 GB for 32 bits applications  (61) , while it is 16 exabytes for 64 bits applications. This memory amount corresponds not only to the data, but also to the program and its libraries. 64 bits applications are not compatible with 32 bits Java runtime environment and 32 bits operating systems. As the source code was not accessible and no internal timer was implemented in these Java applications, computation times were measured with a handheld chronograph giving a time resolution of 1 s for both SVD step and low-rank approximation step.
Matlab Three versions were tested: R2010a, corresponding to the most recent compatible with CULA library free (in version R14)  (62) ; R2014a, corresponding to the most recent for old graphic cards with Compute Capability (CC) less than 2. 0, such as GTX 260; and R2015a, corresponding to a recent version. FID were imported thanks to matNMR  (63). Under R2014a and R2015a, their respective Parallel Computing Toolbox was added to use gpuArray function. Computation times were measured with an internal timer. The source code is available in file Figure_II. 4a_II. 4b. m of  (64).
Python No significant time difference was observed between Python 2. 7 and 3. 5 versions, neither between 32 and 64 bits versions. The source code is compatible with all of these options and is available in file Figure_II. 5. py of (64) (CPU and GPU). Different SVD implementations were tested with NumPy 1. 10. 1 (65), SciPy 0. 16. 1  (66)  and Scikit-Cuda 0. 5. 0  (67). The last one required PyCUDA 2015. 1. 3 (68)  and CULA R18 free  (60). Computation times were measured with an internal timer as well. Under Linux, Automatically Tuned Linear Algebra Software (ATLAS)  (69)  and Open source Basic Linear Algebra Subprograms (OpenBlas) (70) development libraries, separately either one or the other, were installed as rpm packages. For each library, NumPy and SciPy were build with the pip mechanism 1.
Java CPU and GPU benchmarks In an attempt to characterise the hardware needed to compute SVD, we used CPU ranging over 10 years and GPU ranging over 6 years, for both desktop and laptop computers (Tables  1  and 2  and Figure  2b ). According to Figure  2a , we were changing hardware level, supposing that everything was optimised at software level and measuring durations at human level. Study has been restricted to Intel CPU and Nvidia GPU because no AMD CPU was available in the laboratory and AMD GPU are not compatible with CUDA. Results are presented on Figure  3a. We noticed first that computation were much faster for GPU than for CPU, with 2 to 23 s for GPU (dashed lines) and 67 to 500 s for CPU (top right end of plain lines), i. e. 22 to 34 times speed increase, for a quite small matrix of 1025 × 1024 = 1.
0e6 points. This almost square shape was due to construction of the Hankel matrix with a dataset containing an even number of points. Indeed, in order to not overwrite the corner point, it is necessary to add one row over columns. No significant time difference was seen with a true square matrix and thus this small shape difference will be neglected in the following. Surprisingly, even a low-end GPU of 2008 (8400 GS) was surpassing a middlerange CPU of 2013 (Core i5 4670K) (with 23 and 67 s, respectively). This behaviour was really intriguing, as we would expect at least similar computation performance (75). However, when checking CPU activity, we observed that, with the CPU application, only one core was busy, what is called mono-threading. On the contrary, with the GPU application, not only GPU was fully busy, but also all the available CPU cores were used, what is called multi-threading.
This difference between mono-and multithreading is explored in section"Influence of algorithm under Matlab". When increasing the square matrix size, a linear trend was visible in logarithmic scale, down-shifted for faster hardware (Figure  3a ). However, computation time jumped when going from a rectangular matrix (diamond symbol) to a square one (square symbol), even if matrix size was not so different. This behaviour was observed for both CPU and GPU at 1537 × 512 = 7. 9e5 vs. 1025 × 1024 = 1. 0e6 points, and similarly for GPU at 3005 × 1024 = 3. 1e6 vs. 2015 × 2014 = 4. 1e6 points. This denoted that the number of mathematical operations dramatically increased for a square matrix. An explanation could be the use of reduced SVD for rectangular matrices, not computing the last rows and columns of U and V T unitary matrices (see part (I) of this work (  5 )).
Following this rule, the largest tractable matrix was 6657 × 6656 = 4. 4e7 points on a GPU with 2 GB of memory. In this section we have seen influence of hardware on SVD computation time under Java. As stated above, the time difference between CPU and GPU Java application was intriguing, especially when comparing the low-end graphic card 8400 GS to the middle-range processor Core i5 4670K. Additionally, the CPU and GPU application were mono-threaded and multi-threaded, respectively. This was typically an algorithm problem and we explored it using Matlab.
To explore algorithm influence we focussed on two computers, one from 2008 with a Core 2 Quad Q8200 and a GTX 260 under Linux, and the other one from 2013 with a Core i5 4670K and a GTX 660 under Windows. Both were in the same price segment and reflected middle-range equipment available at those dates. The used Matlab versions were detailed in section "Materials and methods". According to Figure  2a , we fixed hardware level and observed software level influence.
Matlab R2010a Under Matlab R2010a, DP computation times on CPU were already smaller than those with Java CPU application for a matrix of 1025 × 1024 = 1. 0e6 points (plain dark blue line on Figures  4a  and 4b ): 78. 2 s instead of 149 s for Core 2 Quad Q8200, and 11. 6 s instead of 67 s for Core i5 4670K. It should be noted that computation times can be divided by two when using single precision (plain red line), and by two again with noncomplex data (not shown). Java CPU application used complex numbers and accordingly to the above results, it presumably used DP. While Java used only monothreading, Matlab computation started with a multi-threaded step and kept on with a mono-threaded one. This already denoted a different algorithm between the two programming languages. Under this Matlab version it was also possible to use GPU with CULA, which was the library implemented under Java GPU application.
Results were similar between Java GPU and Matlab R2010a + CULA + SP GPU applications (dashed red line). However, as we used the free version of CULA, DP computation was not allowed on GPU and R2010a + CULA + DP (plain black line) felt back on CPU with LAPACK library in multi-threading mode. As a consequence, strong improvement was observed against R2010a + DP (plain dark blue line). At this point, an order of magnitude on computation times has already been gained for CPU DP under Matlab.
Matlab R2014a Further improvement on CPU was obtained with R2014a + DP (plain yellow line on Figures  4a  and 4b ) being almost two times faster than R2010a + CULA + DP (plain black line), and 7-33 times faster than R2010a + DP (plain dark blue line), depending on matrix size. This was explained by the divide and conquer approach preferred for SVD starting from R2010b. Gu et al claimed that this algorithm was 9 times faster on bidiagonal matrices  (86) , in agreement with our observations. The extra gain is due to the four cores simultaneously used on the CPU. Again SP (plain green line) was two times faster than DP. Small matrices, up to 1025 × 1024 = 1. 0e6 points for Core 2 Quad Q8200, and up to 3073 × 3072 = 9. 4e6 points for Core i5 4670K, were even computed faster on CPU with R2014a + SP than on GPU with R2010a + CULA + SP, as indicated by the green arrow.
Unfortunately, CULA free was not compatible with R2014a, but it was nevertheless possible to use GPU thanks to the gpuArray Matlab function. Surprisingly, worst results were obtained, with a GPU time longer than its corresponding CPU time. Moreover R2014a + gpuArray + DP times (dashed yellow line) were shorter or equal to R2014a + gpuArray + SP times (dashed green line), what is in contradiction with DP / SP ratio on GPU (1 / 8 and 1 / 24 for GTX 260 and GTX 660, respectively). This revealed that part of the computation was done in DP, despite SP was called. When checking CPU and GPU activity during SVD, it was observed that GPU was only used at the beginning and at the end of the processing. This denoted that SVD using gpuArray under Matlab R2014a was not an optimised algorithm and that this version should be avoided.
Matlab R2015a In order to check if a new version of Matlab could further improve computation times, we used Matlab R2015a. A slight decrease was observed on CPU from 57. 8 to 52. 5 s and from 29. 1 to 27. 0 s for R2014a + DP (plain yellow line on Figures  4a  and 4b ), R2015a + DP (plain brown line), R2014a + SP (plain green line) and R2015a + SP (plain light blue line), respectively, for a matrix of 4097 × 4096 = 1. 7e7 points. Matlab R2015a was not compatible with GTX 260 GPU, due to its compute capability of 1. 3. A algorithm used and on the relative float performance of CPU and GPU. Low-end GPU are thus not recommended as better results are obtained with CPU. In DP mode, despite the half computing power of a Core i5 4670K against SP, even a GTX 660 never overpassed it (not shown). Similarly to Java, matrix size under Matlab is limited by host and device memory.
Nevertheless, memory consumption under Matlab is improved over Java as no overhead is present during low-rank approximation, pushing away maximum matrix size. Additionally, host memory amount is considerably reduced, down to be almost identical to device memory one. A GPU with 2 GB of memory is thus limited to matrices of 7169 × 7168 = 5. 1e7 points. In this section, we highlighted that the divide and conquer algorithm decrease   and 4b. Matlab R2014a is not recommended neither for GPU. Next step was to focus on the libraries used and their call to hardware instructions, which we explored under Python.
SSE and AVX hardware instructions Despite MKL seemed very promising, it was not the only factor changing in the above experiment as the implemented hardware optimisations changed from SSE2 When moving from no SSE (left column) to SSE3 (middle column), computation times were divided by three with an additional gain for SP. Moreover, when using both MKL and AVX2, a huge performance was obtained, outperforming GPU computation with CULA + SP (hatched red line). As underlined here, the time needed to perform SVD was impressively reduced by a factor of 38 on the same CPU under Python, by optimising the used libraries and their hardware calls. Indeed, decomposition of a matrix of size 1025 × 1024 = 1. 0e6 points was done with SciPy in 7. 6 s without optimisations and in 0. 2 s using MKL library and AVX2 instructions.
Computation times In previous sections, Java, Matlab and Python software were used for their specific testing capabilities. But how do they compare to each other? In order to answer this question, Figure  6  shows SVD computation times for a matrix of 1025 × 1024 = 1. 0e6 Python, both with MKL library and all available SIMD instructions activated 6. For this small matrix of 1025 × 1024 = 1. 0e6 points, the CPU outperformed the GPU, due to data transfer delays limiting GPU efficiency. However, for larger matrices, computation was faster on GPU. Comparing Java CPU and Python with MKL, there was a gain of 100 on the same CPU. This was explained as follows: • a factor of 9 using the divide and conquer algorithm • a factor of 3 using hardware instructions such as SSE3 or AVX2 • a factor of 2 using MKL library • a factor of 2 using single precision instead of double precision.
5e7 complex points in 150 + 32 = 182 s. • Matlab R2018a + gpuArray + SP: 12289 × 12288 = 1. 5e8 complex points in 188 + 4 = 192 s. • Python + CULA + SP: 15219 × 15218 = 2. 3e8 complex points in 574 + 6 = 600 s. Under Python, it was thus possible to apply SVD on the full 87 Sr FID, without any truncation. For comparison, a Nvidia P100 GPU, with 4670 DP GFLOPS and 16 GB of memory, completed the full SVD of a 20000 × 20000 = 4. 0e8 real matrix in 90 s, with a highly optimised CPU-GPU algorithm  (46). This result was really impressive as the authors obtained a faster computation on a much larger matrix with less computing power and double precision. There is thus plenty of place to improve SVD denoising. Directly comparing Java, Matlab and Python was a difficult task as they were not optimised in the same way and it was hard to check what was hidden under the hood.
However, Java GPU was less performant, both in speed and in matrix size. Better results may be obtained with optimised libraries. While Matlab was faster, the memory usage was limiting. Python computation was not as fast but could handle the biggest matrix. This time advantage for Matlab was explained by a better CPU usage during SVD on GPU. However memory was more finely managed under Python. Our results suggested that the key parameter was not the software and the programming language, but rather the used libraries and the calls to hardware instructions. An additional advantage of Python is that it is free of charge and rather easy to program. In order to compute SVD in a minimum amount of time, we recommand to install a Python distribution with MKL library included, such as Anaconda  (92) , and to add the following libraries: SciPy (73), scikit-cuda  (67) , PyCUDA  (68) , and nmrglue  (93)  for NMR data.
In addition, CULA  (60)  and CUDA toolkit (43) packages are necessary. We provide in file svd_auto. py of (64) an optimised SVD function using either the CPU or the GPU, depending on the matrix size. Our tests suggested a minimum value of 4096 columns or rows to switch from the CPU to the GPU. This default value will depend on the hardware used and can be checked by running directly the program. The code is designed to be as simple as possible, with only one necessary parameter, namely the matrix two-dimensional array. Automatic thresholding is applied using Malinowski's significant level indicator  (32). This SVD function is also suitable to be used in PCA and related data mining techniques. In addition, we provide a second program (file denoise_nmr. py of (64)), in charge of importing and exporting Bruker NMR data and to prepare the matrix transferred to SVD program.
Then, we report a detailed analysis of the changes in the CF-mediated dendritic Ca 2ϩ transient produced by the local selective inhibition of various Ca 2ϩ or K ϩ channels. Based on this rich experimental dataset, obtained from an ensemble of 59 cells, we built a NEURON model of a simplified PN dendritic compartment. The model is based on prior work  (Anwar et al. , 2012)  and incorporates modified channel models to accurately reproduce the V m and Ca 2ϩ transients associated with the CF-EPSP at different initial V m. The model included P/Q-type high-voltage activated (HVA) VGCCs, T-type low-voltage activated (LVA) VGCCs, A-type LVA-VGKCs, and Ca 2ϩ -activated K ϩ channels (both SK and BK). The consistency of this model was assessed by quantitatively comparing the experimental results of blocking different individual channels, with the elimination of respective channels from the model.
Procedure of matching experimental transients with a neuron model To deduce the kinetics of each channel involved in the dendritic electrical signal associated with the CF-EPSP, we developed an original approach based on matching optical data with a simplified model, implemented in NEURON  (Hines and Carnevale, 1997)  as depicted in Figure  1B. To build the model, we started from a published model designed to predict activation of Ca 2ϩ -activated K ϩ channels in PN dendrites  (Anwar et al. , 2012) , available in the ModelDB database at https://senselab. med. yale. edu/ModelDB/ ShowModel. cshtml?modelϭ138382&fileϭ/AnwarEtAl2010/cdp5. mod# tabs-1. From this model, consisting of a cylinder of 4 m diameter and 20 m length with standard passive membrane properties, several parameters and mechanisms were initially replaced with those from two earlier models  (De Schutter and Bower, 1994; Solinas et al.
, 2007)  also available in ModelDB database at (https://senselab. med. yale. edu/ModelDB/ ShowModel. cshtml?modelϭ7176#tabs-1) and at (https://senselab. med. yale. edu/ModelDB/ShowModel. cshtml?modelϭ112685&fileϭ/Golgi_cell/ Golgi_SK2. mod#tabs-2). From this starting point, the kinetics of ion channels were changed to obtain traces from simulations that matched experimental V m and Ca 2ϩ optical measurements from ϳ17 ϫ 17 m 2 square regions at three different initial V m : hyperpolarized (ϳϪ80 mV), intermediate (ϳϪ65 mV), and depolarized (ϳϪ50 mV). The specific modifications implemented for each channel are described in detail in the paragraph below. After obtaining a satisfactory set of models for each channel in one cell, the same channel models were used to match our experimental data in three more cells, only by tuning the channel densities and the current input associated with the CF-EPSP.
The model that matches V m and Ca 2ϩ transients is available in the ModelDB database at http:// senselab. med. yale. edu/ModelDB/show Model. cshtml?modelϭ244679. While the channel kinetics were the same in all simulations, the density of P/Q-type and T-type VGCCs, of A-type and HVAK VG-KCs, and of BK and SK Ca 2ϩ -activated K ϩ channels was adjusted to match experimental data in each cell, as shown in Figure  7. To validate the consistency of the four variants of the model, we run computer simulations by modeling the replacement of 1 mM Fura-FF with 2 mM OG5N and analyzed Ca 2ϩ ⌬F/F 0 modifications produced by the elimination of 90% of each channel, in this way mimicking the experiments in which individual channels were pharmacologically blocked. As an example, the results for the model variant of Cell 1 in Figure  7  are reported in Figure  8 , showing that 90% reduction in the density of each channel qualitatively reproduced the experimental behavior observed after toxin or drug inhibition.
Precisely reconstructing dendritic or axonal compartments in other systems is possible, in principle, using the same approach by expanding the pharmacological analysis to all the channels involved in a signal. For this purpose, the database ModelDB for NEURON simulations already includes Ͼ1100 published models covering Ͼ130 research topics  (McDougal et al. , 2017) , which can be used as a starting framework to produce more simplified models with realistic channel kinetics matching the complexity of V m and Ca 2ϩ imaging experiments at high temporal resolution. The application of this novel approach also concerns the study of alterations induced by mutated proteins associated with channelopathies.
Nationale de la Recherche  Grant  ANR-14-CE17-0006   WaveFrontImag ,  Labex Ion Channels Science and Therapeutics Program  Grant  ANR-11-LABX-0015 , and  National Infrastructure France Life Imaging Noeud Grenoblois ; and  Federation pour la recherchesur le Cerveau  Grant Espoir en tete, Rotary France. L. F. was also supported by  European Union   COST Action   CA15124  (NEUBIAS). A. T. and P. P. were supported by  European Research Council   Starting Grant  dEMORY  GA 311435. A. T. was supported by  Google EMEA Scholarship ,  Onassis Foundation MSc Scholarship , and  Einstein Foundation Berlin PhD Fellowship. The authors declare no competing financial interests.