dataesr/scientific-paragraphs-categorization huggingface dataset:
Our pipeline does not perform a bad-pixel interpolation because both spectral extraction methods, the optimal extraction (see Sect. 4. 2) and the leastsquares extraction approaches (see Sect. 4. 3), make it easy to exclude all bad pixels from the extraction process altogether. This means that interpolated bad-pixel data will not bias uncertainties due to newly introduced correlations. The SPHERE DRH pipeline, however, interpolates bad pixels based on all their neighbors, which is not ideal in the case of microspectra extraction. For this reason, the SPHERE Data Center (DC) reductions  (Delorme et al. 2017) , and the vlt-sphere Python pipeline  (Vigan 2020) , which implements a wrapper for the SPHERE DRH ESO pipeline and adds additional improvements, both implement an approach that linearly interpolates bad pixels using only the nearest good neighbor in the dispersion direction.
Sky frames taken after the coronagraphic exposure also display noticeable persistence, which introduces additional noise. In an effort to overcome this limit, we have constructed a master template for the undispersed background. We seek a background without any dispersed light (e. g. , from the sky), therefore we began with internal instrument background frames that do not contain sky. We took 234 randomly selected background frames taken between 2015 and 2021 and performed PCA on them. The number of frames was limited by the 64 GB of RAM on our computer and the need to perform PCA. These frames come with the instrument in YH-mode. We repeated the exercise with YJ backgrounds and found that YH-derived calibrations fit YJ data just as well as YJ-derived calibrations do. We used about 98% of the pixels to perform PCA, masking the 1% with the highest variance of the background frames and those with less than half or more than twice the average background level.
A different sky exposure may also contain varying levels of persistence, based on how brightly the coronagraphic images were exposed, which adds additional noise. Subtracting the bestfit four component template produces a narrower distribution (thick blue histogram) because a negligible amount of additional noise is added. This distribution is centered on a positive pixel value because it does not subtract the dispersed sky background and contains detector persistence that is intentionally not included in our model. Our SPHERE IFS pipeline implements the template subtraction described here for the undispersed background. This approach minimizes the noise in the resulting data cubes. A dispersed sky background does appear as a constant, uniform background in the extracted data cubes. However, this is typically indistinguishable from the halo of the stellar PSF and is easily removed by postprocessing algorithms.
Lenslet geometry The native lenslet geometry of the SPHERE IFS is hexagonal. This differs from the similar CHARIS and GPI IFSs, which both contain square lenslet arrays. It also differs from the rectilinear geometry that is most natural to a numerical array. For this reason and for compatibility with the many software tools now available to postprocess high-contrast imaging data (e. g. Wang et al. 2015; Cantalloube et al. 2015; Gomez Gonzalez et al. 2016; Galicher et al. 2018; Dahlqvist et al. 2020; Samland et al. 2021) , the SPHERE DRH interpolates data cubes spatially onto a rectilinear grid. The SPHERE IFS pipeline that we present here offers two options. The first is to interpolate the data cubes onto the same rectilinear grid geometry as that used by the SPHERE IFS pipeline. The second option is to preserve the native hexagonal lenslet geometry, which means that each physical hexagonal lenslet corresponds to an individual hexagonal spaxel in the output image.
Summary of changes to the original CHARIS pipeline The adaptation of the CHARIS pipeline B17 to the SPHERE IFS required a number of changes that we summarize here. The pipeline can now extract data cubes from either instrument. The original CHARIS pipeline was written in Python 2. 7; we have modernized the entire code-base to be compatible with 3 More information about the astrometry and optical distortions of the instrument can be found in  Maire et al. (2016). new Python versions. To enable it to handle both SPHERE IFS and CHARIS data, we isolated all instrument-specific and hardcoded variables and moved them to a new Instrument module. Depending on the data provided to the pipeline, the corresponding instrument class is instantiated and changes the behavior of the pipeline according to the instrument and instrument mode of the data.
The pipeline behavior then reflects the lenslet geometry, wavelength range and sampling, instrument-specific header keywords, and properties of the observatory location and average atmospheric transmission. The construction of calibration files using the buildcal script is substantially identical to the original CHARIS pipeline. The appropriate calibration templates are specified by an instance of the respective instrument class. These calibration templates, including oversampled PSFlets, background templates, and both detector and lenslet flat-fields, were derived as described in Sect. 3. The spectral extraction step incorporates the new undispersed background-fitting routine, which currently only works for SPHERE IFS. To account for the hexagonal lenslet geometry, we modified the bad-lenset detection routine, which cosmetically replaces outlier lenslets by its neighbors in the extracted spectral data cube, as there are six equally distant neighbors as opposed to nine unequally distant neighbors in the rectilinear geometry.
Spectrum of 51 Eridani b As a second test of our pipeline, we extracted and postprocessed data cubes of the directly imaged exoplanet 51 Eri b  (Macintosh et al. 2015). We used two coronagraphic datasets of 51 Eri obtained by SPHERE IFS: the data from 2015  (Samland et al. 2017) , and newer data from 2017  (Brown-Sevilla et al. 2022). We extracted data cubes using our pipeline, and for comparison, we used the extracted image cubes provided by the SPHERE Data Center using the ESO DRH pipeline. The DC by default temporally bins the data with a factor that avoids significant smearing in the IFS FoV. In these data sets, this results in 60 s exposures, using a binning of four for 2015 and two in 2017. We then extracted the planetary spectrum using the TRAP  (Samland et al. 2021)  postprocessing algorithm with the same uniform default settings for all data sets.
Usage and performance The adapted CHARIS pipeline is publicly available on Github. It has been modularized such that all instrument-specific calibrations and parameters are moved into a class structure, such that it can easily be maintained and extended for future instruments if necessary. The pipeline functions can either be called from Python or a command-line tool can be used to extract image cubes using a parameter file. The latter was the main way of interfacing with the original CHARIS pipeline. Calling the Python module directly is recommended for constructing an optimized workflow, however. As with the previous CHARIS-pipeline, it only requires two steps to produce results: (1) 'buildcal' generates the calibration files, the wavelength calibration and (oversampled) PSFlet models, from a monochromatic flat-field for a specific night containing the laser spots, and (2) 'extractcube' extracts an image cube from raw image input and the calibration files.
(3) individual_dits (boolean), if set True every individual exposure (detector integration, dit) in the data cube is extracted separately. Otherwise, the cube is averaged and then extracted. We strongly recommend extracting all exposures for obtaining the highest temporal resolution, but this parameter can be set to False for testing purposes in order to speed up the extraction. For SPHERE IFS data, if bgsub is True and no background images are provided, the pipeline uses the new principal component based fitting of the thermal background to perform the background subtraction. When data are directly reduced from Python, the extractcube. getcube-function can be called. We refer to the corresponding documentation for details of the required input parameters. An example script will be provided.
Discussion and conclusion We presented a new open-source data-reduction pipeline for the SPHERE IFS instrument. Our pipeline is a candidate replacement for the current modified DRH pipeline that is the basis of all IFS data reduction. It makes use of new calibration data that were specifically obtained for the purpose of improving the IFS data reduction. The pipeline is straightforward to use for anyone with a laptop and a current Python distribution, but can also make use of heavy multiprocessing if a large amount of data is processed. We plan to include our pipeline into the SPHERE Data Center (DC) in the future to facilitate access for the community even more. Because of the significant improvement in data quality provided by our pipeline over the current version of the SPHERE-DC pipeline, the SPHERE-DC  (Delorme et al.
2017)  has started adapting our pipeline to its infrastructure. It will then gradually reprocess and publicly release all available public IFS data over the next few years. Our pipeline was also recently used in  Franson et al. (2022)  as the primary reduction for a newly discovered companion. The public DRH has several important and known limitations that prevent its use for the most advanced science, such as limited accuracy in the wavelength calibration (up to 20 nm discrepancy), a grid pattern that is especially visible in low-S/N channels, and the complicated structure of the reduction recipes, which requires expert knowledge. A user wishing to postprocess SPHERE data must therefore either contact the Data Center or use a Python wrapper for the DRH pipeline containing known fixes for some of the issues  (Vigan 2020) , such as correcting the wavelength calibration based on measurements of the position of the satellite spots.
Acknowledgements. M. S. acknowledges support from the  European Research Council  under the  Horizon 2020 Framework Program  via the  ERC Advanced Grant  Origins  83 24 28. This work has made use of the SPHERE Data Centre, jointly operated by  OSUG/IPAG (Grenoble), PYTHEAS/LAM/CESAM (Marseille) ,  OCA/Lagrange (Nice) ,  Observatoire de Paris/LESIA (Paris) , and  Observatoire de Lyon. This work is supported by the  French National Research Agency  in the framework of the  Investissements d'Avenir program  ( ANR-15-IDEX-02 ), through the funding of the " Origin of Life " project of the  Grenoble-Alpes University. This project has received funding from the  European Research Council (ERC)  under the European Union's  Horizon 2020 research and innovation programme (COBREX ; grant agreement n • 885593).
Indeed, this survey was conducted among 95 wireless researchers, and the majority stated that O-RAN will be the foundation of future cellular networks. Then, the authors described the benefits of O-RAN in addition to its current shortcomings and research opportunities. On the other hand, the general architecture, concepts, and requirement of O-RAN first introduced in  [13]. Then, the authors designed an intelligent scheme of radio resource allocation to deal with traffic congestion and show its efficiency by leveraging a real-world dataset. The work concludes with still opened challenges and future research directions. Similarly, the authors provided an overview on O-RAN architecture and its main modules in  [14]. The authors also gave realistic RAN scenarios leveraging AI/ML-based models, on top of the O-RAN architecture, highlighting their disrupting potential.
This framework relies on key emergent technologies, such as AI/ML-driven models, MEC, and disaggregated RAN functions, to optimize network management. It also enables efficient RAN sharing in terms of required resource and service orchestration, which are aligned with the O-RAN architecture. In  [23] ,  [24] , the authors discussed the implementation of ML-based closedloop solutions on top of the O-RAN architecture. They also provided a first demonstration of O-RAN through an experimental testbed. Thus, they deployed O-RAN using Colosseum network emulator. Then, they used the deployed O-RAN to manage multiple network slices. Finally, the authors introduced a ML workflow-based on Working Group (WG) 2 ML specifications of the O-RAN alliance, in  [25]. They then implemented this workflow using the open-source software of O-RAN.
1) THE NON REAL-TIME RAN INTELLIGENT CONTROLLER The non Real-Time (RT) RIC is a logical function implemented at the Service Management and Orchestration level (SMO). It is composed of two main sub-functions: Non-RT RIC framework and Non-RT RIC applications (rApps). The framework is an internal functionality of SMO that provides the needed services to rApps through R1 interface, while the non-RT RIC applications (rApps) leverage the SMO services, such as data monitoring over the O1 interface (stored in a local database) and provisioning services, in order to support intelligent optimization of RAN elements and resources in a non real-time scale (i. e. , greater than one second). Thus, Non-RT RIC aims to provide an intelligent RAN policy to near Real-time intelligent controller, through the A1 interface, based mainly on AI/DL training/inference and data analytics.
But, data are processed close to the users (MEC), when it comes to vRAN and O-RAN. [44]. Thus, decreasing the consumed energy by base stations significantly impacts the total energy consumption of the RAN part, which will also enable to decrease the energy consumption of all ICT sectors and in particular the cellular networks  [45]. In fact, centralizing network functions causes the energy consumption to decrease and virtualizing the network functions results in reducing further the energy consumption  [44]. Therefore, the energy consumption of C-RAN is medium, while the vRAN and O-RAN are low as compared with that of C-RAN. To do so, we have chosen to group existing works based on the related Near-RT RIC module to which they belong. This includes the three Near RT RIC modules (cf. Fig. 5 ).
8 ), according to the targeted challenge (radio resources management, spectrum management, mobility management, etc. ). Thus, predictions will be performed in near real-time based on monitored data from the O-RU level via the O1 interfaces. However, when decisions must be taken to adjust xApp's parameters, for instance, updating the policy of radio resource allocation or that of the scheduler, the Near RT-RIC sends the DL-based decisions to the corresponding O-RAN module (O-DU or O-CU), through the E2 interface (Step 4 in Fig. 8 ). Besides, data are monitored continuously from the O-RU part and stored in the database through the O1 interface. Hence, to build supervised models in a centralized way, the Non RT-RIC generates directly the needed model by leveraging the monitored and stored data in the database, before deploying them (learning models) in the related Near RT-RIC's xApps.
B. NETWORK SLICING INTEGRATION CONCERNS The O-RAN architecture is expected to support network slicing (NS), which will enable the creation of multiple network slices tailored to fulfill diverse requirements. Thus, integrating NS may impact O-RAN in different manners. The O-RAN orchestrator (SMO) must be configured to consider the network slice template. Then, the performed predictions by AI/DL models, at near-RT RIC, must be compared to the slices' requirements in order to anticipate the slices' SLA violations. Moreover, to ensure network slice isolation, a secure of the Non-RT RIC and Near-RT RIC databases must be dedicated to each network slice. This enables to build either AI/DL models proper to each network slice or a global model aggregating network slices models, using, for instance, distributed federated learning  [90].
On the other hand, ETSI Multi-access Edge Computing (MEC) consists of deploying computation and storage capability close to UEs, and thus reducing the network latency  [93]. To do so, it (MEC) exploits mainly the RAN contextual information to enable time-sensitive and traffic redirection applications, provide service-oriented APIs such as radio conditions and users location contexts, etc. Hence, there is a great need for efficient MEC and O-RAN integration in order to enable MEC-related management. It is worth noting that MEC was defined for 4G networks; however, its integration with the 5G network is still in progress. In fact, since MEC aims to provide a low latency network, MEC hosts can act at the near-RT RIC of O-RAN. In addition, the O-RAN databases can integrate the MEC databases storing, Radio Network Information Service (RNIS), cell performance, users locations, etc  [94].
F. ENERGY CONCERNS WITH FUNCTION SPLITTING OF O-RAN With the growing impetus of O-RAN architecture, ensuring an efficient function splitting in O-RAN, while reducing the energy consumption of the RAN hardware and software is becoming highly important. In fact, energy efficiency plays a vital role in decision-making process of cellular networks, due to the considerable increase in their cost and carbon footprint caused by the high demand for data and network densification. In such a context, leveraging renewable energy sources at the RAN level is a promising approach to optimize the energy consumption of RAN hardware and software. However, network managers need to optimize energy usage since they must store this energy in limited batteries in terms of capacities. In addition, renewable energy is intermittent, and the supply is not always guaranteed  [97].
B. ELECTRONICS FOR STIMULATION A complete stimulation system (Fig. 3 ) was designed to combine in vitro stimulation and bioimpedance measurement. This system includes a programmable arbitrary waveform generator (Agilent 33120A), used as a stimulator triggering physiological contractions. A custom Python software was written to configure stimulation patterns on the programmable waveform generator. These patterns are composed of a series of asymmetric biphasic pulses to ensure charge-balancing and safe stimulation  [33]. As illustrated in Fig. 3 (b), individual pulses have adjustable amplitude Ip (from 0. 5 mA to 55 mA, 0. 5 mA step), pulse frequency f (from 0. 1 Hz to 100 Hz, 0. 1 Hz step), and pulse duration Tp (from 1 ms to 1/3f, 1 ms step). Each positive pulse delivers a charge quantity equal to Ip × Tp.
D. MONITORING AND ANALYSIS OF MYOTUBES CONTRACTION Our setup allows visual monitoring of the cells out of the incubator. To evaluate the contractile activity of myotubes, we used a camera (SCOP-CAM4K, ScopPro) mounted on an inverted microscope. During the stimulation sessions, we recorded ~10-second videos of each well every hour, at a frame rate of 60 frames per second and at a resolution of 3840×2160 (~8 MPixels). Image analysis was performed using an ad hoc Python software and the OpenCV library. The program performs a grayscale conversion (256 values; white: 255, black: 0) of video frames, then computes the absolute difference between a reference image (first image, without contraction) and the next frames: img = | img -img | ( 1 ) where imgi is the i-th image and imgref the reference image.
return TRUE 13: end procedure an AP that we denote by a, given a set of possible client devices and a "channel environment", the second auxiliary procedure (Algorithm 2) aims at checking if AP a may be turned-on without impacting the existing connections already set on this channel, and then compute a set of client devices (a cluster) in a greedy approach, knowing that the connection possibilities also depend on the "channel environment". This procedure returns the "score" of this cluster, i. e. its contribution to the objective function. Thanks to the ScoreCluster procedure, the third auxiliary procedure (Algorithm 3) tests all the possible new clusters to add on a current channel and select the "best" one, in a greedy approach too. With this auxiliary procedures, we can now introduce the pseudo-code of the greedy heuristic (Algorithm 4).
Implementation The numerical experiments presented here were ran on a computer with following characteristics: Set of parameters θ ρ Set 0 0. 0001 100 Set 1 0. 0001 10 Set 2 0. 001 100 • CentOS Linux 7 operating system; • 32 processors Intel(R) Xeon(R) CPU E5-2620 v4 @ 2. 10GHz; • 64 GB of RAM. The LU factorization is computed with a variant of the Gilbert and Peierls algorithm for sparse gaussian elimination, implemented in the SuperLU library  [63]. Based on this LU factorization, the linear systems are solved with the same library. This library is accessed through the Scipy wrapper  [71]  for Python.
The idea is to infer multiple multivariate relationships between co-occurring time-series by assuming that the temporal data depends not only on inner variables and intra-temporal relationships (i. e. , observations from itself) but also on outer variables and inter-temporal relationships (i. e. , observations from other-selves). An extensive set of experiments was conducted comparing REGENN with tens of ensemble methods and classical statistical ones. The results outperformed both statistical and ensemble-learning approaches, showing an improvement of 64. 87% over the competing algorithms on the SARS-CoV-2 dataset of the renowned John Hopkins University for 188 countries simultaneously. For further validation, we tested our architecture in two other public datasets of different domains, the PhysioNet Computing in Cardiology Challenge 2012 and Brazilian Weather datasets.
The GSE stands for a graph-based learning-representation layer that enhances the encoding and decoding processes by learning a shared graph across different time-series and timestamps. The results we present are based on an extensive set of experiments, in which REGENN surpassed a set of 49 competing algorithms from the fields of deep learning, machine learning, and time-series; among of which are singletarget, multi-output, and multi-task regression algorithms in addition to univariate and multivariate time-series forecasting algorithms. Aside from surpassing the state-of-the-art, REGENN remained effective after three rounds of 30 ablation tests through distinct hyperparameters. All experiments were carried out over the SARS-CoV-2, Brazilian Weather, and PhysioNet datasets. In the task of epidemiology modeling on the SARS-CoV-2 dataset, we had improvements of at least 64.
87%. We outperformed the task of climate forecasting on the Brazilian Weather dataset by at least 11. 96% and patient monitoring on intensive care units on the PhysioNet dataset by 7. 33%. Furthermore, we analyzed the results using the Evolution Weights from the GSE layers, which are the intermediate hidden adjacency matrices that arise from the graph-evolution process after going through the cosine similarity activation, showing that graphs shed new light on the understanding of non-linear black-box models. Since multiple multivariate time-series is an ascending research topic, we understand REGENN has implications in multiple domains, like economics, social sciences, and biology, in which different time-series share the same timestream and co-occur in time, mutually influencing one another.
Graph Soft Evolution Graph Soft Evolution (GSE) stands for a representationlearning layer that, given a training dataset, builds a graph in the form of an adjacency matrix, as in Fig. 2. The GSE layer receives no graph as input but a set of multiple multivariate time-series. The graph is built by tracking pairs of co-occurring variables, one sample at a time, and merging the results into a single co-occurrence graph shared among samples and timestamps. We define co-occurring variables as two variables, from a multivariate time-series, with a nonzero value in the same timestamp -in that case, we say one variable influences another and is influenced back. The cooccurrence graph is the projection of a tridimensional tensor, T, T ∈ R s×w×v , into a bidimensional one, A, A ∈ R v×v , describing variables' pair-wise time-invariant relationships.