dataesr/scientific-paragraphs-categorization huggingface dataset:
We assume that the shift is a linear function of rotation angle, i. e. v() = v 0 , where is the rotation angle relative to the scan center, and v 0 is the displacement vector in pixels/degree. The algorithm used to find the shift, which has proven to give acceptable results in a considerably fast manner, is the following: (i) Two images are obtained from the sum over the images taken at the motor positions of the first and second half of the dataset, respectively. (ii) The shift vector v 0 between the two resulting images is determined using the scikit-image (van der  Walt et al. , 2014)  function 'registration. phase_cross_correlation'. (iii) The linear shift is in the direction of the normalized shift: v 0 = h v 0 /|v 0 |. (iv) The scaling factor h remains to be determined -it depends on how the intensity of the images varies as function of : v Ã 0 ¼ 2v 0 num images ; " ¼ 3kv Ã 0 k 2 for h Ã 0 to 3" do for i 0 to num images do v Ã ðiÞ ¼ ih Ã v 0 Shift image i using v Ã ðÞ via an affine transformation end for Compute score using normalized variance of the images pixel-wise sum end for h is the h Ã with maximum score: Once v has been determined, the images are shifted with subpixel accuracy using the Discrete Fourier Transform (DFT) algorithm.
Scan analysis After pre-processing, data can be further analyzed according to the type of scan performed during the experiment, see above in Section 2. 1. 3. 3. 1. Rocking curves fitting. As described in Section 2. 1. 1, darfix provides a rocking curve analysis. The rocking curve imaging module provides functions similar to the RCIA code  (Tran Thi et al. , 2017)  within the workflow of darfix. Its main function is to analyze the rocking curve of each pixel by fitting to a peak shape, e. g. a Gaussian. After the fit, maps of the fit parameters (constant background, integrated intensity, peak position and peak width) as a function of pixel position are generated  (Tran Thi et al. , 2017) , as shown in Fig. 4. As a computationally less intensive alternative to fitting, moments of the intensity as a function of the angle can be computed.
, 1985). Depending on the assumptions on the data, different BSS techniques can be used. In DFXM, diffracting elements such as (sub)grains or ferroelastic domains can be interpreted as source signals that contribute to the images. BSS can then be used to identify these elements and extract the corresponding rocking curves, reciprocal space maps, etc. in a given dataset. Specifically, we wish to reconstruct a matrix of observed signals (X) from a linear combination of unknown sources, encoded as rows in a matrix H, so that X ' WH. W is the socalled mixing matrix, and, in our case, we stack the flattened dataset images in rows to form the matrix X. These matrices are obtained by optimizing a distance given by a matrix norm (e. g. the Frobenius norm). The BSS techniques implemented in darfix are: (i) Principal Component Analysis (PCA).
Open source, documentation and tutorials darfix is open source under MIT license. The software can be installed using pip, see https://pypi. org/project/darfix/. User documentation can be found at https://gitlab. esrf. fr/XRD/ darfix/-/blob/main/doc/tutorials/darfix_guide. pdfhttps://gitlab. esrf. fr/XRD/darfix/-/blob/main/doc/tutorials/darfix_guide. pdf.
Data selection and dimension definition The first step of the workflow is to select the input data. Next, the dimensions are defined, i. e. the motor(s) varying during the scan and the number of points along each scan axis is determined (see Fig. 3 ). darfix will attempt an automatic dimension definition. This will result in the automatic identification of the scan motors, the angular ranges, the step size and the number of steps of a scan. In the current version, dimension definition is only available for data in EDF format, where motor positions are automatically extracted from the metadata. Data in HDF5 can be analyzed, but at present the metadata are not used. Afterwards, a region of interest can be defined in order to reduce the data volume and speed up processing.
Blind source separation In Fig. 6  the result of the blind source separation is shown for the rocking curve () obtained for a single crystal of Al in a range of 0. 2 for a (200) reflection. In Fig. 6 (a) the components resulting from the separation are identified by different colors. Fig. 6(b)  follows the same color code to show the rocking curves corresponding to each component. The blind source separation operation can be performed on 2D maps; the output can in that case also be plotted as a reciprocal space map for each component. Once the components are obtained they can be exported as HDF5 files. darfix offers the possibility to link components from two different datasets. This allows tracing features from one layer to the next in multilayer scans or the evolution of features under different external conditions.
Future evolution Due to its modular structure, it is easy to extend darfix with additional functionalities and widgets, for example additional pre-processing tools such as gradient-based threshold removal for the detection of low-intensity features  (Gonzalez et al. , 2020). The functionalities implemented so far are relatively lowlevel. They mostly implement statistical methods and do not include analysis of the diffraction physics. This is an obvious field for future developments. In particular, we envisage implementing the transformation from angle-space to reciprocal-space, including the transformation of peak shifts in mosaicity and strain scans to strain components  (Poulsen et al. , 2021). Furthermore, modules could be developed for the identification of characteristic features in the sample, e.
This is achieved by taking the spatial derivatives of the COM maps, providing information on the dislocation density  (Pantleon, 2008; Simons et al. , 2019). Á = [(Á) 2 + (Á) 2 ] 1/2 Another field for future development is the transformation of a series of 2D scans into a 3D volume model of the sample. At present this is done manually for layer scans  (Yildirim et al. , 2022) , but stacking and registration could be automated. An alternative measurement strategy would be topo-tomo scans  (Ludwig et al. , 2009) , where projections are recorded while the sample is rotated about the scattering vector. Results should be saved in a format compatible with dedicated 3D analysis software such as Paraview  (Ahrens et al. , 2005; Ayachit, 2015). Work for the 3D segmentation of dislocation networks is ongoing  (Huang et al.
Conclusions darfix is a Python package for the analysis of dark-field X-ray microscopy and diffraction topography data. It provides data processing and visualization tools that can be used either as library components or via a graphical user interface as an addon to an Orange workflow. darfix includes analysis functions specific to common scan types used in DFXM, such as rocking curve imaging, mosaicity and strain scans. Through blind source separation, different diffracting elements present on the same map or rocking curve can be identified as distinct sources of diffraction, associating each separated unit with their corresponding rocking curve and reciprocal space map.
3. 3) is imposed. (N)ICA identifies the sources by imposing the constraint that they are statistically independent  (Hyva ¨rinen et al. , 2001). Alternatively, the non-negative matrix factorization (NMF) method, a parts-based representation, imposes the constraint of both the sources and the mixing coefficients (i. e. the contributions of the corresponding source to the final image intensities) to be non-negative. But this method yields nonunique solutions dependent on the initialization of the matrices. The combination of NICA-NMF solves the nonuniqueness issue for the NMF approach. The choice of BSS separation method is left to the user according to the goodness of the source identification for a given dataset.
and it is structured to have a different widget for each step in the data processing workflow. Orange (Dems ˇar & Zupan, 2013) is used to link all these widgets into a single workflow and to pass information between them. Every widget returns a new Dataset object which is the input to the next step of the workflow. Fig. 3(a) shows an example of a typical workflow comprising data selection, different preprocessing steps and shift correction. Figs. 3(b), 3(c) and 3(d) show examples of Grainplot of the intermediate and final results of the workflow (see below for details).
Acknowledgements We acknowledge the  European Synchrotron Radiation Facility  for provision of synchrotron radiation facilities and we would like to thank  Thomas Dufrane  for assistance in using beamline ID06-HXM. We thank  Melanie Gauvin  for providing the sample shown in Fig. 6. Leora Dresselhaus-Marais  is acknowledged for providing the unpublished data shown in Fig. 4 and critical reading of the manuscript. We thank  Thu Nhi Tran Caliste  for stimulating discussions on.
The simulation model is divided into three main modules. The first module is modelling incoming supplies with different types and due dates using MAS. The second module, the crossdocking, is modelled by DES. This module is divided into two parts: the inbound and outbound sides. The inbound side unloads the arrivals of supplies and handles order matching. In case the order is unpaired by the outbound trucks, they are then sent to the warehouse to stay until the next shipment. The shipment of orders follows FCFS (first-come-firstserved) rule. The second part of the cross-docking module, the outbound side, operates the loading process based on order match and load capacity of trucks. The last module, the distributor module, is modelled by MAS. It places various customer demands and manages the loading of the matched orders into its trucks.
Performance evaluation The key performance indicators (KPIs) of the proposed model are defined as follows: • Average stock level: The stock level indicates the number of unloaded pallets, pallets that are currently being moved by the forklifts and the number of pallets in temporary storage. The stock level is the number of pallets inside the cross-dock facility. It is tracked throughout the simulation every hour, and the average level is calculated. • The pallet delay (in minutes) and number of late pallets: The destination of each pallet and each outbound truck are defined by input datasets. The model transfers pallets to the dock door by checking the presence of an outbound truck with the matching destination. If there is no matching outbound truck at the system to load the pallet, the pallet waits in the temporary storage without considering its due date.
Experimentation This section provides the experimentation performed on the proposed model. The model experimentation is carried out on the AnyLogic Simulation Software (University edition 8. 7. 10), using a PC with an Intel(R) Core(TM) i5-8250 U CPU at 1. 60 GHz, and 16. 0 GB of RAM. In this section, we first explain how the input data are generated and the results are evaluated, before giving the experimental results.
Input generation The input datasets are randomly generated with a program written in the C# language. Six traffic level combinations are specified for the experiments with 10 instances generated for each (see Table  4 ). Three different truck-to-door assignment strategies are used in the model to test 60 instances, resulting in a total of 180 experiments. The input generation scheme is explained below and the simulation parameters are delivered in detail in Table  5. • Truck capacity: According to some European shipping companies, a Full Truck Load (FTL) shipment can range between 24 and 33 pallets, but this number can vary depending on the size of the vehicle and type of pallets (e. g. Euro pallets have different dimensions compared to standard pallets used in US)  [40]. On the other hand, according to some Mexican-American-Canadian shipping companies, a full truck contains 26 pallets  [41].
• Outbound truck due date: It is assumed that the outbound truck due date is their arrival time to the system plus three hours. • Number of replications: Following the above-mentioned input data generation scheme, we created an input database composed of 10 instances per traffic level combination (i. e. a total of 60 instances for six traffic levels). For a fair comparison between strategies, each strategy is tested using the same set of 10 instances at each traffic level. Then, we compare the strategies based on the average of 10 instances. This is equivalent to making 10 replications per strategy and per traffic level combination.
The analysis of multidimensional time-varying datasets faces challenges, notably regarding the representation of the data and the visualization of temporal variations. We propose an extension of the well-known Space-Time Cube (STC) visualization technique in order to visualize time-varying 3D spatial data, taking advantage of the interaction capabilities of Virtual Reality (VR). First, we propose the Space-Time Hypercube (STH) as an abstraction for 3D temporal data, extended from the STC concept. Second, through the example of embryo development imaging dataset, we detail the construction and visualization of a STC based on a user-driven projection of the spatial and temporal information. This projection yields a 3D STC visualization, which can also encode additional numerical and categorical data.
, 2019). Methods from the field of Immersive Analytics could notably be beneficial for biological imaging, where large time-varying datasets are more and more present  (Long et al. , 2012) , and constraints emerge from the growing spatio-temporal resolutions and the complexity of the obtained images. For instance, Lattice Light-Sheet Microscopy  (Chen et al. , 2014)  allows the capture of volumes at a resolution of a few hundred nanometers at a few seconds interval. This technology is notably used for recording embryo development  (Guignard et al. , 2017) , creating terabytes of raw data, in contexts of analysis in embryology, morphodynamics, and intracellular dynamics. In such context, extracting surface meshes to describe volumes (see Figure  1 , left) is a commonly used solution in order to display and analyze data, as proposes the web-based application MorphoNet, by  Leggio et al.
(2019). In such complex datasets, identifying the relations between objects, categorical data and their joined evolution over time, becomes a challenge for end users. While, software widely used by biology experts, such as ImageJ  (Rasband, 1997)  or Imaris  (Oxford Instruments plc. , 1993) , can handle the viewing of such data, they lack adequate tools to identify and visualize this type of relations. The main objective of this paper is to propose visualization methods enabling the comparison of 3D spatio-temporal data leveraging the benefits of immersive analytics. For this purpose, we propose an extension of the Space-Time Cube (STC) visualization technique (see  Bach et al. , 2014)  to 4D spatiotemporal data, named hereinafter Space-Time Hypercube (STH). Conceptually, the STH considers that the data to visualize lays in a 4D hypercube with three spatial dimensions and one temporal dimension.
In summary, the main contributions of the paper are: • An extension of the well-known Space-Time Cube technique as an abstraction of 3D temporal data, named Space-Time Hypercube; • A projection operator of the 4D STH to generate a 3D STC that can be visualized in an Immersive Analytics application; • An evaluation of the proposed method on embryology use cases by domain experts. Time-varying data are more and more present in scientific visualization, and numerous methods have been developed to visualize the time component and space-time relationships. Such methods can focus on the visualization of point clouds  (Li et al. , 2013; Lu and Shen, 2008) , trajectory datasets with few  (Amirkhanov et al. , 2019; Andrienko et al. , 2017)  or large amounts of instances  (Neuroth et al. , 2019; Huang et al.
, 2015; Liu et al. , 2016; Hurter et al. , 2009) , simulated flow data  (Waser et al. , 2010; Schindler et al. , 2012)  and surface or volumetric datasets  (Kim et al. , 2017). Immersive visualization methods are also becoming commonplace  (Hurter et al. , 2018; Homps et al. , 2020). However, due to the vast literature in this domain, we constrain the state of the art to visualization methods adapted to datasets representing timevarying 3D spatial data with a particular focus on Space-Time Cube visualizations methods, which is the main scope of the paper. For a comprehensive review of spatio-temporal visualization methods, we refer the reader to the following comprehensive works:  (Aigner et al. , 2011; Andrienko and Andrienko, 2013; Bai et al. , 2020).
Dynamic visualizations Animation is an intuitive method to explore time evolution in 3D temporal datasets. In designs using animation, time increases automatically, showing the evolution of the data through time. The work of  Coffey et al. (2012)  showed that animated and interactive design choices for time exploration can decrease the amount of error in analysis tasks. They also suggested that hybrid methods that would implement interactive, animated and/or static time exploration designs could compensate for the flaws of other methods. For instance,  Akiba et al. (2010)  proposed Aniviz, an interactive animation interface enabling the exploration of timevarying volumetric data by editing a time-dependent transfer function. The user can specify display, animation and data related parameters.
Another example is FluoRender  Wan et al. (2012) , which enabled the visualization of 4D datasets captured using confocal microscopy using a playback mechanism with 4D tracking capabilities. However, animation can be less adequate for comparison tasks  (Kim et al. , 2017)  or the analysis of space-timevalue relationships  (Woodring and Shen, 2006). There is a higher cognitive load for the user who has to remember the state of the data between different moments, which can limit the observation of details, as the slogan "Eyes Beat Memory" suggests. Solteszova et al. (2020)  proposed to use time-warping in animations to cope with this issue. Their method allows the user to manipulate time evolution of a video or a selected point of interest in the video. Yet, other solutions for the issue suggested by the slogan can be found in static visualizations.
Static visualizations In the context of a more recent work by  Johnson et al. (2019)  proposed BentoBox, a Virtual Reality data visualization interface for simulated volumetric and temporal data, which juxtaposes several instances of a volumetric dataset under different parameters, in an array layout. BentoBox disposes of a range of tools focused on the comparison of datasets and manipulation of parameters, using animations, color mapping and 3D bimanual interactions. Finally, a versatile visualization technique which enables the compact visualization and manipulation of time-varying data, is the Space-Time Cube which is detailed in the following section.
We identified two main research topics for extension of STC visualizations. First, high dimensional datasets, with more dimensions than could fit in a 2D temporal STC, could use a similar abstraction and be represented as high dimension volume. Adapted operations on this volume would need to be created to yield visualizable 2D or 3D images. If the idea of a higher dimension STC was evoked in  Bach et al. (2014) ;  Bach et al. (2017a)  reviews, this lead has not been thoroughly explored. Among the works that are the closest to it, the method  Woodring and Shen (2006)  described previously already evoked a time flattening operation on a 3D temporal dataset; in this direction,  Woodring et al. (2003)  proposed a method to generate and render 3D slices based on an arbitrary hyperplane equation.
Non-etheless, other works  (Bach et al. , 2017b)  present methods using immersive technologies to help handle these difficulties. Such method would allow to take advantage of the general overview of the data available with direct 3D rendering. STC visualization can also take advantage to direct interactions in immersive environments. For example, the GeoGate system by  Ssin et al. (2019)  displays geographic trajectories (2D + time) in an augmented reality environment using a STC. A tabletop system displaying satellite information is coupled with the STC visualization to provide additional context information. Furthermore, the user can interact with the data through a tangible ring controller, providing natural interactions. In addition, Wagner  Filho et al. (2019)  also implemented a STC displaying trajectory datasets in immersive environment, with a basic interaction tool set.
They report that it helps in usability and partially addresses the issue of the steep learning curve of the visualization method. This literature review highlighted the lack of methods for the visualization of 3D spatio-temporal data, and especially for surfacemesh based datasets, according to  Kim et al. (2017). The use of the STH, although slightly explored in the literature, showed a high promise in order to produce various meaningful views with reduced dimensionality but with the potential drawback of generating complex and cluttered visualizations. However, the benefits of Virtual Reality systems in terms of depth perception and interaction can overcome such limitations. McIntire et al. (2014)  report in their review that stereoscopic displays are better than desktop displays for tasks of identification and classification of objects in cluttered static environments, overall complex spatial manipulation and spatial understanding.
Space-Time Hypercube visualization This section presents our main visualization method, based on the Space-Time Cube. First, we present a generalization of the STC in 4 dimensions, the STH, then we detail the design choices for the generation and visualization, as well as the interaction methods available. As a support example for this section, we will use a dataset of a time-varying 3D spatial data, a 6-hour-long live recording of an embryo development  Leggio et al. (2019). We detail more thoroughly the context of this dataset in Section 4. In this work, we use 100 of the 180 time points recorded, from a 65 to 383 cells embryo. Figure  1  shows some of the time steps that were used. The dataset is composed of surface meshes, for each cell at each time step. The objects are empty yet stacked very close together, resulting in a dense and cluttered view.
Such visualizations could represent the dimensions differently, providing new points of view and emphasizing different relations between the dimensions, especially time. Although, several constraints appear with this additional dimension. First, while a STC can be directly rendered, a STH cannot be directly rendered as is. To enable a direct visualization, a projection operation on the STH has to be defined in order to yield a 3D visualization. Second, the design of a 4D operation is not straightforward due to the difficulty to approach and visualize a 4D volume. We designed such projection using elementary operators for STCs. Initially, we considered a flattening operation on the spatial dimension, projecting the 3D snapshots of the dataset into 2D planes, in order to reduce the dimension of the STH by one.
With the same analogy that we used to consider our spatio-temporal data as a 4D volume, the plane, as a time-varying object in the 3D space, can be considered as a hyperplane in the 4D space. The operation thus corresponds to a projection such as (x, y, z, t) → (x′, y′, t), with x′, y′ the projected coordinates on the cutting plane. Such projection avoids any spatial distortion that could impair the interpretation of the output shape. Considering the whole dataset as a 4D Space-Time Hypercube, it sums up as a space cutting (see  Bach et al. , 2014 ) by a hyperplane, extracting a 3D spacetime cube. The proposed generation method is data agnostic (either meshbased or volumetric) and its three main steps are illustrated in Figure  2. 1) The user places a clipping plane on the model in order to define the "hyperplane" which will determine the projection.