dataesr/scientific-paragraphs-categorization huggingface dataset:
Experiments with Distributed N-LOC on a Single FPGA Experimental setup We tested our distributed N-LOC model by instantiating three N-LOC blocks on a single Zynq-7045 SoC. The controller is implemented in software on a bare-metal ARM Cortex A9 microprocessor. The FPGA and Cortex A9 are linked through an AXI lite bus. Our distributed N-LOC system is composed of 3 × N-LOC instances of 30 place cell neurons each, for a total of 90 neurons. Hence the total number of neurons in these three place cells is 90, which is 10% smaller than the initial single-block N-LOC instance studied in Section 5. As described earlier, we compare our results with a Python program which makes use of NumPy, OpenCV and is compiled with Cython. The experimental conditions are the same as described in Section 5. 5.
Hence, the first winner-takes-all step is overwhelmingly more computationally intensive than the other steps. The total number of operations to process a single image is ≈11. 1 ×10 6 operations or 11. 1 MOPs. The reference code runs on Nvidia Jetson TX2 with a thermal design power (TDP) up to 15 W, which we used as the baseline to compute the performance per Watt of various configurations. As the table shows, there is a 4× to 6× ratio in favour of our N-LOC design.
g. , via its external DDR memory and a DMA, from the processing system part of the Zynq to its programmable logic part. We implemented tests to validate that tile-to-tile data transfers are indeed correct on the Wizarde platform. We specifically targeted communications between the north and north-west tiles. The benchmarks are carried out at 3. 125 Gbps and 6. 25 Gbps, as the maximum admissible frequency for the Aurora 8B/10B IP is 6. 6 Gbps. The Aurora configuration is shown in Table  13  for 6. 6 Gbps. We use the IP in simplex mode. The north tile will be in the transmit mode while the north-west one will be in the receive mode. Our clock reference on Wizarde is set to 125 MHz, to be able to boost the frequency up to 6. 25 Gbps. Each tile has a pair of MGT links connected to its nearest neighbours (e.
Toward a Distributed N-LOC Architecture on Wizarde This section discusses the possibility to implement a distributed N-LOC architecture using a multi-FPGA platform. We will use Wizarde (See Section 2) as our target. Our reasons to resort to Wizarde are threefold: (1) Beyond the intrinsic overhead induced by a distributed architecture and its associated control signals, one of the reasons our first attempt at implementing distributed N-LOC does not perform as well as a single N-LOC block is the very small size of each neural network, which can be alleviated if a sizeable portion of each tile involved in the design can be leveraged (as one tile is roughly able to store 4600 neurons with 100 place cell neurons); (2) the GTX links coupled with the AXI Stream protocol should offer a more asynchronous way of transferring data between the controller (still implemented in software) and its neighbouring tiles, which should reduce communication overheads (for both throughput and latency); and (3) this is the only way we can eventually implement a large neural network-large enough to be useful in a self-driving car.
Changes to the Original Architecture We target four tiles in the Wizarde board: for instance, the central tile, as well as the north, east and west tiles. The latter tiles implement an instance of the N-LOC IP, combined with a GTX interface (see Figure  8 ). The central tile implements the image processing IP (DoG) and orchestrates communications across all tiles via the GTX interface. The communication scheduler is implemented in software on the central tile, using the ARM Cortex A9 processor. More details of N-LOC data exchange of buffer size are given below: In order to grant our localisation system better capabilities in term of accuracy-based wide-range navigation, we need to expand the bio-inspired neural network architecture implemented as hardware-accelerator-based N-LOC. However, the resource consumption, and memory footprint of that purpose, is very costly.
257 W for the LPMP implementation; 2. 741 W for the whole system) compared to the pure software reference implementation running on a high-end embedded system. We demonstrated Wizarde's multi-FPGA capability to implement the whole initial neural network (≈10,000 neurons) over multiple tiles, by leveraging its gigabit transceivers. The software processing part will be deployed on the FPGA center tile to communicate and control all FPGA tiles, by receiving and assessing the localisation score of each captured image from different NLOC accelerator modules. Future work includes implementing LPMP on the Jetson TX2's GPU, as well as modifying our architecture to increase its clock frequency to improve its performance and compare it to GPU-based embedded systems in terms of performance and power consumption.
Acknowledgments. We thank  Andrei Popescu  for helping us with a coinductive proof that arose when removing the ghosts in the Zipperposition loop. We thank  Uwe Waldmann  for sharing with us his encyclopedic knowledge of the various given clause loops and their history. Finally, we thank  Mark Summerfield  and the anonymous reviewers for many helpful suggestions. This research has received funding from the  Netherlands Organization for Scientific Research (NWO)  under the  Vidi program  (project No. 016. Vidi. 189. 037 ,  Lean Forward ).
Specifically, we develop a proof-of-concept automated pipeline that uses randomised perturbations of an atherosclerotic pig coronary artery to produce a large dataset of unique mesh geometries with variable blood flow. A total of 1,407 geometries were generated from seven reference arteries and were used to simulate blood flow using the CFD solver Abaqus. This CFD dataset was then post-processed using the mesh-domain common-base Proper Orthogonal Decomposition (cPOD) method to obtain Eigen functions and principal coefficients, the latter of which is a product of the individual mesh flow solutions with the POD Eigenvectors. Being a data-reduction method, the POD enables the data to be represented using only the ten most significant modes, which captures cumulatively greater than 95% of variance of flow features due to mesh variations.
Next, the node coordinate data of the meshes were embedded in a two-dimensional coordinate system using the t-distributed Stochastic Neighbor Embedding (t-SNE) algorithm. The reduced dataset for t-SNE coordinates and corresponding vector of POD coefficients were then used to train a Random Forest Regressor (RFR) model. The same methodology was applied to both the volumetric pressure solution and the wall shear stress. The predicted pattern of blood pressure, and shear stress in unseen arterial geometries were compared with the ground truth CFD solutions on "unseen" meshes. The new method was able to reliably reproduce the 3D coronary artery haemodynamics in less than 10 s.
Consequently, any change in feature space will necessitate a DNN recalibration cycle. To overcome the above, physics-based machine learning technologies have raised interest recently. These methods are predicated on capturing the underlying physics either via incorporation of the actual conservation laws  (6)  or by datadriven extraction of physically interpretable flow characteristics  (7)  as features for regression. For instance, Reduced-order modelling of CFD simulations are motivated by the presence of coherent structures, identified from their statistical moments in the datasets available from short duration simulations  (8, 9). By applying orthogonal decomposition theory, it is possible to identify high energy Eigenvectors, also known as modes, of these coherent structures using essential information of the flow solution field (e.
g. 3D velocity and pressure) while reducing dimensionality of the data. Initial studies used both temporal and spatial information of the velocity field to reduce its dimensions in non-health related areas  (7). The first health applications used these methods to study coherent structures in the velocity field of idealised phantoms of bifurcations, saccular and aortic aneurysm  (10). Patient-specific applications, which are noisier, have been successfully studied by accounting for such noise in the signal  (11). In order to apply these reduced order flow solution fields to novel objects, an interpolation needs to be carried out. In light of these advances in closely related fields of research, this paper establishes the foundation of our novel method amalgamating these techniques and applies it to a wellcharacterised experimental dataset of atherosclerotic pig coronary arteries  (12).
Creating a well-annotated synthetic data repository Synthetic data has been proposed to meet the huge data requirement of artificial intelligence (AI)  (13). Here, we developed a hybrid technique which uses a combination of realistic and synthetic data. The realistic data was obtained from a validated 3D reconstruction method of coronary arteries based upon a pullback of OCT images and angiography (Figure  2 ). This 3D vessel anatomy was then used as a seed to generate synthetic data by applying random spatial perturbations to the original mesh. To prevent unnatural, discontinuous geometric differences within each mesh phantom, the perturbations are The data processing pipeline is summarized in this flowchart. OCT images are obtained in the cath. lab. and used to extrapolate a 3D contour. Mesh generation and Computational Fluid Dynamics are done through an automatic pipeline.
The velocity profiles obtained from CFD will act as the ground truth. Synthetic data generation (n = 1407) is done by random purturbation of the length-wise diameter of each independent blood vessel (n = 7). Data reduction is performed on the shear stress, and pressure fields obtained from CFD, via POD (see text for details), and on the input meshes through t-SNE (see text for details). These reduced data sets are used to train (using 90% of the data) and validate (using 10% of data) the machine learning learning module FIGURE 2 10 randomly selected phantom geometries from the dataset are visualised. All phantoms shown were generated from the same OCT image. Variation in shape is due to random synthetic perturbations applied to the artery diameter, the function of which is a composite of two sinusoids with randomised amplitude, frequency, phase and vertical displacement.
This ensures smooth, continuous variation along the length of the artery regardless of input parameters. Morgan et al. 10. 3389/fcvm. 2023. 1221541 based on the amplitude of a sinusoid, which distributed the perturbation lengthwise. The sinusoid components have independently randomised amplitude, frequency, phase and vertical offset. With this method, 200 phantom meshes per each of the 7 unique blood vessels available were generated. Including the 7 natural artery shapes, this results in a total of 1407 3D meshes in this preliminary dataset. These geometries were then input to the CFD solver Abaqus (v16. 2) to obtain the pressure and shear stress field by solving the governing steady-state incompressible Navier-Stokes equations. In the solver, the governing equations were discretised on 100,000 mixed hexahedral and triangular prismatic elements in accordance with the second order of approximation.
In our methodology for obtaining common mode functions underlying multiple meshes, the time domain is replaced with the domain of the mesh geometries. It is assumed that a few smoothly varying variables can be used to represent the mesh cases. The goal is to obtain the hidden common modes in the stationary solutions, on multiple meshes, while the mesh is smoothly varied. To obtain the modes underlying the variations in pressure and shear stress fields, we use the method of SVD. We begin with a dataset of CFD simulated steady-state flow solutions. For one simulation, the chosen output variable (e. g. pressure and wall shear) is organised into a N-length vector, where N is the number of nodes in the mesh. These vectors are oriented horizontally and then stacked vertically. With M meshes, the resulting 2D solution matrix A has the dimensions M Â N.
Both vector matrices U and V are organised in terms of the singular values, from most to least significant. The summed energy of each leading mode, being their corresponding singular values, are then used to define a tolerance threshold for information loss. Due to spatial coherence of particular modes of variation of the flow with respect to the mesh shape, the number of modes that capture the majority of useful information are the first few, as compared to the full dataset. Modes that fall outside of a chosen threshold in terms of correlative significance can be truncated from the dataset, drastically reducing the dimensionality of the data whilst incurring a tolerable underestimation of the concerned node-wise flow parameter. Additionally, although not implemented in the current case, explicit smoothing can also be applied in the correlation matrix space to enhance numerical properties of the meh-wise POD coefficients  (8).
A filled feature space with an intact causal link will aid an interpolative machine learning model to make accurate coefficient predictions for an unseen geometry (Figures  5  and 6 ). It is worth noting that what constitutes a "filled" feature space is highly dependent on the chosen t-SNE parameters and the natural limits of the data that is being reduced. The "natural limit" is in reference to the fact that a hypothetical dataset containing all possible natural variations of the artery shape will produce a "filled" feature space, and the regions that are not populated will represent shapes that do not occur naturally, and thus may not be useful for a diagnostic tool. Hence, we aim to produce synthetic data, which is not so different from the natural data as to have its shape fall outside of this hypothetical set.
Random forest regressor and regressor chain SVD re-organizes the modes based on their energy level content and the number of modes are truncated when >95% of the variance of the field is preserved. This resulted in the first 10 modes for the pressure field and the shear stress field for the dataset we use for this study, which when used for reconstructing the solution leads to a root mean squared error less than 5%. In order to interpolate the POD principal coefficient field that enables predictions of future objects, simple feed-forward neural networks and classical machine learning methods were compared. It was found that the RFR algorithm combined with the Regressor Chain algorithm were best suited for this task. The RFR algorithm is a supervised machine learning technique that integrates multiple independent decision trees on a training data set: the obtained results are ensembled to obtain a more robust single model compared to the results of each tree separately  (18).
The 1407 t-SNE data points with their respective pressure and shear stress modes were shuffled and divided into a training data set (80% of the overall data) and a validation data set (remaining 20%). The training dataset was used for ten iterations to train the RFR model, where the best maximum tree depth was found to be 20, and the best maximum number of trees for the model was found to be 70. The machine learning model was applied for the test data set as well. Figures  8  and 9  show the results for shear stress and pressure for the two most significant POD modes respectively. The mean Root Mean Square Error (RMSE) of the prediction of the dominant mode coefficient was 15. 2% for pressure and 19. 7% for shear stress. With the regression for cPOD principal coefficients completed, the mesh-wise modes previously generated by the cPOD method A collection of meshes generated using various OCT images and perturbation parameters, coloured by the pressure (left) and wall shear (right) solutions from CFD simulations.
The mesh dimensions are normalised for the sake of visualisation. (left) Root-mean-squared error for the reconstruction of the original mesh-wise pressure solution from a truncated set of 10 principal coefficients per mesh. The error is normalised against the range of pressure values across all meshes. (right) Singular values for the decomposition of the pressure solution, normalised against the largest value. These singular values are ordered by magnitude and represent the relative contribution of each POD mode to the energy of the overall pressure solution. Subsequent values quickly decay to <1% of the highest value, as the first several modes represent the overwhelming majority of the information in the pressure field. This indicates that many of these trailing modes can be safely discarded from the dataset without losing a significant amount of information.
Morgan et al. 10. 3389/fcvm. 2023. 1221541 FIGURE 5 The mesh-wise reconstruction error for wall shear (left) is much lower than pressure reconstruction using the same number of coefficients. Additionally, the singular values (right) decay to 0 in a fewer number of modes compared to the pressure decomposition. These factors are indicative of the wall shear solution being easier for the POD method to decompose than static pressure, possibly due to the fewer number of CFD nodes for which it is computed. FIGURE 6 The distribution of all meshes in the database embedded in 2D t-SNE space with colours representing the principal coefficients of the static pressure solutions for the first (left) and second (right) mesh wise POD modes. FIGURE 7 The distribution of all meshes in the database embedded in 2D t-SNE space with colours representing the principal coefficients of the wall shear solutions for the first (left) and second (right) mesh wise POD modes.
The errors were calculated using the dominant 10 POD modes for the test dataset of 20% of the meshes in accordance with the following definitions: NMAE(i ¼ 1,. , imeshmax) ¼ P j¼ jnodemax j¼1 jf ML ij À f GT ij j jnodemax Á ( max (f ) À min (f )) Á 100% NRMSE(i ¼ 1,. , imeshmax) ¼ ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi P j¼ jnodemax j¼1 (f ML ij À f GT ij ) 2 q jnodemax Á ( max (f ) À min (f )) Á 100% where jnodemax is the total number of CFD data points in the considered volumetric/surface distributions, imeshmax is the number of meshes in the test dataset, ML and GT denote the machine learning and the ground truth (CFD) solutions respectively, and f stands for the pressure or wall shear stress solution component.
e. the flow in the coronary vessel is quasi-steady. In this case, the time history of inflow velocity variation can be decoupled into a series of time frames, where each frame may be represented by a steady process at a different inlet velocity scale. In turn, the shear stress and pressure fields at each frame can be rapidly reconstructed from the inflow velocity and the shear stress and pressure fields of a baseline dataset using the scaling law introduced by Taylor et al. (26). A more serious limitation of the current study is the neglect of the natural flexibility and heterogeneity of vessel walls in the flow modelling process. Whilst the rigid wall assumption significantly accelerates the solution of the governing Navier-Stokes equations, modelling of the Fluid Structure Interaction (FSI) is essential to correctly capture the coronary artery flow behaviour  (27).
Hence, future developments will incorporate the FSI model into the simulation driven dataset of the suggested cPOD-tSNE framework. Despite the overall salutary results of the RFR method, to further refine accuracy of the machine learning model predictions in future, the RFR algorithm may be replaced by more advanced methods such as those based on Gaussian processes; one advantage of which being uncertainty quantification to provide an overall error estimate for the user. Such estimations would be an invaluable addition to a model that is intended for use as a diagnostic tool for clinicians. Finally, in line with many recent works devoted to the proofof-concept data-driven modelling of cardio-vascular flows (20), we simplified the model by considering the vessel without side branches. However, it is known that bifurcations occur in the main stem of the left coronary artery, which might affect the inflow conditions.
3) To evaluate the efficiency of our approach, we conduct an experiment consisting of classifying network probes. For this purpose, we rely on the traffic captured by network telescopes (darknets). We collect 30, 000 network flows identified by their source IP addresses, and we label them based on reports originating from worldwide sources and aggregated in the AbuseIPDB database  [4]. Then, we evaluate the 3 embedding methods: AWE, GAE and Variational GAE. The evaluation is performed in two steps: first, latent representations of network flows are inferred without using network probes' labels, then, we use a separate single-layer neural network classifier to deduce the classes from the obtained latent representations. Second, we compare these methods with a Graph Convolutional Network (GCN) that is trained in a fully-supervised fashion (using labels).
The obtained embedding representing the input network flow graph is concatenated with additional features that could not be included in the network flow graph. These features may include temporal properties or payload-related characteristics of the input network flow that might have been dismissed while transitioning to a graph structure, or  eventual features related to the downstream task that may reinforce the latent representation. This way, latent representations of network flows are obtained in an unsupervised fashion, independently from any downstream task, making them reusable and processable by classical machine learning algorithms. Finally, we evaluate our approach on a downstream task consisting of classifying probing flows captured by two /20 network telescopes into categories of abuse extracted from reports submitted to AbuseIPDB, a cyber-security incidents database.
VIII. EVALUATION DOWNSTREAM TASK: DARKNET PROBING FLOW CLASSIFICATION To evaluate the performance of the AWE, GAE and VGAE models in inferring efficient latent representations of network flows, we conduct experiments using two separate sources of network flows: a /20 network telescope deployed in French and another one deployed in Japan. The evaluation experiments consist of classifying darknet probing flows based on reports on source IP addresses aggregated by the AbuseIPDB database  [4]. The performances of the AWE, GAE and VGAE models which infer latent representations independently from the downstream task (self-supervised inference) are compared to the performance of a Graph Convolutional Network serving as a baseline and trained using labels (supervised learning). In the following, we provide details about conducted experiments and evaluations.
C. PROBING FLOW LABELING To evaluate the designed embedding models, a reliable and labeled network flow dataset is needed. To build the latter, we collect and extract probing flows using the aforementioned network telescopes. We consider the probing activities recorded during the period from January 2018 to March 2022. We start by grouping the traffic targeting the whole darknet by source IP address. Since a source IP address could have many independent probing activities during the studied period, these probing activities are split into many probing flows as follows. For each IP address, we identify the first and the last packets. Next, we identify time gaps during which no probing activities were performed by the IP address. If an identified period of inactivity exceeds 7 days, then we consider that the probing activity has been terminated at the last packet which marks the start of the identified period of inactivity.