Miracle-dz/UniFAR-FTUs huggingface dataset:
A Robust Approach to Fine-tune Pre-trained Transformer-based models for Text Summarization through Latent Space Compression

We proposed a technique to reduce the decoder’s number of parameters in a sequence-to-sequence (seq2seq) architecture for automatic text summarization. This approach uses a pre-trained Autoencoder (AE) trained on top of an encoder’s output to reduce its embedding dimension, which significantly reduces the summarizer model’s decoder size. Two experiments were performed to validate the idea: a custom seq2seq architecture with various pre-trained encoders and incorporating the approach in an encoder-decoder model (BART) for text summarization. Both studies showed promising results in terms of ROUGE score. However, the impressive outcome is the 54% decrease in the inference time and a 57% drop in GPU memory usage while fine-tuning with minimal quality loss (4.5% R1 score). It significantly reduces the hardware requirement to fine-tune large-scale pre-trained models. It is also shown that our approach can be combined with other network size reduction techniques (e.g. Distillation) to further reduce any encoder-decoder model parameters count. The implementation and checkpoints are available on GitHub.1
Prediction of Heart Attacks using Data Mining Techniques

Data mining tools have proven to be highly effective in recognizing and predicting early signs of heart disease, especially in countries where doctors lack extensive cardiovascular knowledge. This study introduces a new comprehensive combined dataset consisting of Cleveland, Hungarian, Swiss, Long Beach VA, and Stalog (Heart) datasets. The combined dataset contains 11 unique attributes and 918 observations. A total of eight different algorithms were used to analyze this newly combined dataset, including Logistic Regression (LR), Support Vector Machine (SVM), Decision Tree (DT), Random Forest (RF), K-Nearest Neighbor, Gradient Boosting, AdaBoosting, and XGBoosting. Using the Cleveland data set alone, the best prediction result was obtained by the LR algorithm with an accuracy of 82.6%, while the RF algorithm provided the best prediction accuracy of 89.9% using the combined dataset. In addition, we have identified the five most important features of the combined dataset using a deep exploratory data analysis. As a result of refining our dataset to include only the most relevant features, the XGB algorithm provided the highest prediction accuracy of 89.1%. Even though this decreases accuracy by 0.8%, it reduces time and money spent on non-essential tests.
Metallic abundances of the 2002 Leonid meteor deduced from high-definition TV spectra

High-definition TV spectra in the ultraviolet-visible region were obtained during the 2002 Leonid aircraft campaign. We analyze the spectra of the brightest fireball that appeared at 03 h 47 m 54 s UT on Nov. 19, 2002 and identify the neutral atoms, mainly MgI, FeI, CaI, and NaI in the observed wavelengths between 300-650 nm. The singly ionized atomic emissions, CaII and MgII lines, also appeared in the spectrum in several epochs during the series of video frames. From analysis of the spectra, time variation in the abundances of metallic atoms, along with their electronic excitation and blackbody temperatures, were obtained assuming the Local Thermal Equilibrium (LTE) condition. Both Fe and Ca abundances relative to Mg are lower than the solar abundance, while Na is slightly higher. We found correlation between the excitation temperature and the abundance of Ca, which suggests incomplete evaporation of the Ca due to intrinsic refractoriness. A search for bands of CHON-related molecules, such as OH and CN, is not successful in the brightest fireball in this study.
Approximate Orthogonal Spectral Autoencoders for Community Analysis in Social Networks

Discovery and partitioning of complex graphs such as social networks into communities can be useful for analyzing behavior of individuals in the network. Spectral clustering is one useful tool for performing this clustering, but it suffers from scalability and generalizability to new data. In this paper we introduce an approximate orthogonal spectral autoencoder. We apply this model to a political campaign contribution social network to show its effectiveness on out-of-sample embedding for clustering and classification. The resulting embedding is used with hierarchical fuzzy spectral clustering to show embedding generalization for a behavioral prediction problem for nodes in the social network.
A 28nm 2D/3D Unified Sparse Convolution Accelerator with Block-Wise Neighbor Searcher for Large-Scaled Voxel-Based Point Cloud Network

3D processing plays an important role in many emerging applications such as autonomous driving, visual navigation and virtual reality. Recent research shows that adopting 3D voxel-based sparse convolution (SCONV) as a backbone can achieve better performance than a point-based network in large-scale outdoor scenarios [1]. Moreover, 2D SCONVs are still necessary for Bird's-Eye-View (BEV) neck layers or fusion with image processing. Hardware acceleration is needed for multiple key operations, including 3D submanifold SCONV (S-SCONV), 3D non-submanifold SCONV (N-SCONV) and 2D SCONV. Recently, several processors have been developed for point-based networks [2] or SCONV [3]–[5]. However, for large-scale voxel-based sparse networks, three key challenges have not been fully addressed thereby limiting practical application, as shown in Fig. 22.2. 1. 1) Frequent and random external memory accesses (EMA) arise from gather and scatter operations for sparse-neighbor search (SNS), causing high search overhead and irregular EMA with low efficiency. 2) Low throughput between SNS and CNN, reducing the computing performance (e.g. >50% SNS time consumed by the 3D SECOND [6] network in a GPU). 3) Unbalanced workload and limited data reuse for multiple cores, which results in low core utilization and duplicated data loads without specific optimizations.
Gorenstein injective and projective complexes

In this article we extend the notion of Gorenstein injective and projective modules to that of complexes and characterize such complexes. We prove that over an n-Gorenstein ring every complex has a Gorenstein injective envelope and we show that every such envelope is a quasi-isomorphim. When the ring is commutative, local and Gorenstein, Auslander announced that every finitely generated R-module has a finitely generated Gorenstein projective cover. We show that every bounded above complex having all terms finitely generated over such a ring has a Gorenstein projective cover and we show that these covers are quasi-isomorphisms.
Enhanced-feature pyramid network for semantic segmentation

Semantic segmentation is a complicated topic when they require strictly the object boundary accuracy. For autonomous driving applications, they have to face a long range of objective sizes in the street scenes, so a single field of views is not suitable to extract input features. Feature pyramid network (FPN) is an effective method for computer vision tasks such as object detection and semantic segmentation. The architecture of this approach composes of a bottom-up pathway and a top-down pathway. Based on the structure, we can obtain rich spatial information from the largest layer and extract rich segmentation information from lower-scale features. The traditional FPN efficiently captures different objective sizes by using multiple receptive fields and then predicts the outputs from the concatenated features. The final feature combination is not optimistic when they burden the hardware with huge computation and reduce the semantic information. In this paper, we propose multiple predictions for semantic segmentation. Instead of combining four-feature scales together, the proposed method processes separately three lower scales as the contextual contributor and the largest features as the coarser-information branch. Each contextual feature is concatenated with the coarse branch to generate an individual prediction. By deploying this architecture, a single prediction effectively segments specific objective sizes. Finally, score maps are fused together in order to gather the prominent weights from the different predictions. A series of experiments is implemented to validate the efficiency on various open data sets. We have achieved good results 76.4% $m$IoU at 52 FPS on Cityscapes and 43.6% $m$IoU on Mapillary Vistas.
Solar Neutrino Emission Deduced from a Seismic Model

Three helioseismic instruments on the Solar and Heliospheric Observatory have observed the Sun almost continuously since early 1996. This has led to detailed study of the biases induced by the instruments that measure intensity or Doppler velocity variation. Photospheric turbulence hardly influences the tiny signature of conditions in the energy-generating core in the low-order modes, which are therefore very informative. We use sound-speed and density profiles inferred from GOLF and MDI data including these modes, together with recent improvements to stellar model computations, to build a spherically symmetric seismically adjusted model in agreement with the observations. The model is in hydrostatic and thermal balance and produces the present observed luminosity. In constructing the model, we adopt the best physics available, although we adjust some fundamental ingredients, well within the commonly estimated errors, such as the p-p reaction rate (+1%) and the heavy-element abundance (+3.5%); we also examine the sensitivity of the density profile to the nuclear reaction rates. Then, we deduce the corresponding emitted neutrino fluxes and consequently demonstrate that it is unlikely that the deficit of the neutrino fluxes measured on Earth can be explained by a spherically symmetric classical model without neutrino flavor transitions. Finally, we discuss the limitations of our results and future developments.
EEG-based Emotion Recognition via Transformer Neural Architecture Search

Emotion recognition based on electroencephalogram (EEG) plays an increasingly important role in the field of brain–computer interfaces. Recently, deep learning has been widely applied to EEG decoding owning to its excellent capabilities in automatic feature extraction. Transformer holds great superiority in processing time-series signals due to its long-term dependencies extraction ability. However, most existing transformer architectures are designed manually by human experts, which is a time-consuming and resource-intensive process. In this article, we propose an automatic transformer neural architectures search (TNAS) framework based on multiobjective evolution algorithm (MOEA) for the EEG-based emotion recognition. The proposed TNAS conducts the MOEA strategy that considers both accuracy and model size to discover the optimal model from well-trained supernet for the emotion recognition. We conducted extensive experiments to evaluate the performance of the proposed TNAS on the DEAP and DREAMER datasets. The experimental results showed that the proposed TNAS outperforms the state-of-the-art methods.
Speculative Register Reclamation

Large number of in-flight instructions were envisioned two decades ago. They are finally happening now. While more in-flight instructions enable higher ILP and therefore better single-thread performance, it comes at a price. The price is larger structures within the core such as the physical register file. In this work, we propose to reduce register file size while maintaining (or even increasing) number of in-flight instructions. We leverage the insight that within loops, where most time is spent in general, most logical registers are redefined in the same or immediate next iteration. The physical registers allocated to most of these logical registers can thus be aggressively and speculatively released at redefinition instead of being released when the redefining instruction finally commits as in current designs. For correct mis-speculation recovery, only registers that are actually used (i.e used without prior redefinition) across iterations need to remain allocated beyond redefinition, leading to much reduced register file pressure. We show that using our design, register file sizes can be reduced by 50% while still achieving a 1.05x performance improvement over existing designs on a variety of applications even when other core resources are kept the same. The power consumption among various core structures in reduced by 26% on average. In addition, the performance improvement jumps to 1.14x when this reduction in register file size is complemented with an increase of other structures within the core.
DeepWafer: A Generative Wafermap Model with Deep Adversarial Networks

A certain amount of process deviations characterizes semiconductor manufacturing processes. Automated detection of these production issues followed by an automated root cause analysis has the potential to increase the effectiveness of semiconductor production. Manufacturing defects exhibit typical patterns in measured wafer test data, e.g., rings, spots, repetitive textures, or scratches. Recognizing these patterns is an essential step for finding the root cause of production issues. This paper demonstrates that combining Information Maximizing Generative Adversarial Network (InfoGAN) and Wasserstein GAN (WGAN) with a new loss function is suitable for extracting the most characteristic features from extensive real-world sensory wafer test data, which in various aspects outperforms traditional unsupervised techniques. These features are then used in subsequent clustering tasks to group wafers into clusters according to their exhibit patterns. The primary outcome of this work is a statistical generative model for recognizing spatial wafermaps patterns using deep adversarial neural networks. We experimentally evaluate the performance of the proposed approach over a real dataset.
Variational Inference via Rényi Upper-Lower Bound Optimization

Variational inference provides a way to approximate probability densities. It does so by optimizing an upper or a lower bound on the likelihood of the observed data (the evidence). The classic variational inference approach suggests to maximize the Evidence Lower BOund (ELBO). Recent proposals suggest to optimize the variational Rényi bound (VR) and χ upper bound. However, these estimates are either biased or difficult to approximate, due to a high variance.In this paper we introduce a new upper bound (termed VRLU) which is based on the existing variational Rényi bound. In contrast to the existing VR bound, the Monte Carlo (MC) approximation of the VRLU bound is unbiased. Furthermore, we devise a (sandwiched) upper-lower bound variational inference method (termed VRS) to jointly optimize the upper and lower bounds. We present a set of experiments, designed to evaluate the new VRLU bound, and to compare the VRS method with the classic VAE and the VR methods over a set of digit recognition tasks. The experiments and results demonstrate the VRLU bound advantage, and the wide applicability of the VRS method.
Learning to Process Topic Aware Queries on Geo-Textual Streaming Data

In the past few years, location-aware systems have become an integral and important part of modern applications, including social media applications that have seen an unprecedented rise in popularity. Fast-paced, highly dynamic streams of geo-textual data are generated from such applications consisting of millions of users who demand fast query processing and performance. During a search in location-based systems, topic-aware search allows retrieving relevant data based on a specified ‘topic’ of interest. This paper presents efficient and scalable techniques to process topic-aware spatial keyword search on streaming data. Modern applications require quick system responses for topic-aware queries with spatial and keyword filters. Processing such large amounts of geo-tagged stream data is time-consuming and inefficient with current methods for topic-based search. To solve this problem, we build techniques for indexing and processing by introducing an index called Topic Aware Grid (TAG). TAG integrates spatial and topic information using a variation of the Grid index where cells of the index are attached to local topic models. We perform extensive experiments on real datasets with real and synthetic query workloads that verify the increased performance of our work over alternate techniques for geo-textual streams.
Estimation of the Length Distribution of Marine Populations in the Gaussian-multinomial Setting using the Method of Moments

Abstract In this paper, the problem of estimation of the length distribution of marine populations in the Gaussian-multinomial model is considered. For the purpose of the mean and covariance parameter estimation, the method of moments estimators are developed. That is, minimum variance linear unbiased estimator for the mean frequency vector is derived and a consistent estimator for the covariance matrix of the length observations is presented. The usefulness of the proposed estimators is illustrated with an analysis of real cod length measurement data.
Performance of the Navy’s Tropical Cyclone Prediction Model in the Western North Pacific Basin during 1996

Abstract The U.S. Navy’s operational implementation of the hurricane prediction system developed at the National Oceanographic and Atmospheric Administration’s Geophysical Fluid Dynamics Laboratory is described, and the performance of the model during the 1996 western North Pacific tropical cyclone season is analyzed. The model was highly reliable in terms of maintaining and tracking tropical cyclones, maintaining 96%, 93%, and 93% of all tropical storms and typhoons at 24-, 48-, and 72-h forecast periods. Subsequent model improvements raised these percentages to 96%, 93%, and 100%. Overall track errors were 176, 316, and 466 km for the same periods. Errors for tropical storms and typhoons were 75–150 km smaller than those for tropical depressions. The difference generally grew with forecast length. Large track errors were generally associated with a sheared environment, spurious interactions with elevated terrain, or poorly timed recurvature. On average, the model slightly underforecast intensity, but in.
Determining Association between Fatal Heart Failure and Chronic Kidney Disease: A Machine Learning Approach

Cardiorenal syndrome is a term that refers to a spectrum of heart and kidney disorders that demonstrate how a condition affecting one of the organs impairs the other. In this work, the association between two of the most persistent conditions: chronic kidney disease (CKD) and fatal heart failure (HF), was investigated using machine learning approaches. The research visualizes dependencies and identifies patterns in a subtype of cardiorenal syndrome with the primary goal of determining the risk of fatal heart failure in individuals with chronic kidney disease using state-of-the-art techniques. Firstly, heart failure and chronic kidney disease datasets were used for disease prediction with five classifiers: Random Forest (RF), XGBoost, CatBoost, Logistic Regression, and Support Vector Machine. The prediction accuracy for heart failure was between 70%-76%, and CKD was between 97%-99%. The top predicting models were random forest, XGBoost, and CatBoost classifiers. In the second stage, the feature importance scores of the best predictors were analyzed to gauge the relationship between the conditions. Numerous features of HF and CKD that were common and obtained high importance scores for the top classifiers were age, serum creatinine, serum sodium, and diabetes mellitus. Finally, a variety of visualization techniques were employed to acquire insight into the relevance of different features, resulting in medically sound findings. The analysis of the physiological attributes and their importance with the help of machine learning was aided in successfully reaffirming the medical findings of a crucial subtype of cardiorenal syndrome, associating fatal heart failure with chronic kidney disease.
Learning Task-independent Joint Control for Robotic Manipulators with Reinforcement Learning and Curriculum Learning

We present a deep reinforcement learning-based approach to control robotic manipulators and construct task-independent trajectories for point-to-point motions. The research objective in this work is to learn control in the joint action space, which can be generalized to various industrial manipulators. The approach necessitates that the neural network learns a mapping from joint movements to the reward landscape determined by the distance to the goal and nearby obstacles. In addition, curriculum learning is embedded in this approach to facilitate learning by reducing the complexity of the environment. Conducted experiments demonstrate how the reinforcement learning-based approach can be applied to three different industrial manipulators in simulation with minimal configuration changes. The results of our contribution demonstrate that a model can be trained in a simulation environment, transferred to the real world, and used in complex environments. Furthermore, the Sim2Real transfer, augmented by curriculum learning, highlights that the robots behave in the same way in the real world as in the simulation and that the operations in the real world are safe from a control and trajectory point-of-view.
Compression-Aware and Performance-Efficient Insertion Policies for Long-Lasting Hybrid LLCs

Emerging non-volatile memory (NVM) technologies can potentially replace large SRAM memories such as the last-level cache (LLC). However, despite recent advances, NVMs suffer from higher write latency and limited write endurance. Recently, NVM-SRAM hybrid LLCs are proposed to combine the best of both worlds. Several policies have been proposed to improve the performance and lifetime of hybrid LLCs by intelligently steering the incoming LLC blocks into either the SRAM or NVM part, regarding the cache behavior of the LLC blocks and the SRAM/NVM device properties. However, these policies neither consider compressing the contents of the cache block nor using partially worn-out NVM cache blocks.This paper proposes new insertion policies for byte-level fault-tolerant hybrid LLCs that collaboratively optimize for lifetime and performance. Specifically, we leverage data compression to utilize partially defective NVM cache entries, thereby improving the LLC hit rate. The key to our approach is to guide the insertion policy by both the reuse properties of the block and the size resulting from its compression. A block is inserted in NVM only if it is a read-reuse block or its compressed size is lower than a threshold. It will be inserted in SRAM if the block is a write-reuse or its compressed size is greater than the threshold. We use set-dueling to tune the compression threshold at runtime. This compression threshold provides a knob to control the NVM write rate and, together with a rule-based mechanism, allows balancing performance and lifetime.Overall, our evaluation shows that, with affordable hardware overheads, the proposed schemes can nearly reach the performance of an SRAM cache with the same associativity while improving lifetime by 17× compared to a hybrid NVM-unaware LLC. Our proposed scheme outperforms the state-of-the-art insertion policies by 9% while achieving a comparative lifetime. The rule-based mechanism shows that by compromising, for instance, 1.1% and 1.9% performance, the NVM lifetime can be further increased by 28% and 44%, respectively.