Miracle-dz/UniFAR-FTUs huggingface dataset:
Infor-Coef: Information Bottleneck-based Dynamic Token Downsampling for Compact and Efficient language model

The prevalence of Transformer-based pre-trained language models (PLMs) has led to their wide adoption for various natural language processing tasks. However, their excessive overhead leads to large latency and computational costs. The statically compression methods allocate fixed computation to different samples, resulting in redundant computation. The dynamic token pruning method selectively shortens the sequences but are unable to change the model size and hardly achieve the speedups as static pruning. In this paper, we propose a model accelaration approaches for large language models that incorporates dynamic token downsampling and static pruning, optimized by the information bottleneck loss. Our model, Infor-Coef, achieves an 18x FLOPs speedup with an accuracy degradation of less than 8\% compared to BERT. This work provides a promising approach to compress and accelerate transformer-based models for NLP tasks.
Evaluation of medium-large Language Models at zero-shot closed book generative question answering

Large language models (LLMs) have garnered significant attention, but the definition of “large” lacks clarity. This paper focuses on medium-sized language models (MLMs), defined as having at least six billion parameters but less than 100 billion. The study evaluates MLMs regarding zero-shot generative question answering, which requires models to provide elaborate answers without external document retrieval. The paper introduces an own test dataset and presents results from human evaluation. Results show that combining the best answers from different MLMs yielded an overall correct answer rate of 82.7% which is better than the 60.9% of ChatGPT. The best MLM achieved 71.8% and has 33B parameters, which highlights the importance of using appropriate training data for fine-tuning rather than solely relying on the number of parameters. More finegrained feedback should be used to further improve the quality of answers. The open source community is quickly closing the gap to the best commercial models.
Snowball sampling from graphs

We develop unbiased strategies to probabilistic T-wave snowball sampling from graphs, where the interest of estimation may concern finite-order subgraphs such as triangles, cycles or stars. Our approaches encompass also the finite-population sampling strategies to multiplicity sampling and adaptive cluster sampling, both of which can be recast as snowball sampling aimed at graph node totals. A general snowball sampling theory offers greater flexibility in terms of scope and efficiency of graph sampling, in addition to the existing random node or edge sampling methods.
Energy-frugal and Interpretable AI Hardware Design using Learning Automata

Energy efficiency is a crucial requirement for enabling powerful artificial intelligence applications at the microedge. Hardware acceleration with frugal architectural allocation is an effective method for reducing energy. Many emerging applications also require the systems design to incorporate interpretable decision models to establish responsibility and transparency. The design needs to provision for additional resources to provide reachable states in real-world data scenarios, defining conflicting design tradeoffs between energy efficiency. is challenging. Recently a new machine learning algorithm, called the Tsetlin machine, has been proposed. The algorithm is fundamentally based on the principles of finite-state automata and benefits from natural logic underpinning rather than arithmetic. In this paper, we investigate methods of energy-frugal artificial intelligence hardware design by suitably tuning the hyperparameters, while maintaining high learning efficacy. To demonstrate interpretability, we use reachability and game-theoretic analysis in two simulation environments: a SystemC model to study the bounded state transitions in the presence of hardware faults and Nash equilibrium between states to analyze the learning convergence. Our analyses provides the first insights into conflicting design tradeoffs involved in energy-efficient and interpretable decision models for this new artificial intelligence hardware architecture. We show that frugal resource allocation coupled with systematic prodigality between randomized reinforcements can provide decisive energy reduction while also achieving robust and interpretable learning.
Judgments of research co-created by generative AI: experimental evidence

The introduction of ChatGPT has fuelled a public debate on the use of generative AI (large language models; LLMs), including its use by researchers. In the current work, we test whether delegating parts of the research process to LLMs leads people to distrust and devalue researchers and scientific output. Participants (N=402) considered a researcher who delegates elements of the research process to a PhD student or LLM, and rated (1) moral acceptability, (2) trust in the scientist to oversee future projects, and (3) the accuracy and quality of the output. People judged delegating to an LLM as less acceptable than delegating to a human (d = -0.78). Delegation to an LLM also decreased trust to oversee future research projects (d = -0.80), and people thought the results would be less accurate and of lower quality (d = -0.85). We discuss how this devaluation might transfer into the underreporting of generative AI use.
Feasibility of Transfer Learning: A Mathematical Framework

Transfer learning is a popular paradigm for utilizing existing knowledge from previous learning tasks to improve the performance of new ones. It has enjoyed numerous empirical successes and inspired a growing number of theoretical studies. This paper addresses the feasibility issue of transfer learning. It begins by establishing the necessary mathematical concepts and constructing a mathematical framework for transfer learning. It then identifies and formulates the three-step transfer learning procedure as an optimization problem, allowing for the resolution of the feasibility issue. Importantly, it demonstrates that under certain technical conditions, such as appropriate choice of loss functions and data sets, an optimal procedure for transfer learning exists. This study of the feasibility issue brings additional insights into various transfer learning problems. It sheds light on the impact of feature augmentation on model performance, explores potential extensions of domain adaptation, and examines the feasibility of efficient feature extractor transfer in image classification.
Quantum current and holographic categorical symmetry

We establish the formulation for quantum current. Given a symmetry group G, let \mathcal{C}𝒞:=Rep G be its representation category. Physically, symmetry charges are objects of \mathcal{C}𝒞 and symmetric operators are morphisms in \mathcal{C}𝒞. The addition of charges is given by the tensor product of representations. For any symmetric operator O crossing two subsystems, the exact symmetry charge transported by O can be extracted. The quantum current is defined as symmetric operators that can transport symmetry charges over an arbitrary long distance. A quantum current exactly corresponds to an object in the Drinfeld center Z_1(\mathcal{C})Z1(𝒞). The condition for quantum currents to be is also specified, which corresponds to condensation of anyons in one higher dimension. To express the local conservation, the internal hom must be used to compute the charge difference, and the framework of enriched category is inevitable. To illustrate these ideas, we develop a rigorous scheme of renormalization in one-dimensional lattice systems and analyse the fixed-point models. It is proved that in the fixed-point models, superconducting quantum currents form a Lagrangian algebra in Z_1(\mathcal{C})Z1(𝒞) and the boundary-bulk correspondence is verified in the enriched setting. Overall, the quantum current provides a natural physical interpretation to the holographic categorical symmetry.
Pseudospin-orbit coupling and non-Hermitian effects in the Quantum Geometric Tensor of a plasmonic lattice

We theoretically predict the full quantum geometric tensor, comprising the quantum metric and the Berry curvature, for a square lattice of plasmonic nanoparticles. The gold nanoparticles act as dipole or multipole antenna radiatively coupled over long distances. The photonic-plasmonic eigenfunctions and energies of the system depend on momentum and polarization (pseudospin), and their topological properties are encoded in the quantum geometric tensor. By T-matrix numerical simulations, we identify a TE-TM band splitting at the diagonals of the first Brillouin zone, that is not predicted by the empty lattice band structure nor by the highly symmetric nature of the system. Further, we find quantum metric around these regions of the reciprocal space, and even a non-zero Berry curvature despite the trivial lattice geometry and absence of magnetic field. We show that this non-zero Berry curvature arises exclusively from non-Hermitian effects which break the time-reversal symmetry. The quantum metric, in contrast, originates from a pseudospin-orbit coupling given by the polarization and directional dependence of the radiation.
Using Wavelet Transformation in Blind Sources Separation of the Fetal Electrocardiogram

One of the most important problems in heart signal processing is the extraction of the fetal electrocardiogram (FECG). One of the reasons that we are interested in FECG extraction is that this signal consists of important characteristics about healthy conditions of the fetus. Based on available conditions, Blind Source Separation is a suitable method for this problem. Existence of noise in observed signals from electrodes on the mother's body, can affect the separation performance. Therefore, signal de-noising is an important stage in this problem. In this study, using wavelet transform and optimum selection of its parameters in FECG extraction has been investigated. The first reason for using wavelet transform is to remove noise from the observed signals and the second reason is to apply it into BBS algorithms. Depending to the noise level in signals, wavelet transform can be used before or after signal separation, also it can be used both before and after signal separation. Simulation results show the performance of each method in different conditions for obtaining the desired signal at the presence of noise.
ALMERIA: Boosting pairwise molecular contrasts with scalable methods

Searching for potential active compounds in large databases is a necessary step to reduce time and costs in modern drug discovery pipelines. Such virtual screening methods seek to provide predictions that allow the search space to be narrowed down. Although cheminformatics has made great progress in exploiting the potential of available big data, caution is needed to avoid introducing bias and provide useful predictions with new compounds. In this work, we propose the decision-support tool ALMERIA (Advanced Ligand Multiconformational Exploration with Robust Interpretable Artificial Intelligence) for estimating compound similarities and activity prediction based on pairwise molecular contrasts while considering their conformation variability. The methodology covers the entire pipeline from data preparation to model selection and hyperparameter optimization. It has been implemented using scalable software and methods to exploit large volumes of data -- in the order of several terabytes -- , offering a very quick response even for a large batch of queries. The implementation and experiments have been performed in a distributed computer cluster using a benchmark, the public access DUD-E database. In addition to cross-validation, detailed data split criteria have been used to evaluate the models on different data partitions to assess their true generalization ability with new compounds. Experiments show state-of-the-art performance for molecular activity prediction (ROC AUC: $0.99$, $0.96$, $0.87$), proving that the chosen data representation and modeling have good properties to generalize. Molecular conformations -- prediction performance and sensitivity analysis -- have also been evaluated. Finally, an interpretability analysis has been performed using the SHAP method.
HighLight: Efficient and Flexible DNN Acceleration with Hierarchical Structured Sparsity

Due to complex interactions among various deep neural network (DNN) optimization techniques, modern DNNs can have weights and activations that are dense or sparse with diverse sparsity degrees. To offer a good trade-off between accuracy and hardware performance, an ideal DNN accelerator should have high flexibility to efficiently translate DNN sparsity into reductions in energy and/or latency without incurring significant complexity overhead. This paper introduces hierarchical structured sparsity (HSS), with the key insight that we can systematically represent diverse sparsity degrees by having them hierarchically composed from multiple simple sparsity patterns. As a result, HSS simplifies the underlying hardware since it only needs to support simple sparsity patterns; this significantly reduces the sparsity acceleration overhead, which improves efficiency. Motivated by such opportunities, we propose a simultaneously efficient and flexible accelerator, named HighLight, to accelerate DNNs that have diverse sparsity degrees (including dense). Due to the flexibility of HSS, different HSS patterns can be introduced to DNNs to meet different applications' accuracy requirements. Compared to existing works, HighLight achieves a geomean of up to 6.4x better energy-delay product (EDP) across workloads with diverse sparsity degrees, and always sits on the EDP-accuracy Pareto frontier for representative DNNs.
Bi-Lipschitz embeddings of the space of unordered $m$-tuples with a partial transportation metric

Let $\Omega\subset \mathbb{R}^n$ be non-empty, open and proper. Consider $Wb(\Omega)$, the space of finite Borel measures on $\Omega$ equipped with the partial transportation metric introduced by Figalli and Gigli that allows the creation and destruction of mass on $\partial \Omega$. Equivalently, we show that $Wb(\Omega)$ is isometric to a subset of all Borel measures with the ordinary Wasserstein distance, on the one point completion of $\Omega$ equipped with the shortcut metric \[\delta(x,y)= \min\{\|x-y\|, \operatorname{dist}(x,\partial \Omega)+\operatorname{dist}(y,\partial\Omega)\}. \] In this article we construct bi-Lipschitz embeddings of the set of unordered $m$-tuples in $Wb(\Omega)$ into Hilbert space. This generalises Almgren's bi-Lipschitz embedding theorem to the setting of optimal partial transport.
Block Coordinate Plug-and-Play Methods for Blind Inverse Problems

Plug-and-play (PnP) prior is a well-known class of methods for solving imaging inverse problems by computing fixed-points of operators combining physical measurement models and learned image denoisers. While PnP methods have been extensively used for image recovery with known measurement operators, there is little work on PnP for solving blind inverse problems. We address this gap by presenting a new block-coordinate PnP (BC-PnP) method that efficiently solves this joint estimation problem by introducing learned denoisers as priors on both the unknown image and the unknown measurement operator. We present a new convergence theory for BC-PnP compatible with blind inverse problems by considering nonconvex data-fidelity terms and expansive denoisers. Our theory analyzes the convergence of BC-PnP to a stationary point of an implicit function associated with an approximate minimum mean-squared error (MMSE) denoiser. We numerically validate our method on two blind inverse problems: automatic coil sensitivity estimation in magnetic resonance imaging (MRI) and blind image deblurring. Our results show that BC-PnP provides an efficient and principled framework for using denoisers as PnP priors for jointly estimating measurement operators and images.
EEG and EMG dataset for the detection of errors introduced by an active orthosis device

This paper presents a dataset containing recordings of the electroencephalogram (EEG) and the electromyogram (EMG) from eight subjects who were assisted in moving their right arm by an active orthosis device. The supported movements were elbow joint movements, i.e., flexion and extension of the right arm. While the orthosis was actively moving the subject's arm, some errors were deliberately introduced for a short duration of time. During this time, the orthosis moved in the opposite direction. In this paper, we explain the experimental setup and present some behavioral analyses across all subjects. Additionally, we present an average event-related potential analysis for one subject to offer insights into the data quality and the EEG activity caused by the error introduction. The dataset described herein is openly accessible. The aim of this study was to provide a dataset to the research community, particularly for the development of new methods in the asynchronous detection of erroneous events from the EEG. We are especially interested in the tactile and haptic-mediated recognition of errors, which has not yet been sufficiently investigated in the literature. We hope that the detailed description of the orthosis and the experiment will enable its reproduction and facilitate a systematic investigation of the influencing factors in the detection of erroneous behavior of assistive systems by a large community.
Towards Consistent Stochastic Human Motion Prediction via Motion Diffusion

Stochastic Human Motion Prediction (HMP) aims to predict multiple possible upcoming pose sequences based on past human motion trajectories. Although previous approaches have shown impressive performance, they face several issues, including complex training processes and a tendency to generate predictions that are often inconsistent with the provided history, and sometimes even becoming entirely unreasonable. To overcome these issues, we propose DiffMotion, an end-to-end diffusion-based stochastic HMP framework. DiffMotion's motion predictor is composed of two modules, including (1) a Transformer-based network for initial motion reconstruction from corrupted motion, and (2) a Graph Convolutional Network (GCN) to refine the generated motion considering past observations. Our method, facilitated by this novel Transformer-GCN module design and a proposed variance scheduler, excels in predicting accurate, realistic, and consistent motions, while maintaining an appropriate level of diversity. Our results on benchmark datasets show that DiffMotion significantly outperforms previous methods in terms of both accuracy and fidelity, while demonstrating superior robustness.
PO-VINS: An Efficient Pose-Only LiDAR-Enhanced Visual-Inertial State Estimator

The pose-only (PO) visual representation has been proven to be equivalent to the classical multiple-view geometry, while significantly improving computational efficiency. However, its applicability for real-world navigation in large-scale complex environments has not yet been demonstrated. In this study, we present an efficient pose-only LiDAR-enhanced visual-inertial navigation system (PO-VINS) to enhance the real-time performance of the state estimator. In the visual-inertial state estimator (VISE), we propose a pose-only visual-reprojection measurement model that only contains the inertial measurement unit (IMU) pose and extrinsic-parameter states. We further integrated the LiDAR-enhanced method to construct a pose-only LiDAR-depth measurement model. Real-world experiments were conducted in large-scale complex environments, demonstrating that the proposed PO-VISE and LiDAR-enhanced PO-VISE reduce computational complexity by more than 50% and over 20%, respectively. Additionally, the PO-VINS yields the same accuracy as conventional methods. These results indicate that the pose-only solution is efficient and applicable for real-time visual-inertial state estimation.
Model Debiasing via Gradient-based Explanation on Representation

Machine learning systems produce biased results towards certain demographic groups, known as the fairness problem. Recent approaches to tackle this problem learn a latent code (i.e., representation) through disentangled representation learning and then discard the latent code dimensions correlated with sensitive attributes (e.g., gender). Nevertheless, these approaches may suffer from incomplete disentanglement and overlook proxy attributes (proxies for sensitive attributes) when processing real-world data, especially for unstructured data, causing performance degradation in fairness and loss of useful information for downstream tasks. In this paper, we propose a novel fairness framework that performs debiasing with regard to both sensitive attributes and proxy attributes, which boosts the prediction performance of downstream task models without complete disentanglement. The main idea is to, first, leverage gradient-based explanation to find two model focuses, 1) one focus for predicting sensitive attributes and 2) the other focus for predicting downstream task labels, and second, use them to perturb the latent code that guides the training of downstream task models towards fairness and utility goals. We show empirically that our framework works with both disentangled and non-disentangled representation learning methods and achieves better fairness-accuracy trade-off on unstructured and structured datasets than previous state-of-the-art approaches.
Systematic Literature Review on Application of Machine Learning in Continuous Integration

This research conducted a systematic review of the literature on machine learning (ML)-based methods in the context of Continuous Integration (CI) over the past 22 years. The study aimed to identify and describe the techniques used in ML-based solutions for CI and analyzed various aspects such as data engineering, feature engineering, hyper-parameter tuning, ML models, evaluation methods, and metrics. In this paper, we have depicted the phases of CI testing, the connection between them, and the employed techniques in training the ML method phases. We presented nine types of data sources and four taken steps in the selected studies for preparing the data. Also, we identified four feature types and nine subsets of data features through thematic analysis of the selected studies. Besides, five methods for selecting and tuning the hyper-parameters are shown. In addition, we summarised the evaluation methods used in the literature and identified fifteen different metrics. The most commonly used evaluation methods were found to be precision, recall, and F1-score, and we have also identified five methods for evaluating the performance of trained ML models. Finally, we have presented the relationship between ML model types, performance measurements, and CI phases. The study provides valuable insights for researchers and practitioners interested in ML-based methods in CI and emphasizes the need for further research in this area.