Reviewerly/exHarmony huggingface dataset:
Neural Feature Learning for Engineering Problems

Using deep neural networks as elements of engineering solutions can potentially enhance the overall performance of the system. However, most existing practices that use DNNs as black boxes make integrating DNN modules in engineering systems hard. In this paper, we address one of such difficulties: in engineering solutions, we often look for parameterized solutions that perform well in a collection of scenarios. In most problems, this means the DNN modules are trained and used in different environments. Instead of using a transfer learning or multi-task learning formulation, which are common in the literature, we argue that such problems are intrinsically about the multivariate dependence between the input, the target, and the environment parameters. Using an example of symbol detection over wireless fading channels with interference, we demonstrate that such problems can generally be formulated as a decomposition of multivariate dependence. We establish a geometric structure for the space of feature functions, based on which we develop new metrics to measure the information contents of features. We also develop some basic neural network architectures to perform geometric operations on features. With these building blocks, we discuss the steps to build a receiver that does not require any online training but can adapt to different fading scenarios when given the channel state information (CSI). We also discuss some key issues and steps to include DNN modules in general engineering systems.
Verification and Design of Robust and Safe Neural Network-enabled Autonomous Systems

Neural networks are being applied to a wide range of tasks in autonomous systems, such as perception, prediction, planning, control, and general decision making. While they may improve system performance over traditional physical model-based methods, pressing concerns have been raised on the uncertain behaviors of neural networks under varying inputs, especially for safety-critical systems such as autonomous vehicles and robots. In this paper, we will discuss the challenges in ensuring the safety and robustness of neural network-enabled autonomous systems, and present our recent work in addressing these challenges. These include methods for certifying the robustness of neural networks, verifying the safety of neural network-controlled systems, designing these systems with safety assurance, and conducting safety-assured runtime adaptation.
Sequential Dependence Decomposition and Feature Learning

In this paper, we introduce an approach to decompose statistical dependence and learn informative features from sequential data. We first present a sequential decomposition of dependence, which extends the chain rule of mutual information. To learn this decomposition from data, we investigate the optimal feature representations associated with decomposed dependence modes and develop corresponding learning algorithms. Specifically, for stationary processes, we demonstrate applications of the learned features in computing optimal Markov approximation and testing the order of Markov processes.
Block-missing data in linear systems: An unbiased stochastic gradient descent approach

Achieving accurate approximations to solutions of large linear systems is crucial, especially when those systems utilize real-world data. A consequence of using real-world data is that there will inevitably be missingness. Current approaches for dealing with missing data, such as deletion and imputation, can introduce bias. Recent studies proposed an adaptation of stochastic gradient descent (SGD) in specific missing-data models. In this work, we propose a new algorithm, ℓ-tuple mSGD, for the setting in which data is missing in a block-wise, tuple pattern. We prove that our proposed method uses unbiased estimates of the gradient of the least squares objective in the presence of tuple missing data. We also draw connections between ℓ-tuple mSGD and previously established SGD-type methods for missing data. Furthermore, we prove our algorithm converges when using updating step sizes and empirically demonstrate the convergence of ℓ-tuple mSGD on synthetic data. Lastly, we evaluate ℓ-tuple mSGD applied to real-world continuous glucose monitoring (CGM) device data.
An Achievable Low Complexity Encoding Scheme for Coloring Cyclic Graphs

In this paper, we utilize graph coloring for functions captured by cycle graphs to achieve lossless function compression.For functional compression of characteristic graphs or cycles and their OR graphs, we present a feasible scheme for coloring the odd cycles and their OR power graphs.In addition, we calculate the chromatic number for coloring the n-th power of even cycles, which paves the way for calculating the minimum entropy coloring.Moreover, we determine the number of distinct eigenvalues in each OR power and the largest eigenvalue for each power, which allows us to derive a bound on the chromatic number of each OR power of a cycle.Our method provides a proxy for understanding the fundamental limits of compressing cyclic graphs, which has broad applications in various practical use cases of cryptography and computer algebra.
Robust Transfer Learning Based on Minimax Principle

The similarity between target and source tasks is a crucial quantity for theoretical analyses and algorithm designs in transfer learning studies. However, this quantity is often difficult to be precisely captured. To address this issue, we make a boundedness assumption on the task similarity and then propose a mathematical framework based on the minimax principle, which minimizes the worst-case expected population risk under this assumption. Furthermore, our proposed minimax problem can be solved analytically, which provides a guideline for designing robust transfer learning models. According to the analytical expression, we interpret the influences of sample sizes, task distances, and the model dimensionality in knowledge transferring. Then, practical algorithms are developed based on the theoretical results. Finally, experiments conducted on image classification tasks show that our approaches can achieve robust and competitive accuracies for practical datasets.
Inverse Reinforcement Learning with Learning and Leveraging Demonstrators’ Varying Expertise Levels

A common assumption in most Inverse Reinforcement Learning (IRL) methods is that human demonstrations are drawn from an optimal policy. However, this assumption poses a problem in real-world applications, as humans may have varying aptitudes for different tasks. To address this, preference-based IRL methods, using human rankings of trajectories, have been explored as a potential solution. More recently, demonstrations from humans with different levels of expertise have also been utilized to relax this assumption. In this paper, we consider scenarios where there is no prior knowledge of the demonstrators’ expertise levels and humans are not readily available to provide rankings. Therefore, we propose a new method called Inverse Reinforcement Learning from Varying Expertise Levels (IRL-VEL), which combines demonstrations from different demonstrators with different levels of expertise to learn a high performance policy from their proficiency scores—an indication of how optimal their demonstrations are. In particular, we solve an optimization problem to learn these scores and subsequently use them to efficiently learn a reward function that optimizes a policy through reinforcement learning. To assess the performance of our algorithm, we learn two separate policies from two reward functions: an initial reward function directly obtained from these combined demonstrations and a final reward function that incorporates the proficiency scores. We evaluate the proposed algorithm in two simulation environments, namely, the continuous mountain car and the MuJoCo swimmer. The simulation results demonstrate that the proposed method can effectively learn and leverage the expertise levels underlying the demonstrations to learn a policy, significantly outperforming the methods that do not account for the expertise levels.
Application of an Adaptive IMM Algorithm with Optimized Tracking Performance in PD Radar

In the target observation scenario of PD radar, interacting multiple model algorithm (IMM) is most widely used. The probability transfer matrix of the conventional IMM is usually based on stationary priori information, and it is not conducive to improving the tracking accuracy of the target. If the target makes a maneuver change, it may trigger a series of problems such as tracking misalignment or interruption. Therefore, this paper proposed a method to adaptively modify Markov probability transfer matrix in real time. In addition, a fuzzy control system is added to redistribute the probabilities of each model, this will improve tracking accuracy for maneuvering targets. The simulation results show that the tracking accuracy of the algorithm in this paper is improved by 19.63% compared with the conventional IMM algorithm.
Advanced Neuro-PTF Method for A Four-Pole Waveguide Filter

In the electromagnetic (EM) parametric modeling area, the neuro-transfer function (neuro-TF) is a wide application and significant technology. The coefficients discontinuity issue of the neuro-TF method has been a hot research topic. This paper applies an advanced modeling technique using polynomial transfer functions (neuro-PTF) to a four-order microwave waveguide filter, building a precise parametric model, and improving the discontinuity issue.
The Analytical Solution for the Bistatic Quantum Radar Scattering of the 2D Elliptical Plate

The quantum radar technology is very much in its infancy. There still exist fundamental issues about the calculation of its radar signatures, especially in the 2D elliptical plate. In this study, we have developed an analytical solution for the bistatic quantum radar scattering of the 2D elliptical plate. Besides, the influence of incident frequency and the angles on the bistatic quantum cross section (BIQRCS) for the object were obtained. Finally, we also revealed that the envelope curves of sidelobes in BIQRCS at the given incidence almost display the invariance in frequency.
Simulation Approximators Using Linear and Nonlinear Integration Neural Networks

The construction of approximators for simulations, such as the finite element method using machine learning, has the problem of both reducing training data generation time and achieving approximation accuracy. Hybrid neural networks have been proposed to solve this problem as a fast approximator for simulation. Even if built with a simple perceptron with a linear activation function created based on deductive knowledge using conventional approximation techniques such as multiple regression analysis, the number of phenomena that can be modeled by deductive knowledge is limited in simulations of complex structures. As a result, there will be errors in the predictions of the approximators. In contrast, hybrid neural networks allow neural networks to learn errors in predictions to create correction approximators, allowing approximators to account for effects that cannot be expressed by multiple regression analysis. This paper proposes a neural network with a structure that integrates these approximators. The first proposed Hybrid Neural Network (HNN) approximator trains a linear approximator, and then a nonlinear approximator learns the error part. In contrast, the Integration Neural Network (INN) simultaneously learns the linear and nonlinear approximators to optimize the learning ratio by training. This method allows INNs to improve the accuracy of approximators and reduce the conflict between the number of training data and accuracy.
Fault location and separation method of Distributed Inertial Measurement Units based on IAC

With the development of unmanned technology, Distributed Inertial Measurement Units (DIMU) play an increasingly important role in Unmanned Aerial Vehicles (UAV). The large number of sensors and high data redundancy of DIMU bring huge challenges to its fault location and separation. However, the fault calculation complexity of the conventional methods increases with the number of sensors. Existing intelligent fault diagnosis methods usually determine the fault type for a single Inertial Measurement Units (IMU), and there are relatively few studies on fault location for DIMU. In this paper, we propose a DIMU fault location and separation method based on Improved Attention-CNN (IAC). First, we extract temporal and spatial nonlinear features of all sensor-measured data by the IAC encoder. Then, we complete the location and separation of fault sensor in DIMU by decoding the extracted fault features. We can better extract the spatio-temporal correlation features of data between sensors in DIMU to improve the accuracy of fault location and separation by the attention mechanism in IAC. IAC is not limited by the number of sensors because it is a data-driven fault diagnosis method. IAC fills the research gap of DIMU fault location and fault separation. The IAC can be used to identify the location number and fault type of the faulty IMU in the DIMU. Through simulation experiments, the fault location accuracy rate of our proposed IAC method reaches 95% and the fault separation accuracy rate after a fault occurs reaches 99%.
BIR-AHC: Balanced Iterative Reducing and Agglomerative Hierarchical Clustering for Stair Detection

Recently, stair environment perception has attracted considerable attention in humanoid robots. In complex stair environments, there are some problems, such as incomplete stairs, obstacle occlusion, and unequal stair heights, resulting in relatively low accuracy in constructing stairs. To solve this problem, we propose a balanced iterative reduction and cohesive hierarchical clustering method for stair environment perception, which consists of two main parts: a point cloud preprocessing module and a stair modeling module. Specifically, given a fixed resolution point cloud, we first preprocess and initially cluster the data using a hierarchy-based balanced iterative reduction and clustering algorithm to reduce point cloud noise and establish contextual links between point clouds. Then, we construct the graph between points and calculate the mean square error between individual points on this basis to efficiently construct the stair plane and effectively improve the accuracy of the 3D model of the stair. To evaluate the effectiveness and efficiency of the algorithm, we conduct experiments on RGB-D sensor datasets collected from different scenes, and the experimental results show that the proposed algorithm has a more satisfactory performance in terms of modeling accuracy and computation time for the stair environment.
HyperFed: Free-riding Resistant Federated Learning with Performance-based Reputation Mechanism and Adaptive Aggregation using Hypernetworks

Traditional machine learning solutions rely on Cloud based services, which could potentially lead to major problems including security, privacy data leakage, unacceptable latency, and excessive operating expenses. Federated Learning techniques (FL) were introduced to tackle these challenges by allowing distributed edge nodes/servers to collaboratively train AI models without sharing raw training data. However, some of the nodes may intentionally or unintentionally upload virtual (fake) models to the main server. This behavior is called "Free-riding", and it could potentially have a negative effect on the overall performance of the FL system. In this paper, we propose a new adaptive contribution-based aggregation technique using hypernetworks, namely "HyperFed", and evaluate it on two important aspects: resistance against free-riders’ fake contributions, and average convergence speed of global model on local datasets. Our simulation results on Federated EMNIST dataset display promising performance in comparison to FedAvg and AdaFed aggregation techniques.
Validity Analysis of Software Defect Prediction Model for Mis-label Correction Based on CleanLab

In code-based software defect prediction models, the problem of mislabeling often occurs, resulting in reduced model performance and inaccurate decisions. To tackle this issue, this study presents Cleanlab, a confidence learning-based method, for detecting and rectifying mis-labels. Unlike conventional approaches, our emphasis is on code-based defect prediction models. We validate the efficacy of Cleanlab by conducting case studies on 32 versions of nine projects, using three well-known models: CNN, DBN, and LINE-DP. The experimental findings demonstrate substantial enhancements in the robustness and accuracy of our code-based defect prediction model when incorporating Cleanlab.
Prototypical Networks with Dual Attention and Regularization for Few-Shot Relation Classification

Relation extraction is a vital subtask of information extraction. It is also an important component for constructing the knowledge graphs. The purpose of relation extraction is to find out the semantic relationships between the entities from the natural language so as to discover the connections between them. Most existing models for relation extraction heavily rely on annotation data which is difficult to obtain. Few-shot learning can overcome the problem of data sparsity effectively. Prototypical networks are important models for dealing with few-shot learning problems because of their simplicity and efficiency. In this paper we proposed a novel model (PNAR) that integrates the dual attention and regularization into prototypical networks for few-shot relation extraction. The experimental results verify that our model outperforms the several baseline model.
Blockchain-based Model for Privacy-enhanced Data Sharing

In the era of the digital economy, data has emerged as the "new era's oil," possessing significant value that necessitates refinement. Secure and efficient data sharing is the most effective approach to enhance data utilization and unlock its value. The demand for collaboration and data sharing among various organizations, businesses, and individuals is steadily increasing. However, the challenges of privacy, data security, and laws and regulations around the world make it difficult for all parties to engage in open circulation and value mining of data, resulting in data silos. The article proposes a generic model for sharing data between parties based on blockchain and privacy computing techniques, namely BPDS, to address the above challenges. Firstly, the model leverages blockchain and federated learning techniques to ensure the sharing process's traceability, immutability, and security. Additionally, a federated differential privacy algorithm is implemented to provide stringent privacy guarantees for the training data. Secondly, a gradient pruning method is devised to compress the model parameters, reducing communication overhead. Finally, the proposed model's utility, scalability, and security are evaluated using a real dataset. Experimental results demonstrate that the model achieves privacy protection while ensuring the quality of shared data, has low communication overhead, and its communication and computation overhead increases linearly with the number of participants. Furthermore, a comparative analysis is conducted to compare and analyze the proposed model with other data sharing models, confirming its security and efficiency.
Analyzing Software Supply Chain Security Risks in Industrial Control System Protocols: An OpenSSF Scorecard Approach

With a significant annual increase in software supply chain attacks over the past three years, concerns arise regarding the vulnerability of Industrial Control Systems (ICS) environments, especially given the growing use of open-source protocols’ implementations. Therefore, it is essential to assess the software supply chain security risks associated with these protocols. This study investigates the prevalent software supply chain security risks in open source protocol implementations used in ICS, compares software supply chain security risks between ICS and non-ICS protocols, evaluates assessed protocols’ strengths and weaknesses in terms of software supply chain security risks, and identifies opportunities to enhance their security. Using the Open Source Security Foundation (OpenSSF) Scorecard, the study analyzes nine ICS and five non-ICS protocols, highlighting strengths, weaknesses, and potential improvements. The study also identifies opportunities to enhance protocol and software supply chain security.
Optimizing Named Entity Recognition for Improving Logical Formulae Abstraction from Technical Requirements Documents

Requirements engineering involves obtaining a complete and consistent set of requirements for a particular system. Requirement engineers often formulate requirements in a textual form where automated proving consistency and completeness is impossible. Converting textual requirements into logical formulae would enable analysis and reasoning tasks. In this work, we contribute to converting textual requirements into logical sentences. In particular, we focus on the named entity recognition task on industrial requirements documents to improve semantic abstraction using logical formalisms. We found that using general-purpose models for named entity recognition is not working well for specialized domains. Hence, we focused on retraining such models and investigated the ratio of domain-specific versus general-purpose data for retraining. Our results demonstrate significant improvements in F1-Scores compared to the respective pre-trained baseline models when performing domain-specific tasks.
Maintainability Assessment for SOFL Formal Specifications

Formal specifications written in the SOFL specification language have been used in several industrial projects to help enhance software reliability. Experience suggests that specifications need to be frequently changed in different ways during the specification construction process and the specification-based implementation process. To make the changes easy to conduct without introducing potential faults, specifications must be ensured to have a high maintainability. To this end, we must first be able to assess the maintainability of formal specifications, but the problem we are facing is the lack of appropriate metrics for this purpose. To address this problem, we put forward new metrics for assessing the maintainability of formal specifications in this paper. Compared with metrics for code maintainability assessment, our metrics deal with the features of formal specifications that are distinct from code. Most rules in the proposed metrics can be applied to model-based formal specifications in general and only a small part is specific for the SOFL language. We discuss the principle and the assessment rules of our metrics and use a small case study to demonstrate how the metrics work in practice.