Reviewerly/exHarmony huggingface dataset:
Explaining Aha! moments in artificial agents through IKE-XAI: Implicit Knowledge Extraction for eXplainable AI

During the learning process, a child develops a mental representation of the task he or she is learning. A Machine Learning algorithm develops also a latent representation of the task it learns. We investigate the development of the knowledge construction of an artificial agent through the analysis of its behavior, i.e., its sequences of moves while learning to perform the Tower of Hanoï (TOH) task. The TOH is a well-known task in experimental contexts to study the problem-solving processes and one of the fundamental processes of children's knowledge construction about their world. We position ourselves in the field of explainable reinforcement learning for developmental robotics, at the crossroads of cognitive modeling and explainable AI. Our main contribution proposes a 3-step methodology named Implicit Knowledge Extraction with eXplainable Artificial Intelligence (IKE-XAI) to extract the implicit knowledge, in form of an automaton, encoded by an artificial agent during its learning. We showcase this technique to solve and explain the TOH task when researchers have only access to moves that represent observational behavior as in human-machine interaction. Therefore, to extract the agent acquired knowledge at different stages of its training, our approach combines: first, a Q-learning agent that learns to perform the TOH task; second, a trained recurrent neural network that encodes an implicit representation of the TOH task; and third, an XAI process using a post-hoc implicit rule extraction algorithm to extract finite state automata. We propose using graph representations as visual and explicit explanations of the behavior of the Q-learning agent. Our experiments show that the IKE-XAI approach helps understanding the development of the Q-learning agent behavior by providing a global explanation of its knowledge evolution during learning. IKE-XAI also allows researchers to identify the agent's Aha! moment by determining from what moment the knowledge representation stabilizes and the agent no longer learns.
LinE: Logical Query Reasoning over Hierarchical Knowledge Graphs

Logical reasoning over Knowledge Graphs (KGs) for first-order logic (FOL) queries performs the query inference over KGs with logical operators, including conjunction, disjunction, existential quantification and negation, to approximate true answers in embedding spaces. However, most existing work imposes strong distributional assumptions (e.g., Beta distribution) to represent entities and queries into presumed distributional shape, which limits their expressive power. Moreover, query embeddings are challenging due to the relational complexities in multi-relational KGs (e.g., symmetry, anti-symmetry and transitivity). To bridge the gap, we propose a logical query reasoning framework, Line Embedding (LinE), for FOL queries. To relax the distributional assumptions, we introduce the logic space transformation layer, which is a generic neural function that converts embeddings from probabilistic distribution space to LinE embeddings space. To tackle multi-relational and logical complexities, we formulate neural relation-specific projections and individual logical operators to truthfully ground LinE query embeddings on logical regularities and KG factoids. Lastly, to verify the LinE embedding quality, we generate a FOL query dataset from WordNet, which richly encompasses hierarchical relations. Extensive experiments show superior reasoning sensitivity of LinE on three benchmarks against strong baselines, particularly for multi-hop relational queries and negation-related queries.
User-Event Graph Embedding Learning for Context-Aware Recommendation

Most methods for context-aware recommendation focus on improving the feature interaction layer, but overlook the embedding layer. However, an embedding layer with random initialization often suffers in practice from the sparsity of the contextual features, as well as the interactions between the users (or items) and context. In this paper, we propose a novel user-event graph embedding learning (UEG-EL) framework to address these two sparsity challenges. Specifically, our UEG-EL contains three modules: 1) a graph construction module is used to obtain a user-event graph containing nodes for users, intents and items, where the intent nodes are generated by applying intent node attention (INA) on nodes of the contextual features; 2) a user-event collaborative graph convolution module is designed to obtain the refined embeddings of all features by executing a new convolution strategy on the user-event graph, where each intent node acts as a hub to efficiently propagate the information among different features; 3) a recommendation module is equipped to integrate some existing context-aware recommendation model, where the feature embeddings are directly initialized with the obtained refined embeddings. Moreover, we identify a unique challenge of the basic framework, that is, the contextual features associated with too many instances may suffer from noise when aggregating the information. We thus further propose a simple but effective variant, i.e., UEG-EL-V, in order to prune the information propagation of the contextual features. Finally, we conduct extensive experiments on three public datasets to verify the effectiveness and compatibility of our UEG-EL and its variant.
Automatically Discovering User Consumption Intents in Meituan

Consumption intent, defined as the decision-driven force of consumption behaviors, is crucial for improving the explainability and performance of user-modeling systems, with various downstream applications like recommendation and targeted marketing. However, consumption intent is implicit, and only a few known intents have been explored from the user consumption data in Meituan. Hence, discovering new consumption intents is a crucial but challenging task, which suffers from two critical challenges: 1) how to encode the consumption intent related to multiple aspects of preferences, and 2) how to discover the new intents with only a few known ones. In Meituan, we designed the AutoIntent system, consisting of the disentangled intent encoder and intent discovery decoder, to address the above challenges. Specifically, for the disentangled intent encoder, we construct three groups of dual hypergraphs to capture the high-order relations under the three aspects of preferences and then utilize the designed hypergraph neural networks to extract disentangled intent features. For the intent discovery decoder, we propose to build intent-pair pseudo labels based on the denoised feature similarities to transfer knowledge from known intents to new ones. Extensive offline evaluations verify that AutoIntent can effectively discover unknown consumption intents. Moreover, we deploy AutoIntent in the recommendation engine of the Meituan APP, and the further online evaluation verifies its effectiveness.
Simultaneous Control and Guidance of an AUV Based on Soft Actor–Critic

The marine environment is a hostile setting for robotics. It is strongly unstructured, uncertain, and includes many external disturbances that cannot be easily predicted or modeled. In this work, we attempt to control an autonomous underwater vehicle (AUV) to perform a waypoint tracking task, using a machine learning-based controller. There has been great progress in machine learning (in many different domains) in recent years; in the subfield of deep reinforcement learning, several algorithms suitable for the continuous control of dynamical systems have been designed. We implemented the soft actor-critic (SAC) algorithm, an entropy-regularized deep reinforcement learning algorithm that allows fulfilling a learning task and encourages the exploration of the environment simultaneously. We compared a SAC-based controller with a proportional integral derivative (PID) controller on a waypoint tracking task using specific performance metrics. All tests were simulated via the UUV simulator. We applied these two controllers to the RexROV 2, a six degrees of freedom cube-shaped remotely operated underwater Vehicle (ROV) converted in an AUV. We propose several interesting contributions as a result of these tests, such as making the SAC control and guiding the AUV simultaneously, outperforming the PID controller in terms of energy saving, and reducing the amount of information needed by the SAC algorithm inputs. Moreover, our implementation of this controller allows facilitating the transfer towards real-world robots. The code corresponding to this work is available on GitHub.
Communication-Efficient Federated Learning Over MIMO Multiple Access Channels

Communication efficiency is of importance for wireless federated learning systems. In this paper, we propose a communication-efficient strategy for federated learning over multiple-input multiple-output (MIMO) multiple access channels (MACs). The proposed strategy comprises two components. When sending a locally computed gradient, each device compresses a high dimensional local gradient to multiple lower-dimensional gradient vectors using block sparsification. When receiving a superposition of the compressed local gradients via a MIMO-MAC, a parameter server (PS) performs a joint MIMO detection and the sparse local-gradient recovery. Inspired by the turbo decoding principle, our joint detection-and-recovery algorithm accurately recovers the high-dimensional local gradients by iteratively exchanging their beliefs for MIMO detection and sparse local gradient recovery outputs. We then analyze the reconstruction error of the proposed algorithm and its impact on the convergence rate of federated learning. From simulations, our gradient compression and joint detection-and-recovery methods diminish the communication cost significantly while achieving identical classification accuracy for the case without any compression.
Indian Sign Language Recognition Using Deep Learning Techniques

By automatically translating Indian sign language into English speech, a portable multimedia Indian sign language translation program can help the deaf and/or speaker connect with hearing people. It could act as a translator for those that do not understand sign language, eliminating the need for a mediator and allowing communication to take place in the speaker's native language. As a result, Deaf-Dumb people are denied regular educational opportunities. Uneducated Deaf-Dumb people have a difficult time communicating with members of their culture. We provide an incorporated Android application to help ignorant Deaf-Dumb people fit into society and connect with others. The newly launched program includes a straight forward keyboard translator that really can convert any term from Indian sign language to English. The proposed system is an interactive application program for mobile phones created with application software. The mobile phone is used to photograph Indian sign language gestures, while the operating system performs vision processing tasks and the constructed audio device output signals speech, limiting the need for extra devices and costs. The perceived latency between both the hand signals as well as the translation is reduced by parallel processing. This allows for a very quick translation of finger and hand motions. This is capable of recognizing one-handed sign representations of the numbers 0 through 9. The findings show that the results are highly reproducible, consistent, and accurate.
Sensitive information protection in blockchain-based supply-chain management for aerospace

Supply chain management is a critical aspect for every manufacturing business, in particular for safety critical systems where quality assessment and requirement satisfaction are evaluated in order to guarantee a certification of the final system. In this paper we present a blockchain-based methodology that focuses on enforcing strict data access control and that securely includes manufacturing equipment in the supply chain management process. The methodology provides cyber-security guarantees exploiting secure automated interaction between a blockchain supply chain management system and industrial manufacturing devices. A prototype of the proposed approach is implemented with state-of-the-art distributed ledger technologies integrating low-power devices hardened through the application of hardware security modules.
Intraoral 3-D Measurement by Means of Group Coding Combined With Consistent Enhancement for Fringe Projection Pattern

Intraoral 3D measurement can obtain digital impressions in real time. However, owing to saliva, enamel, metallic denture, etc., the quality of the captured 2D image using the intraoral equipment, which is used for feature measurement and 3D reconstruction, is usually degraded. In this paper, we propose a equipment with high performance for intraoral 3D measurement. First, group-multi-line coding algorithm instead of sinusoidal or gray code for fringe projection profilometry (FPP) was applied for stripe pattern decoding. The distance between adjacent stripes was large enough to avoid the scatter of the tooth. It can also overcome the shortcoming of sinusoidal-based phase computing, leading to accuracy degradation owing to scanner jittering. Second, a consistent stripe feature enhancement method is proposed. The stripe feature in the mirror, exposure, and low-contrast area was homogeneously enhanced. Consequently, an individual scene is reconstructed using more effective and accurate points. This not only improves the precision of point cloud registration, it also reduces the probability of mismatch and shortens the scanning time for a full-arch. Our results showed that the accuracy of intraoral 3D measurements conducted through the proposed equipment by using novel algorithms increased to 10 - 20 &#x03BC;m and the measurement efficiency increased to 30%.
Improved YOLOv4 algorithm for safety management of on-site power system work

To achieve real-time safety management of on-site power system work, the object detection algorithm of the cloud platform is transplanted into the edge computing device. However, due to the limited capacity and arithmetic power of the AI chip carried in the edge computing device, the algorithm needs to be studied in a lightweight manner. Therefore, this paper proposes an improved edge-side lightweight YOLOv4 (You Only Look Once version 4) algorithm, which is compressed and accelerated by improving the network structure and the confidence loss function of the algorithm without losing too much detection accuracy. Firstly, a lightweight convolutional neural network based on depthwise separable convolution and mobile inverted bottleneck convolution is constructed as the backbone network of YOLOv4, and the depthwise separable convolution is used instead of the conventional convolution, which can greatly reduce the parameters number and computation of the algorithm. Secondly, an improved bidirectional feature fusion network is constructed as the neck network of YOLOv4 algorithm to improve the information availability of multi-semantic features and enhance the extraction capability of small-scale features. Then, the confidence loss function of the algorithm is improved to alleviate the problems of uneven distribution of positive samples and negative samples and uneven distribution of easy-to-classify samples and hard-to-classify samples. Finally, the improved YOLOv4 algorithm is tested on an edge computing device equipped with an NVIDIA Jetson Xavier NX chip, and the dataset based on the compliance of on-site dress code in the electric power system field is tested. The experimental results show that, compared with the original YOLOv4 algorithm, the improved YOLOv4 algorithm reduces the parameters number by 93.11%, and the real-time video stream detection speed on the edge computing device is increased by 22.00%, reaching 15.05 FPS. The lightweight improved YOLOv4 algorithm is suitable for deployment at the edge end and can achieve real-time accurate identification of risks in the on-site work.
Tilt-Then-Blur or Blur-Then-Tilt? Clarifying the Atmospheric Turbulence Model

Imaging at a long distance often requires advanced image restoration algorithms to compensate for the distortions caused by atmospheric turbulence. However, unlike many standard restoration problems such as deconvolution, the forward image formation model of the atmospheric turbulence does not have a simple expression. Thanks to the Zernike representation of the phase, one can show that the forward model is a combination of tilt (pixel shifting due to the linear phase terms) and blur (image smoothing due to the high order aberrations). Confusions then arise between the ordering of the two operators. Should the model be tilt-then-blur, or blur-then-tilt? Some papers in the literature say that the model is tilt-then-blur, whereas more papers say that it is blur-then-tilt. This paper clarifies the differences between the two and discusses why the tilt-then-blur is the correct model. Recommendations are given to the research community.
Hand Gestures Identification for Fine-Grained Human Activity Recognition in Smart Homes

Most developed countries are facing important demographic issues related to ageing populations. Maintaining elders at home while ensuring their safety and well-being often constitutes the main goal of these countries. An interesting solution to this challenge is to develop a smart home, able to monitor the routines of the resident, to recognize the on-going activities, and to provide support when required. In the literature, most works focus on monitoring high-level behaviors such as eating, sleeping, etc. However, to provide live guidance, the system needs to have a far more detailed recognition process able to identify the specific steps of the on-going task and the erroneous executions. In this paper, we propose an algorithmic approach for hand gesture recognition designed to be used as the core component of a fine-grained activity recognition model. It is based on the processing of inertial data collected from a wristband equipped with triaxial accelerometer and gyroscope, and machine learning techniques. A simple set of gestures for cooking activities as been defined, enabling characterizing high-level cooking tasks. To do that, we constructed a labelled dataset of atomic gestures performed by participants that we made available to the scientific community. We obtained an average accuracy of 0.83 in recognizing the gestures with the leave-one-subject-out strategy.
Deep Learning-Based 3D Measurements with Near-Infrared Fringe Projection

Fringe projection profilometry (FPP) is widely applied to 3D measurements, owing to its advantages of high accuracy, non-contact, and full-field scanning. Compared with most FPP systems that project visible patterns, invisible fringe patterns in the spectra of near-infrared demonstrate fewer impacts on human eyes or on scenes where bright illumination may be avoided. However, the invisible patterns, which are generated by a near-infrared laser, are usually captured with severe speckle noise, resulting in 3D reconstructions of limited quality. To cope with this issue, we propose a deep learning-based framework that can remove the effect of the speckle noise and improve the precision of the 3D reconstruction. The framework consists of two deep neural networks where one learns to produce a clean fringe pattern and the other to obtain an accurate phase from the pattern. Compared with traditional denoising methods that depend on complex physical models, the proposed learning-based method is much faster. The experimental results show that the measurement accuracy can be increased effectively by the presented method.
Face Frontalization for Image Set Based Face Recognition

Image set based face recognition has recently become a popular topic as it has better performance than single image based face recognition. However, preprocessing is needed to remove the effects of some adverse conditions such as different pose angles, illumination, and expression differences within the set. One of the most effective preprocessing to improve the face recognition rate is face frontalization. Face frontalization is defined as the artificial acquisition of a face image with a different pose angle to a frontal pose. It has been observed that this process increases the face recognition performance. In this paper, image set based face recognition was performed by applying face frontalization to all images in the sets. Firstly, the faces in the IJBA database were frontalized by using the Rotate and Render hybrid frontalization method, which is based on a Three-Dimensional and Generative Adversial Network. Then, discriminative convex classifier is used for set based face recognition. In face recognition experiments, when the frontalized IJBA database and its non-frontalized version were compared, it was observed that the accuracy of face recognition increased with the frontalized face images.
Accelerated Convergence of Contracted Quantum Eigensolvers through a Quasi-Second-Order, Locally Parameterized Optimization

A contracted quantum eigensolver (CQE) finds a solution to the many-electron Schrödinger equation by solving its integration (or contraction) to the two-electron space─a contracted Schrödinger equation (CSE)─on a quantum computer. When applied to the anti-Hermitian part of the CSE (ACSE), the CQE iterations optimize the wave function, with respect to a general product ansatz of two-body exponential unitary transformations that can exactly solve the Schrödinger equation. In this work, we accelerate the convergence of the CQE and its wave function ansatz via tools from classical optimization theory. By treating the CQE algorithm as an optimization in a local parameter space, we can apply quasi-second-order optimization techniques, such as quasi-Newton approaches or nonlinear conjugate gradient approaches. Practically, these algorithms result in superlinear convergence of the wave function to a solution of the ACSE. Convergence acceleration is important because it can both minimize the accumulation of noise on near-term intermediate-scale quantum (NISQ) computers and achieve highly accurate solutions on future fault-tolerant quantum devices. We demonstrate the algorithm, as well as some heuristic implementations relevant for cost-reduction considerations, comparisons with other common methods such as variational quantum eigensolvers, and a Fermionic-encoding-free form of the CQE.
What do our sampling assumptions affect: How we encode data or how we reason from it?

In describing how people generalize from observed samples of data to novel cases, theories of inductive inference have emphasized the learner's reliance on the contents of the sample. More recently, a growing body of literature suggests that different assumptions about how a data sample was generated can lead the learner to draw qualitatively distinct inferences on the basis of the same observations. Yet, relatively little is known about how and when these two sources of evidence are combined. Do sampling assumptions affect how the sample contents are encoded, or is any influence exerted only at the point of retrieval when a decision is to be made? We report two experiments aimed at exploring this issue. By systematically varying both the sampling cover story and whether it is given before or after the training stimuli we are able to determine whether encoding or retrieval issues drive the impact of sampling assumptions. We find that the sampling cover story affects generalization when it is presented before the training stimuli, but not after, which suggests that sampling assumptions are integrated during encoding. (PsycInfo Database Record (c) 2023 APA, all rights reserved).