Reviewerly/exHarmony huggingface dataset:
An Explainable and Personalized Cognitive Reasoning Model Based on Knowledge Graph: Toward Decision Making for General Practice

General practice plays a prominent role in primary health care (PHC). However, evidence has shown that the quality of PHC is still unsatisfactory, and the accuracy of clinical diagnosis and treatment must be improved in China. Decision making tools based on artificial intelligence can help general practitioners diagnose diseases, but most existing research is not sufficiently scalable and explainable. An explainable and personalized cognitive reasoning model based on knowledge graph (CRKG) proposed in this article can provide personalized diagnosis, perform decision making in general practice, and simulate the mode of thinking of human beings utilizing patients' electronic health records (EHRs) and knowledge graph. Taking abdominal diseases as the application point, an abdominal disease knowledge graph is first constructed in a semiautomated manner. Then, the CRKG designed referring to dual process theory in cognitive science involves the update strategy of global graph representations and reasoning on a personal cognitive graph by adopting the idea of graph neural networks and attention mechanisms. For the diagnosis of diseases in general practice, the CRKG outperforms all the baselines with a precision@1 of 0.7873, recall@10 of 0.9020 and hits@10 of 0.9340. Additionally, the visualization of the reasoning process for each visit of a patient based on the knowledge graph enhances clinicians' comprehension and contributes to explainability. This study is of great importance for the exploration and application of decision making based on EHRs and knowledge graph.
VR/MR systems integrated with heat transfer simulation for training of thermoforming: A multicriteria decision-making user study

The emerging Industry 5.0 paradigm emphasizes the importance of collaboration between machines and human operators in smart factories, thereby preventing full automation and elimination of the skilled workforce. One technology that has been proposed recently as a means of achieving this collaboration, is the Extended Reality (XR), particularly for operators training and upskilling. However, current XR applications in advanced manufacturing sector are rather limited, mainly focusing on maintenance or sequential training. Additionally, most of earlier studies in the field have examined the use of one type of XR environment solely, such as Virtual Reality (VR), Augmented Reality (AR), or Mixed Reality (MR) in a respective application. The present study aims to develop and compare the performance of both MR and VR environments for training operators during the monitor and control of a given thermoforming manufacturing process. To achieve this, Meta Quest 2 (VR headset) and Microsoft HoloLens 2 (MR headset) were utilized in combination with heat transfer numerical simulations of the process. A video training tool was also created and employed as a reference/conventional training method, for comparisons. A user study involving 26 participants (with a mix of prior experience) was conducted, assessing different performance criteria including the usefulness, trust, reliability, user satisfaction, training effectiveness, and overall user preference over each training tool. Results, using statistical analysis along with five Multiple Criteria Decision Making (MCDM) algorithms, collectively indicated that the users preferred MR over VR and video training in the present case study; particularly as MR provided greater presence experience and satisfaction, higher user-friendliness and reliability. A demonstration of the developed applications can be found in the Supplementary Material.
Learnable convolutional attention network for knowledge graph completion

Recently, graph convolutional networks (GCNs) and graph attention networks (GATs) have been used extensively in knowledge graph completion (KGC), which aims to solve the incompleteness of knowledge graphs (KGs). However, both GCNs and GATs have limitations in the KGC task, and the best method is analyzing the neighbors of each entity (pre-validating), while this process is prohibitively expensive. Furthermore, relations in KGs have specific semantics and should be considered when aggregating neighbor information (message passing). To address the above limitations, we propose a learnable convolutional attention network for knowledge graph completion named LCA-KGC. LCA-KGC introduces a knowledge graph convolutional attention network using a convolution operation before the attention mechanism to ensure structural information acquisition and avoid redundant information stacking. Furthermore, to complete the autonomous switching of GNNs types and eliminate the necessity of pre-validating the local structure of KGs, LCA-KGC designs a learnable knowledge graph convolutional attention network by comprising three types of GNNs in one learnable formulation. Moreover, a learnable message function is proposed to emphasize relational semantics when aggregating neighbor information. Extensive experiments on standard KG datasets validate the effectiveness of the proposed innovations, and LCA-KGC achieves state-of-the-art (SOTA) performance compared to existing approaches (e.g., compared to SOTA approaches, LCA-KGC improves MRR from 0.360 to 0.372 on FB15k-237 dataset, and Hits@3 from 0.561 to 0.581 on YAGO3-10 dataset).
HSCNet++: Hierarchical Scene Coordinate Classification and Regression for Visual Localization with Transformer

Abstract Visual localization is critical to many applications in computer vision and robotics. To address single-image RGB localization, state-of-the-art feature-based methods match local descriptors between a query image and a pre-built 3D model. Recently, deep neural networks have been exploited to regress the mapping between raw pixels and 3D coordinates in the scene, and thus the matching is implicitly performed by the forward pass through the network. However, in a large and ambiguous environment, learning such a regression task directly can be difficult for a single network. In this work, we present a new hierarchical scene coordinate network to predict pixel scene coordinates in a coarse-to-fine manner from a single RGB image. The proposed method, which is an extension of HSCNet, allows us to train compact models which scale robustly to large environments. It sets a new state-of-the-art for single-image localization on the 7-Scenes, 12-Scenes, Cambridge Landmarks datasets, and the combined indoor scenes.
A New Approach Based on Collective Intelligence to Solve Traveling Salesman Problems

This paper presents a novel approach based on the ant system algorithm for solving discrete optimization problems. The proposed method is based on path construction, path improvement techniques, and the footprint mechanism. Some information about the optimization problem and collective intelligence is used in order to create solutions in the path construction phase. In the path improvement phase, neighborhood operations are applied to the solution, which is the best of the population and is obtained from the path construction phase. The collective intelligence in the path construction phase is based on a footprint mechanism, and more footprints on the arc improve the selection chance of this arc. A selection probability is also balanced by using information about the problem (e.g., the distance between nodes for a traveling salesman problem). The performance of the proposed method has been investigated on 25 traveling salesman problems and compared with state-of-the-art algorithms. The experimental comparisons show that the proposed method produced comparable results for the problems dealt with in this study.
Relation fusion propagation network for transductive few-shot learning

Previous graph-based meta-learning approaches have explored pairwise feature similarity to learn instance-level relations of samples, however, the gap between the sample relations in feature and label spaces is often ignored. It is empirically observed that instances with different labels may display considerable similarity in visual characteristics, making it challenging to distinguish between them in the feature space. To this end, we propose a dual-branch Relation Fusion Propagation Network (RFPN) for transductive few-shot learning, which explicitly models both feature and label relations across support-query pairs. Specifically, we design a Relation Fusion Block (RFB) to fuse instance-level and class-level relations, thus obtaining more robust fusion-level relations to guide feature propagation. In addition to the feature propagation branch, we encode the label relations and construct a label shortcut branch for label propagation. To alleviate the error accumulation during propagation, which is caused by uncertain pseudo-labels for query samples, we propose a Label Shortcut Mechanism (LSM) to progressively update the sample relations with the initial labels. Our full method is plug-and-play and can be easily applied in existing graph-based approaches for transductive few-shot learning. Extensive experiments demonstrate that our proposed RFPN yields significant improvements over the baselines and achieves promising performance on four popular few-shot classification benchmarks.
Hierarchical variable clustering via copula-based divergence measures between random vectors

This article considers rank-invariant clustering of continuous data via copula-based Φ-dependence measures. The general theoretical framework establishes dependence quantification between random vectors (groups of variables), which is used for measuring the similarity between variable clusters in an agglomerative hierarchical procedure afterwards. Special attention is devoted to meta-elliptical copulas, where we present an improved kernel estimator for the density generator and a corresponding bandwidth selector. This allows for non-Gaussian similarities also capturing e.g. tail dependence. Further, a fully non-parametric estimator is considered, enabling cluster detection in contexts where other measures fail. The theory is supported by simulations and a real data example, focusing on cluster analysis of continuous variables.
A Fast Attention Network for Joint Intent Detection and Slot Filling on Edge Devices

Intent detection and slot filling are two main tasks in natural language understanding and play an essential role in task-oriented dialogue systems. The joint learning of both tasks can improve inference accuracy and is popular in recent works. However, most joint models ignore the inference latency and cannot meet the need to deploy dialogue systems at the edge. In this paper, we propose a Fast Attention Network (FAN) for joint intent detection and slot filling tasks, guaranteeing both accuracy and latency. Specifically, we introduce a clean and parameter-refined attention module to enhance the information exchange between intent and slot, improving semantic accuracy by more than 2%. FAN can be implemented on different encoders and delivers more accurate models at every speed level. Our experiments on the Jetson Nano platform show that FAN inferences fifteen utterances per second with a small accuracy drop, showing its effectiveness and efficiency on edge devices.
Multi-view class incremental learning

Multi-view learning (MVL) has gained great success in integrating information from multiple perspectives of a dataset to improve downstream task performance. To make MVL methods more practical in an open-ended environment, this paper investigates a novel paradigm called multi-view class incremental learning (MVCIL), where a single model incrementally classifies new classes from a continual stream of views, requiring no access to earlier views of data. However, MVCIL is challenged by the catastrophic forgetting of old information and the interference with learning new concepts. To address this, we first develop a randomization-based representation learning technique serving for feature extraction to guarantee their separate view-optimal working states, during which multiple views belonging to a class are presented sequentially; Then, we integrate them one by one in the orthogonality fusion subspace spanned by the extracted features; Finally, we introduce selective weight consolidation for learning-without-forgetting decision-making while encountering new classes. Extensive experiments on synthetic and real-world datasets validate the effectiveness of our approach.
Semantic-Preserving Surgical Video Retrieval With Phase and Behavior Coordinated Hashing

Medical professionals rely on surgical video retrieval to discover relevant content within large numbers of videos for surgical education and knowledge transfer. However, the existing retrieval techniques often fail to obtain user-expected results since they ignore valuable semantics in surgical videos. The incorporation of rich semantics into video retrieval is challenging in terms of the hierarchical relationship modeling and coordination between coarse- and fine-grained semantics. To address these issues, this paper proposes a novel semantic-preserving surgical video retrieval (SPSVR) framework, which incorporates surgical phase and behavior semantics using a dual-level hashing module to capture their hierarchical relationship. This module preserves the semantics in binary hash codes by transforming the phase and behavior similarities into high- and low-level similarities in a shared Hamming space. The binary codes are optimized by performing a reconstruction task, a high-level similarity preservation task, and a low-level similarity preservation task, using a coordinated optimization strategy for efficient learning. A self-supervised learning scheme is adopted to capture behavior semantics from video clips so that the indexing of behaviors is unencumbered by fine-grained annotation and recognition. Experiments on four surgical video datasets for two different disciplines demonstrate the robust performance of the proposed framework. In addition, the results of the clinical validation experiments indicate the ability of the proposed method to retrieve the results expected by surgeons. The code can be found at https://github.com/trigger26/SPSVR.
Inter-domain mixup for semi-supervised domain adaptation

Semi-supervised domain adaptation (SSDA) aims to bridge source and target domain distributions, with a small number of target labels available, achieving better classification performance than unsupervised domain adaptation (UDA). However, existing SSDA work fails to make full use of label information from both source and target domains for feature alignment across domains, resulting in label mismatch in the label space during model testing. This paper presents a novel SSDA approach, Inter-domain Mixup with Neighborhood Expansion (IDMNE), to tackle this issue. Firstly, we introduce a cross-domain feature alignment strategy, Inter-domain Mixup, that incorporates label information into model adaptation. Specifically, we employ sample-level and manifold-level data mixing to generate compatible training samples. These newly established samples, combined with reliable and actual label information, display diversity and compatibility across domains, while such extra supervision thus facilitates cross-domain feature alignment and mitigates label mismatch. Additionally, we utilize Neighborhood Expansion to leverage high-confidence pseudo-labeled samples in the target domain, diversifying the label information of the target domain and thereby further increasing the performance of the adaptation model. Accordingly, the proposed approach outperforms existing state-of-the-art methods, achieving significant accuracy improvements on popular SSDA benchmarks, including DomainNet, Office-Home, and Office-31.
Pseudo-label estimation via unsupervised Identity Link Prediction for one-shot person Re-Identification

In this paper, we propose an unsupervised identity link prediction (ILP) method for label estimation in one-shot person Re-ID. ILP aims to relax the constraints of labeled samples and group a set of unlabeled pedestrians by their potential identities. The main idea is that the category relationships between pedestrians (nodes) can be inferred from their local context in the feature space. Specifically, an Identity Link Subgraph (ILS) describes the link relationship between nodes and their nearest neighbors, which is constructed by a two-step procedure. Meanwhile, a Dynamic Penalty Module (DPM) is introduced at each ILS construction step to infer which linkage between pairs in the ILS should be pruned to assign higher-quality classification labels. To fully use the accurate identity information in initial labeled samples, we jointly use identity pseudo-labels (which are estimated by adopting the Nearest Neighbors classifier) with classification pseudo-labels for model training. Moreover, we design a Dual-Branch Fusion network (DBF-Net) to optimize the CNN model simultaneously through all (pseudo-)labeled samples. Results on multiple datasets prove that DBF-Net outperforms traditional one-shot Re-ID methods by a large margin.
Simion Zoo: A training workbench for reinforcement learning allowing distributed experimentation

Simion Zoo is a Reinforcement Learning (RL) workbench developed for training of novel users that can be deployed over computer farms allowing extensive experimentation over distributed resources. In this paper, we present this software platform and share some of the insights gained during the development. This workbench provides a complete set of tools to design, run, carry out statistical analysis of the results, report preparation, and have qualitative visual assessment of the simulation evolution of continuous RL control experiments. The main features that set apart Simion Zoo from other software packages for introduction to RL experimentation are its easy-to-use GUI, its support for distributed execution including deployment over graphics processing units (GPUs), and the possibility to explore concurrently the RL hyper-parameter space, which is key to successful RL experimentation.
Recursive posterior Cramér–Rao lower bound on Lie groups

The recursive posterior Cramér–Rao Lower Bound (CRLB) introduced by Tichavsky et al. (1998) is an efficient tool to estimate the minimum variance of an estimator. Lately, filters and observers on matrix Lie groups have raised interest in control and nonlinear estimation. However, the recursive CRLB applies to variables belonging to the Euclidean space, and a recursive formulation on matrix Lie groups was not introduced yet. Hence, this paper derives a recursive formula of the Fisher information matrix on Lie groups, which reveals to be a natural extension of the information matrix on the Euclidean space. This generic result is applied to nonlinear Gaussian systems on Lie groups and tested on a navigation problem. The proposed recursive CRLB is consistent with state-of-the-art filters and shows a representative behavior with respect to the estimation errors. This paper provides a simple method to recursively compute the minimal variance of an estimator on matrix Lie groups. It enables to assess the behavior of an estimator, which is fundamental to implement robust algorithms.
BiPR-RL: Portrait relighting via bi-directional consistent deep reinforcement learning

Portrait relighting via deep reinforcement learning could conduct portrait relighting by sequentially predicting local light editing strokes, simulating image editing by artists using brush strokes. It is a locally effective, scale-invariant and interpretable method, which yields state-of-the-art results in relighting wild portrait images. However, it still has problems due to the lack of supervision information on intermediate processes, which makes the reinforcement learning agent not receive sufficient feedback during the exploration process. This will cause incorrect strokes in the relighting. To further optimize the relighting effect, we take advantage of the fact that the light editing actions are invertible and the inverse actions can generate a backward sequence of states. The backward state sequence also provide useful supervision signals as they convey information about the future states given the action choices. Therefore, we propose to combine the forward and the backward state sequence predictions to improve the learning efficiency for portrait relighting. Using the bi-directional state sequences, we design corresponding bi-directional consistent rewards to guide the model to explore the actions with higher accuracy to maximize the performance of the proposed method. The proposed approach is used for portrait relighting tasks based on both SH-lighting and reference images. The results show that our method has further improved the performance by avoiding wrong strokes. Meanwhile, our method performs better than SOTA methods in producing locally effective relighting images for wild portrait images.
A novel partition strategy for efficient implementation of 3D Cellular Genetic Algorithms

Solving optimization problems while fulfilling real-time constraints requires high algorithmic and processing performance. Cellular Genetic Algorithms (cGAs) have been competitive at difficult single objective combinatorial and continuous domain problems. Moreover, it has been demonstrated that structural properties in cGAs, such as population topology dimension, local neighborhood configuration and ad-hoc selection mechanisms, allow not only further algorithmic improvement but also, these characteristics can be combined at hardware level for acceleration. In this article, a novel partition strategy to exploit 3D cGAs population dynamics on a 2D processing array using Field Programmable Gate Arrays (FPGAs) as the target processing platform is presented. The proposed architecture fits as an optimization module within an embedded system where real-time constraints must be fulfilled. Therefore, it is important to find an optimal trade-off between hardware resources usage and searching time. Overall results demonstrate that the proposed architecture can run up to 90 MHz when tackling continuous benchmark functions. Moreover, speed-up of up to three and two orders of magnitude are achieved in comparison to a single CPU and a parallel GPU respectively.
Identifying Expert Behavior in Offline Training Datasets Improves Behavioral Cloning of Robotic Manipulation Policies

This paper presents our solution for the Real Robot Challenge III, aiming to address dexterous robotic manipulation tasks through learning from offline data. In this competition, participants were given two types of datasets for each task: expert and mixed. Each expert dataset is collected by a high-skill policy, whereas the mixed dataset is collected using both expert and non-expert policies. We found that the vanilla behavioural cloning (BC) can learn a very proficient policy with minimal human intervention when trained on expert datasets. Notably, BC outperformed even the most advanced offline reinforcement learning (RL) algorithms. However, when applied to mixed datasets, the performance of BC deteriorates; similarly, the performance of offline RL algorithms is also less than satisfactory. Upon examining the provided datasets, it was apparent that each mixed dataset contained a significant proportion of expert data, which should enable the training of a proficient BC agent. However, the expert data is not labelled in the datasets. As a result, we propose a classifier to identify the pattern of the expert behaviour within a mixed dataset and then utilize it to isolate the expert data. To further boost the BC performance, we take advantage of the geometric symmetry of the arena to augment the training dataset through mathematical transformations. Ultimately, our submission outperformed that of all other participants. Our solution scripts are available on our website: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/wq13552463699/</uri> Real-Robot-Challenge-III-Winning-Solution.