Reviewerly/exHarmony huggingface dataset:
Byzantine Resistant Secure Blockchained Federated Learning at the Edge

The emerging blockchained federated learning, known for its security properties such as decentralization, immutability and traceability, is evolving into an important direction of next-generation AI. With the booming edge computing technologies, blockchained federated learning can take advantage of computing, communication and storage resources geo-distributed at the edge, so that blockchained federated learning can gather edge intelligence from more widely distributed devices more efficiently. However, untrustworthy devices at the edge also bring serious security threats, namely byzantine attacks. Existing solutions focus on selecting local models that are most likely to be honest, rather than detecting byzantine models and identifying attackers, because verifying each local model separately brings intolerable verification delay. In this paper, we propose a byzantine resistant secure blockchained federated learning framework named BytoChain. BytoChain improves the efficiency of model verification by introducing verifiers to execute heavy verification workflows in parallel, and detects byzantine attacks through a byzantine resistant consensus Proof-of-Accuracy (PoA). We analyze how BytoChain can mitigate five types of attacks, and demonstrate its effectiveness by simulations. Finally, we envision some open issues about security, including attacks on privacy, confidentiality, and backdoors.
A new technique for guided filter based image denoising using modified cuckoo search optimization

In this work, a novel and efficient approach for image denoising is proposed. More often, noise affecting the pixels in image is Gaussian in nature and uniformly deters information pixels in image irrespective of their intensity values. This behaviour of noise can also be identified as Additive White Gaussian Noise (AWGN). For restoration of AWGN affected images, the proposed denoising approach is inspired by image adaptive guided image filtering using modified cuckoo search algorithm. The guidance image is itself derived from the noisy image for this purpose. Bilateral filtering smoothed noisy image is sharpened by unsharp masking and then employed as guidance image for the proposed optimal guided filtering approach. Optimal evaluation of parameters like guided filter smoothing parameter (regularization parameter or degree of smoothing (DoS)) and guided filter’s neighbourhood (kernel) size is done appropriately with the help of the modified cuckoo search algorithm. Two-dimensional search space is explored and exploited for deciding the behaviour of guided filtering adaptively as per the input image requirements. This guided image filter has a better behaviour at it acts as an edge preserving smoothing operator. It is considerably effective as its computational complexity is independent of filtering kernel size. A novel attempt is made by incorporating the Markov Random Field based Energy Minimization based objective/fitness function for imparting adaptive image denoising using metaheuristic intelligence. The proposed method is tested in terms of the performance metrics like peak signal to noise ratio, structural similarity index and mean square error. Performance of the proposed approach is compared with the already proposed image denoising techniques. For this comparison, only those methods are considered which were proposed for filtering of Gaussian Noise. Qualitative (visual) as well as quantitative (objective) results underlines the efficacy of the proposed method for filtering of Gaussian Noise.
Knowledge Graph Question Answering with semantic oriented fusion model

Knowledge Graph Question Answering (KGQA) is a major branch of question answering tasks, which can answer fact questions effectively by using the reasonable characteristics of the knowledge graph. Currently, lots of related works combined with a variety of deep learning models are presented for the KGQA task. However, there are still some challenges, such as topic entity recognition under ambiguity expression, semantic level representation of natural language, efficient construction of searching space for answers, etc. In this paper, we propose a comprehensive approach for complex question answering over KG. Firstly, during the stage of topic entity recognition, a deep transition model is constructed to extract topic entities, and an efficient entity linking strategy is presented, which combines character matching and entity disambiguation model. Secondly, for candidate path ranking, a dynamic candidate path generation algorithm is proposed to efficiently create the candidate answer set. And four dedicated similarity calculation models are designed to handle the intricate condition of complex questions with long sequence and diversity expression. Moreover, a fusion policy is proposed to make decision for the final correct answer. We evaluate our approach on CKBQA, a Chinese knowledge base question answering dataset, from CCKS2019 competition. Experimental results demonstrate that the improvements in each process are effective and our approach achieves better performance than the best team in CCKS2019 competition.
Smart Assistive Architecture for the Integration of IoT Devices, Robotic Systems, and Multimodal Interfaces in Healthcare Environments

This paper presents a new architecture that integrates Internet of Things (IoT) devices, service robots, and users in a smart assistive environment. A new intuitive and multimodal interaction system supporting people with disabilities and bedbound patients is presented. This interaction system allows the user to control service robots and devices inside the room in five different ways: touch control, eye control, gesture control, voice control, and augmented reality control. The interaction system is comprised of an assistive robotic arm holding a tablet PC. The robotic arm can place the tablet PC in front of the user. A demonstration of the developed technology, a prototype of a smart room equipped with home automation devices, and the robotic assistive arm are presented. The results obtained from the use of the various interfaces and technologies are presented in the article. The results include user preference with regard to eye-base control (performing clicks, and using winks or gaze) and the use of mobile phones over augmented reality glasses, among others.
Dissecting scientific explanation in AI (sXAI): A case for medicine and healthcare

Explanatory AI (XAI) is on the rise, gaining enormous traction with the computational community, policymakers, and philosophers alike. This article contributes to this debate by first distinguishing scientific XAI (sXAI) from other forms of XAI. It further advances the structure for bona fide sXAI, while remaining neutral regarding preferences for theories of explanations. Three core components are under study, namely, i) the structure for bona fide sXAI, consisting in elucidating the explanans, the explanandum, and the explanatory relation for sXAI: ii) the pragmatics of explanation, which includes a discussion of the role of multi-agents receiving an explanation and the context within which the explanation is given; and iii) a discussion on Meaningful Human Explanation, an umbrella concept for different metrics required for measuring the explanatory power of explanations and the involvement of human agents in sXAI. The kind of AI systems of interest in this article are those utilized in medicine and the healthcare system. The article also critically addresses current philosophical and computational approaches to XAI. Amongst the main objections, it argues that there has been a long-standing interpretation of classifications as explanation, when these should be kept separate.
Graph Attention Networks Adjusted Bi-LSTM for Video Summarization

The high redundancy among keyframes is a critical issue for the prior summarizing methods in dealing with user-created videos. To address the critical issue, we present a Graph Attention Networks (GAT) adjusted Bi-directional Long Short-term Memory (Bi-LSTM) model for unsupervised video summarization. First, the GAT is adopted to transform an image's visual features into higher-level features by the Contextual Features based Transformation (CFT) mechanism. Specifically, a novel Salient-Area-Size-based spatial attention model is presented to extract frame-wise visual features on the observation that humans tend to focus on sizable and moving objects. Second, the higher-level visual features are integrated with semantic features processed by Bi-LSTM to refine the frame-wise probability of being selected as keyframes. Extensive experiments demonstrate that our method outperforms state-of-the-art methods.
Security-aware task scheduling with deadline constraints on heterogeneous hybrid clouds

Hybrid cloud is a cost-effective way to address the problem of insufficient resources for satisfying its users' requirements in a private cloud by elastically scaling up or down its service capability by combining the private cloud and public clouds. However, it is a challenge to schedule tasks on hybrid resources concerning their performance and security requirements. To address the challenge, this paper aims at improving the number of finished tasks with deadline and security requirements and the resource usage cost in heterogeneous hybrid clouds, based on data protection technologies providing various security levels with different overheads for data transfers and task executions in public clouds. We first formulate the problem as a bi-objective binary nonlinear programming (BOBNP) model which is a NP-hard problem. Then, to solve the problem in polynomial time, we propose a Task Scheduling method concerning Security (TSS). To improve the cost, TSS iteratively assigns the task requiring maximum cost of public resources to the local cluster, and rents the public resource with the best cost-performance ratio first for outsourced tasks. To complete as many tasks as possible, TSS assigns tasks cannot be finished by public clouds to the local cloud at first, and employs the idea of Least Slack Time First (LSTF) with Earliest Deadline First (EDF) in each computing node. Extensive experimental results show the superior performance of TSS in satisfying task requirements, and in resource efficiency when task deadlines are not too tight, compared with four hybrid cloud scheduling methods proposed recently.
Articulated Object Interaction in Unknown Scenes with Whole-Body Mobile Manipulation

A kitchen assistant needs to operate human-scale objects, such as cabinets and ovens, in unmapped environments with dynamic obstacles. Autonomous interactions in such environments require integrating dexterous manipulation and fluid mobility. While mobile manipulators in different form factors provide an extended workspace, their real-world adoption has been limited. Executing a high-level task for general objects requires a perceptual understanding of the object as well as adaptive whole-body control among dynamic obstacles. In this paper, we propose a two-stage architecture for autonomous interaction with large articulated objects in unknown environments. The first stage, object-centric planner, only focuses on the object to provide an action-conditional sequence of states for manipulation using RGB-D data. The second stage, agent-centric planner, formulates the whole-body motion control as an optimal control problem that ensures safe tracking of the generated plan, even in scenes with moving obstacles. We show that the proposed pipeline can handle complex static and dynamic kitchen settings for both wheel-based and legged mobile manipulators. Compared to other agent-centric planners, our proposed planner achieves a higher success rate and a lower execution time. We also perform hardware tests on a legged mobile manipulator to interact with various articulated objects in a kitchen. For additional material, please check: www.pair.toronto.edularticulated-mm/.
Unsupervised video anomaly detection via normalizing flows with implicit latent features

In contemporary society, surveillance anomaly detection, i.e., spotting anomalous events such as crimes or accidents in surveillance videos, is a critical task. As anomalies occur rarely, most training data consists of unlabeled videos without anomalous events, which makes the task challenging. Most existing methods use an autoencoder (AE) to learn to reconstruct normal videos; they then detect anomalies based on their failure to reconstruct the appearance of abnormal scenes. However, because anomalies are distinguished by appearance as well as motion, many previous approaches have explicitly separated appearance and motion informationfor example, using a pre-trained optical flow model. This explicit separation restricts reciprocal representation capabilities between two types of information. In contrast, we propose an implicit two-path AE (ITAE), a structure in which two encoders implicitly model appearance and motion features, along with a single decoder that combines them to learn normal video patterns. For the complex distribution of normal scenes, we suggest normal density estimation of ITAE features through normalizing flow (NF)-based generative models to learn the tractable likelihoods and identify anomalies using out-of-distribution detection. NF models intensify ITAE performance by learning normality through implicitly learned features. Finally, we demonstrate the effectiveness of ITAE and its feature distribution modeling on six benchmarks, including databases that contain various anomalies in real-world scenarios.
Deep learning‐based action recognition with 3D skeleton: A survey

Action recognition based on 3D skeleton data has attracted much attention due to its wide application, and it is one of the most popular research topics in computer vision. The 3D skeleton data is an effective representation of motion dynamics and is not easily affected by light, scene variation, etc. Previous research on action recognition has mainly focused on video or RGB data methods. In recent years, the advantages of combining skeleton data and deep learning have been gradually demonstrated, many impressive methods have been proposed, especially GCN-based methods. In this survey, we first introduce the development process of 3D skeleton-data action recognition and the classification of graph convolutional network, then introduce the commonly used NTU RGB + D and NTU RGB + D 120 datasets. Finally, a detailed review of existing variants of three mainstream technologies is provided based on deep learning and their performance was compared from three dimensions. To the best of our knowledge, this is the first research to integrate the research of GCN-based method and its various evolutionary methods. Comparative investigation of existing variants of research in action-recognition task from different perspectives is made, a generic framework is described, state-of-the-art practices are summarized, and the emerging trends of this topic are explored.
Motion Planning and Control of an Omnidirectional Mobile Robot in Dynamic Environments

Motion control in dynamic environments is one of the most important problems in using mobile robots in collaboration with humans and other robots. In this paper, the motion control of a four-Mecanum-wheeled omnidirectional mobile robot (OMR) in dynamic environments is studied. The robot’s differential equations of motion are extracted using Kane’s method and converted to discrete state space form. A nonlinear model predictive control (NMPC) strategy is designed based on the derived mathematical model to stabilize the robot in desired positions and orientations. As a main contribution of this work, the velocity obstacles (VO) approach is reformulated to be introduced in the NMPC system to avoid the robot from collision with moving and fixed obstacles online. Considering the robot’s physical restrictions, the parameters and functions used in the designed control system and collision avoidance strategy are determined through stability and performance analysis and some criteria are established for calculating the best values of these parameters. The effectiveness of the proposed controller and collision avoidance strategy is evaluated through a series of computer simulations. The simulation results show that the proposed strategy is efficient in stabilizing the robot in the desired configuration and in avoiding collision with obstacles, even in narrow spaces and with complicated arrangements of obstacles.
A Memetic Path Planning Algorithm for Unmanned Air/Ground Vehicle Cooperative Detection Systems

The studies of Unmanned Air/Ground Vehicle (UAV/UGV) cooperative detection systems have received much attention due to their wide applications in the disaster rescue, target tracking, intelligent surveillance, and automatical package delivery missions. UAVs provide a broad view and have a fast speed in the air, while UGVs have sufficient load capacity and can serve as repeater stations on the ground. The path planning of a UAV/UGV cooperative system is an important but difficult issue, which aims to plan paths for both the UAVs and the UGVs in the system to cooperatively complete a mission. In this article, we consider the path planning problem of the UAV/UGV cooperative system for illegal urban building detection, by taking the limits of UGV speed, UAV load power, and UAV/UGV communication restriction into consideration. To solve this problem, we first model the path planning problem as a constraint optimization problem which tries to minimize an overall execution time for completing the illegal urban building detection tasks, and then propose a two-level memetic algorithm (called Two-MA) to solve the path planning problems of both the UAV and the UGV. Experiments on both synthetic and real-world data sets show the superiority of the proposed Two-MA over several states-of-the-art algorithms in solving the path planning problems of the UAV and UGV for illegal urban building detection tasks. Note to Practitioners—This article was motivated by the task of detecting illegal buildings in cities by unmanned vehicles. Previous works mainly focus on path planning of either UAVs or UGVs in this task. This article proposes a new approach using an Unmanned Air/Ground Vehicle (UAV/UGV) cooperative system for detecting illegal buildings in parks, by taking the limits of UGV speed, UAV load power, and UAV/UGV communication restriction into consideration. This cooperative system consists of a UAV, UGV, and control center. The UAV equipped with cameras takes aerial photography in the air, and can transmit collected photos to the control center. The UGV executes loading and transportation on the ground, and can serve as takeoff and landing platforms for the UAV. The control center executes computationally intensive tasks such as data transmission and processing, task scheduling, and vehicle coordination. To quickly complete all detection tasks, a memetic algorithm is proposed for path planning of both the UAV and the UGV. The simulated results show that the proposed algorithm enables the UAV/UGV cooperative system to visit all buildings in cities with a minimum task execution time.
A weighted feature enhanced Hidden Markov Model for spam SMS filtering

Short message service (SMS) is a most favored communication service people use in daily life. However, this service is being misused by spammers. Rule based systems (RBS) and content based filtering (CBF) techniques have been developed to filter out spam messages. New rules can be easily added into RBS, but the throughput usually reduces as the rules increase. The bag-of-words (BoW) assumption based CBF techniques ignore the word order, which use machine learning methods to extract features from SMS message body according to word frequency and distribution. Striving to improve performance, researchers developed hybrid models that made algorithms ever-more complex. In addition, frequently conducting the time consuming models training and deployment forces the anti-spam industry still rely mainly on rule-based systems with unsolved throughput issue. A discrete Hidden Markov Model (HMM) was proposed in our previous study to address these issues, and the HMM method achieved a comparable performance to the deep learning methods. To further improve the performance of HMM method, we propose a new approach to weight and label words in SMS for formatting the observation sequence in HMM method. The weighted feature enhanced HMM achieves higher accuracy, and much faster training and filtering speed for meeting the anti-spam industry requirement. The performance comparison with other machine learning methods is conducted on the same open respiratory data set maintained by University of California, Irvine (UCI). Experimental results show that the weighted features enhanced HMM outperforms the LSTM (long short-term memory model) and close to CNN (convolutional neural network) in terms of classification accuracy. In addition, a Chinese SMS data set is used to further validate filtering accuracy and filtering speed.
Global citation recommendation employing generative adversarial network

The variety and plethora of research papers available on the Web motivated researchers to propose models that could assist users with personalized citation recommendations. In recent years, citation recommendation models using Network Representation Learning (NRL) methods have shown promising results. Nevertheless, existing NRL-based models are limited in terms of exploiting semantic relations and contextual information between the objects of bibliographic papers’ networks. Additionally, these models cannot adequately explore the structure of heterogeneous information networks, topical relevance, and relevant semantics. Consequently, they suffer from network sparsity and inadequate personalization problems. To overcome these shortcomings, we present a network embedding model termed as Global Citation Recommendation employing Generative Adversarial Network (GCR-GAN). The proposed model exploits the Heterogeneous Bibliographic Network (HBN) to generate personalized citation recommendations. In particular, the proposed model utilizes semantic relations corresponding to the objects of the heterogeneous bibliographic network and captures network structure proximity employing the Scientific Paper Embeddings using Citation-informed Transformers (SPECTER) and Denoising Auto-encoder networks to learn semantic-preserving graph representations. Compared to baseline models, the recommendations generated by our model over the DBLP and ACM datasets prove that it outperforms baseline methods by gaining almost 11% and 12% improvement in terms of Mean Average Precision (MAP) and Normalized Discounted Cumulative Gain (nDCG) metrics, respectively. Furthermore, we analyzed the effectiveness of the proposed model considering network sparsity issue, where our model gains almost 7% better [email protected] score against the second-best counterpart.