Reviewerly/exHarmony huggingface dataset:
Algebraic Signature-Based Public Data Integrity Batch Verification for Cloud-IoT

With the rapid development of Internet of Things, the related data are growing explosively. However, IoT devices have limited storage and computing capabilities so that they cannot deal with massive data storage and computing locally. The integration of IoT and cloud is regarded as an effective solution to the above issue, i.e., IoT devices outsource collected data to cloud to enjoy powerful storage and computing resources. Because the data stored in cloud are out of the control of IoT users, some security risks need to be addressed in advance. In this paper, we propose a public data integrity verification scheme, called AIVCI, to check the data integrity for Cloud-IoT scenarios. Firstly, based on algebraic signature and homomorphic hash function, AIVCI can efficiently complete data auditing. Secondly, AIVCI adopts blind technology to prevent the privacy leakage of IoT data and further protect the privacy of IoT users. Thirdly, batch auditing is implemented to improve auditing efficiency and meet realistic demands for Cloud-IoT scenarios. And a new data structure named Improved Divide and Conquer Table (ID&CT) is designed to realize efficient data dynamics. Finally, the security and performance analysis demonstrates that AIVCI is more secure and efficient.
Secure Convolutional Neural Network-Based Internet-of-Healthcare Applications

Convolutional Neural Networks (CNNs) have gained popularity for Internet-of-Healthcare (IoH) applications such as medical diagnostics.However, new research shows that adversarial attacks with slight imperceptible changes can undermine deep neural network techniques in healthcare.This raises questions regarding the safety of deploying these IoH devices in clinical situations.In this paper, we review the techniques used in fighting against cyber-attacks.Then, we propose to study the robustness of some well-known CNN architectures' belonging to sequential, parallel, and residual families, such as LeNet5, MobileNetV1, VGG16, ResNet50, and InceptionV3 against Fast Gradient Sign Method (FGSM) and Projected Gradient Descent (PGD) attacks, in the context of classification of chest radiographs (Xrays) based on the IoH application.Finally, we propose to improve the security of these CNN structures by studying standard and adversarial training.The results show that, among these models, smaller models with lower computational complexity are more secure against hostile threats than larger models that are frequently used in IoH applications.In contrast, we reveal that when these networks are learned adversarially, they can outperform standard trained networks.The experimental results demonstrate that the model performance breakpoint is represented by γ = 0.3 with a maximum loss of accuracy tolerated at 2%.
The StarCraft Multi-Agent Exploration Challenges: Learning Multi-Stage Tasks and Environmental Factors Without Precise Reward Functions

In this paper, we propose a novel benchmark called the StarCraft Multi-Agent Exploration Challenges(SMAC-Exp), where agents learn to perform multi-stage tasks and to use environmental factors without precise reward functions. The previous challenges (SMAC) recognized as a standard benchmark of Multi-Agent Reinforcement Learning are mainly concerned with ensuring that all agents cooperatively eliminate approaching adversaries only through fine manipulation with obvious reward functions. SMAC-Exp, on the other hand, is interested in the exploration capability of MARL algorithms to efficiently learn implicit multi-stage tasks and environmental factors as well as micro-control. This study covers both offensive and defensive scenarios. In the offensive scenarios, agents must learn to first find opponents and then eliminate them. The defensive scenarios require agents to use topographic features. For example, agents need to position themselves behind protective structures to make it harder for enemies to attack. We investigate a total twelve MARL algorithms under both sequential and parallel episode settings of SMAC-Exp and observe that recent approaches perform well in similar settings to the previous challenge, but we discover that current multi-agent approaches place relatively less emphasis on exploration perspectives. To a limited extent, we observe that an enhanced exploration method has a positive effect on SMAC-Exp, however there is still gap that state-of-art algorithms cannot resolve the most challenging scenarios of SMAC-Exp. Consequently, we propose a new axis for future research into Multi-Agent Reinforcement Learning studies.
ST-DepthNet: A spatio-temporal deep network for depth completion using a single non-repetitive circular scanning Lidar

In this paper, we propose a novel depth image completion technique based on sparse consecutive measurements of a non-repetitive circular scanning (NRCS) Lidar, demonstrating the capabilities of a new, compact, and accessible sensor technology for dense range mapping of highly dynamic scenes.Our deep network called ST-DepthNet is composed of a spatio-temporally (ST) extended U-Net architecture, which accepts a very sparse range data sequence as input and produces a dense depth image stream of the same field-of-view ensuring a high level of spatial details and accuracy.For evaluation, we have constructed a new urban dataset, that -to our best knowledge as the first open Benchmark in this field -comprises various simulated and realworld NRCS Lidar data samples, allowing us to simultaneously train our model on synthetic data with Ground Truth, and to validate the result via real NRCS Lidar measurements.Using this new dataset, we have shown the superiority of our method against a densified depth map obtained from the raw sensor stream, and against two independent state-of-the-art deep-learning based Lidar-only depth completion methods.
Fusion of Gaze and Scene Information for Driving Behaviour Recognition: A Graph-Neural-Network- Based Framework

Accurate recognition of driver behaviours is the basis for a reliable driver assistance system. This paper proposes a novel fusion framework for driver behaviour recognition that utilises the traffic scene and driver gaze information. The proposed framework is based on the graph neural network (GNN) and contains three modules, namely, the gaze analysing (GA) module, scene understanding (SU) module and the information fusion (IF) module. The GA module is used to obtain gaze images of drivers, and extract the gaze features from the images. The SU module provides trajectory predictions for surrounding vehicles, motorcycles, bicycles and other traffic participants. The GA and SU modules are parallel and the outputs of both modules are sent to the IF module that fuses the gaze and scene information using the attention mechanism and recognises the driving behaviours through a combined classifier. The proposed framework is verified on a naturalistic driving dataset. The comparative experiments with the state-of-the-art methods demonstrate that the proposed framework has superior performance for driving behaviour recognition in various situations.
Self-consistent Graph Neural Networks for Semi-supervised Node Classification

Graph Neural Networks (GNNs), the powerful graph representation technique based on deep learning, have attracted great research interest in recent years. Although many GNNs have achieved the state-of-the-art accuracy on a set of standard benchmark datasets, they are still limited to traditional semi-supervised framework and lack of sufficient supervision information, especially for the large amount of unlabeled data. To overcome this issue, we propose a novel self-consistent graph neural networks (SCGNN) framework to enrich the supervision information from two aspects: the self-consistency of unlabeled data and the label information of labeled data. First, in order to extract the self-supervision information from the numerous unlabeled nodes, we perform graph data augmentation and leverage a self-consistent constraint to maximize the mutual information of the unlabeled nodes across different augmented graph views. The self-consistency can sufficiently utilize the intrinsic structural attributes of the graph to extract the self-supervision information from unlabeled data and improve the subsequent classification result. Second, to further extract supervision information from scarce labeled nodes, we introduce a fusion mechanism to obtain comprehensive node embeddings by fusing node representations of two positive graph views, and optimize the classification loss over labeled nodes to maximize the utilization of label information. We conduct comprehensive empirical studies on six public benchmark datasets in node classification task. In terms of accuracy, SCGNN improves by an average of 2.08% over the best baseline, and specifically by 5.8% on the Disease dataset.
Minimum Variance Embedded Intuitionistic Fuzzy Weighted Random Vector Functional Link Network

Randomized neural networks such as random vector functional link network have been successfully employed in regression and classification problems. In real world, most of the data contaminated by outliers and noisy samples and hence, intelligent models are needed to classify such data. To make the model robust to noise and incorporate the geometric structure of the data, we propose minimum variance intuitionistic fuzzy RVFL (MVIFRVFL) model. In the proposed MVIFRVFL model, each sample is assigned an intuitionistic fuzzy number that is defined based on membership and non membership values. Membership value of a sample is considered according to its distance from the class center and the nonmembership value is assigned to each sample based on the ratio of the number of heterogeneous points to the total number of points in its neighborhood. Moreover, we incorporate the geometric aspect of the data, we minimise the variance among the data points of each class to improve the generalization performance. The performance of the classification models are analysed on 33 datasets taken from UCI and KEEL repository. From the experimental results, it is evident that the proposed MVIFRVFL model outperformed the state-of-the-art algorithms with best accuracy, highest number of wins among all the datasets and grabbed lowest rank amongst the baseline models.
A Deep Learning-based Efficient Firearms Monitoring Technique for Building Secure Smart Cities

Violence, in any form, is a disgrace to our civilized world.Nevertheless, even in modern days, violence is an integral part of our society and causes the deaths of many innocent lives.One of the conventional means of violence is using a firearm.Firearm-related death is currently a global phenomenon.It is a threat to society and a challenge to law enforcement agencies.A significant portion of such crimes happens in semi-urban areas or cities.Nowadays, CCTV-based surveillance is widely used by governments and private organizations for monitoring and prevention.However, human-based monitoring requires a significant amount of person-hours as a resource and is prone to mistakes.On the other hand, automated smart surveillance for violent activities is more suitable for scale and reliability.The paper's main focus is to showcase that deep learning-based techniques can be used in combination to detect firearms (particularly guns).This paper uses different detection techniques, such as Faster Region-Based Convolutional Neural Networks (Faster RCNN) and the latest EfficientDet-based architectures for detecting guns and human faces.An ensemble (stacked) scheme has improved the detection performance to identify human faces and guns at the post-processing level using Non-Maximum Suppression, Non-Maximum Weighted, and Weighted Boxes Fusion techniques.This paper has empirically discussed the comparative results of various detection techniques and their ensembles.It helps the police to gather quick intelligence about the incident and take preventive measures at the earliest.Also, the same method can be used to identify social-media videos for gun-based content detection.Here, the Weighted Boxes Fusion-based ensemble detection scheme provides mean average precision 77.02%, 16.40%, 29.73% for the mAP 0.5 , mAP 0.75 and mAP [0.500.95], respectively.The results achieve the best performance among all the experimented alternatives.The model has been rigorously tested with unknown test images and movie clips.The obtained ensemble schemes are satisfactory and consistently improve over primary models.
Event Extraction With Dynamic Prefix Tuning and Relevance Retrieval

We consider event extraction in a generative manner with template-based conditional generation. Although there is a rising trend of casting the task of event extraction as a sequence generation problem with prompts, these generation-based methods have several significant challenges, including using suboptimal prompts, static event type information, and the overwhelming number of irrelevant event types. In this paper, we propose a generative template-based method with dynamic prefixes and a relevance retrieval framework for event extraction ( <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">GREE</small> ) by first integrating context information with type-specific prefixes to learn a context-specific prefix for each context, and then retrieving the relevant event types with an adaptive threshold. Experimental results show that our model achieves competitive results with the state-of-the-art classification-based model <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">OneIE</small> on ACE 2005 and achieves the best performances on ERE. Additionally, our model is proven to be portable to new types of events effectively.
IDPL: Intra-subdomain Adaptation Adversarial Learning Segmentation Method Based on Dynamic Pseudo Labels

Unsupervised domain adaptation(UDA) has been applied to image semantic segmentation to solve the problem of domain offset. However, in some difficult categories with poor recognition accuracy, the segmentation effects are still not ideal. To this end, in this paper, Intra-subdomain adaptation adversarial learning segmentation method based on Dynamic Pseudo Labels(IDPL) is proposed. The whole process consists of 3 steps: Firstly, the instance-level pseudo label dynamic generation module is proposed, which fuses the class matching information in global classes and local instances, thus adaptively generating the optimal threshold for each class, obtaining high-quality pseudo labels. Secondly, the subdomain classifier module based on instance confidence is constructed, which can dynamically divide the target domain into easy and difficult subdomains according to the relative proportion of easy and difficult instances. Finally, the subdomain adversarial learning module based on self-attention is proposed. It uses multi-head self-attention to confront the easy and difficult subdomains at the class level with the help of generated high-quality pseudo labels, so as to focus on mining the features of difficult categories in the high-entropy region of target domain images, which promotes class-level conditional distribution alignment between the subdomains, improving the segmentation performance of difficult categories. For the difficult categories, the experimental results show that the performance of IDPL is significantly improved compared with other latest mainstream methods.
Filter Pruning via Similarity Clustering for Deep Convolutional Neural Networks

Network pruning is a technique to obtain a smaller lightweight model by removing the redundant structure from pre-trained models. However, existing methods are mainly based on the importance of filters in the whole network. Unlike previous methods, in this paper, we propose a filter pruning strategy, called Filter Pruning via Similarity Clustering(FPSC). FPSC uses the Euclidean distance between filters to measure their similarity, and then selects the filter with the smaller sum of k-nearest neighbor distances among the similar filters for removal. We consider that the selected filter is more likely to be replaced by neighbor filters. FPSC is applied to a variety of different networks, and compared with the existing filter pruning approaches. The experimental results show that FPSC has better pruning performance. On CIFAR-10, it is worth noting that FPSC reduces more than 70 $$\%$$ FLOPs and parameters on GoogLeNet, and the accuracy is even 0.09 $$\%$$ higher than the baseline model. Moreover, on ImageNet, FPSC reduces more than 43.1 $$\%$$ FLOPs and 42.2 $$\%$$ parameters, the accuracy only dropped 0.66 $$\%$$ on ResNet-50.
DOM2R-Graph: A Web Attribute Extraction Architecture with Relation-Aware Heterogeneous Graph Transformer

Web attribute extraction refers to extracting structured entities with specific attributes (e.g. title, director, genre and mpaa rating for a movie) from HTML documents. Since each part of a web page corresponds to an unique node in the DOM tree, most of existing methods formulate web attribute extraction as a multi-class classification task of DOM tree nodes. However, they rarely focus on the multiple structural relations between DOM tree nodes, which will influence node semantic interactions. In this paper, we propose a novel web attribute extraction architecture called DOM2R-Graph, which integrates both node semantic information and heterogeneous structure information of DOM tree. Specifically, we first construct a heterogeneous graph by connecting DOM tree nodes and their contexts with edges indicating structural relations. Then, we propose a Relation-aware Heterogeneous Graph Transformer (RHGT), to effectively capture the heterogeneous features of structural relations and learn representations of nodes on the graph at a fine-grained level. Extensive experimental results on the public SWDE dataset show that DOM2R-Graph outperforms the state-of-the-art methods.
A Cross-Scale and Illumination Invariance-Based Model for Robust Object Detection in Traffic Surveillance Scenarios

Robust object detection methods in traffic surveillance scenarios often encounters challenges due to large-scale deformations and illumination variations in outdoor scenes. To enhance the tolerance of such methods against these variations, we design a cross-scale and illumination-invariant detection model (CSIM) based on the You Only Look Once (YOLO) architecture. A main cause of false detection in large-scale detection tasks is the inconsistency between various feature scales. To address this issue, we introduce an adaptive cross-scale feature fusion model to ensure the consistency of the constructed feature pyramid. To overcome the influence of uneven light, we build an illumination-invariant chromaticity space on the CSIM model, which is independent of the correlated color temperature. In addition, we adopt spatial attention modules, K-means clustering and the Mish activation function for further model optimization. The obtained experimental results show that the proposed CSIM produces excellent detection results for addressing the challenges derived from large-scale deformations and the illumination changes encountered during traffic surveillance. Compared with state-of-the-art object detection methods on public datasets, our proposed model has achieved competitive results in robust object detection tasks in traffic surveillance scenarios.
A Human Activity Recognition Method Based on Lightweight Feature Extraction Combined With Pruned and Quantized CNN for Wearable Device

Human Activity Recognition (HAR) is becoming an essential part of human life care. Existing HAR methods are usually developed using a two-level approach, wherein a first-level Machine Learning (ML) classifier is employed to distinguish the static and dynamic activities, followed by a second-level classifier to identify the specific activity. These approaches are not suitable for wearable devices, due to the high computational and memory consumption. Our rigorous analysis of various HAR datasets opens up a new possibility that static or dynamic activities can be discriminated against through a simple statistical technique. Therefore, we propose to utilize a statistical feature extraction technique to replace the first-level ML classifier, thus achieving more lightweight computation. Next, we employ Random Forest (RF) and Convolutional Neural Networks (CNN) to classify the specific activities, achieving higher accuracy compared to the state-of-the-art results. We further reduce the computation and memory consumption of the above combined approach by applying pruning and quantizing techniques to CNN (PQ-CNN). Experimental results show the proposed lightweight HAR method achieved an F1 score of 0.9417 and 0.9438 for unbalanced and balanced datasets, respectively. On top of lightweight and accuracy, the proposed HAR method is practical for wearable devices by using a single accelerometer.
SimpleMind: An open-source software environment that adds thinking to deep neural networks

Deep neural networks (DNNs) detect patterns in data and have shown versatility and strong performance in many computer vision applications. However, DNNs alone are susceptible to obvious mistakes that violate simple, common sense concepts and are limited in their ability to use explicit knowledge to guide their search and decision making. While overall DNN performance metrics may be good, these obvious errors, coupled with a lack of explainability, have prevented widespread adoption for crucial tasks such as medical image analysis. The purpose of this paper is to introduce SimpleMind, an open-source software environment for Cognitive AI focused on medical image understanding. It allows creation of a knowledge base that describes expected characteristics and relationships between image objects in an intuitive human-readable form. The knowledge base can then be applied to an input image to recognize and understand its content. SimpleMind brings thinking to DNNs by: (1) providing methods for reasoning with the knowledge base about image content, such as spatial inferencing and conditional reasoning to check DNN outputs; (2) applying process knowledge, in the form of general-purpose software agents, that are dynamically chained together to accomplish image preprocessing, DNN prediction, and result post-processing, and (3) performing automatic co-optimization of all knowledge base parameters to adapt agents to specific problems. SimpleMind enables reasoning on multiple detected objects to ensure consistency, providing cross-checking between DNN outputs. This machine reasoning improves the reliability and trustworthiness of DNNs through an interpretable model and explainable decisions. Proof-of-principle example applications are provided that demonstrate how SimpleMind supports and improves deep neural networks by embedding them within a Cognitive AI environment.
A new iterative initialization of EM algorithm for Gaussian mixture models

The expectation maximization (EM) algorithm is a common tool for estimating the parameters of Gaussian mixture models (GMM). However, it is highly sensitive to initial value and easily gets trapped in a local optimum.To address these problems, a new iterative method of EM initialization (MRIPEM) is proposed in this paper. It incorporates the ideas of multiple restarts, iterations and clustering. In particular, the mean vector and covariance matrix of sample are calculated as the initial values of the iteration. Then, the optimal feature vector is selected from the candidate feature vectors by the maximum Mahalanobis distance as a new partition vector for clustering. The parameter values are renewed continuously according to the clustering results.To verify the applicability of the MRIPEM, we compared it with other two popular initialization methods on simulated and real datasets, respectively. The comparison results of the three stochastic algorithms indicate that MRIPEM algorithm is comparable in relatively high dimensions and high overlaps and significantly better in low dimensions and low overlaps.