Reviewerly/exHarmony huggingface dataset:
Detection of AI-Manipulated Fake Faces via Mining Generalized Features

Recently, AI-manipulated face techniques have developed rapidly and constantly, which has raised new security issues in society. Although existing detection methods consider different categories of fake faces, the performance on detecting the fake faces with “unseen” manipulation techniques is still poor due to the distribution bias among cross-manipulation techniques. To solve this problem, we propose a novel framework that focuses on mining intrinsic features and further eliminating the distribution bias to improve the generalization ability. First, we focus on mining the intrinsic clues in the channel difference image (CDI) and spectrum image (SI) view of two different aspects, including the camera imaging process and the indispensable step in AI manipulation process. Then, we introduce the Octave Convolution and an attention-based fusion module to effectively and adaptively mine intrinsic features from CDI and SI view of these two different but intrinsic aspects. Finally, we design an alignment module to eliminate the bias of manipulation techniques to obtain a more generalized detection framework. We evaluate the proposed framework on four categories of fake faces datasets with the most popular and state-of-the-art manipulation techniques and achieve very competitive performances. We further conduct experiments on cross-manipulation techniques, and the results of our method show the superior advantages on improving generalization ability.
GeoRec: Geometry-enhanced semantic 3D reconstruction of RGB-D indoor scenes

Semantic indoor 3D modeling with multi-task deep neural networks is an efficient and low-cost way for reconstructing an indoor scene with geometrically complete room structure and semantic 3D individuals. Challenged by the complexity and clutter of indoor scenarios, the semantic reconstruction quality of current methods is still limited by the insufficient exploration and learning of 3D geometry information. To this end, this paper proposes an end-to-end multi-task neural network for geometry-enhanced semantic 3D reconstruction of RGB-D indoor scenes (termed as GeoRec). In the proposed GeoRec, we build a geometry extractor that can effectively learn geometry-enhanced feature representation from depth data, to improve the estimation accuracy of layout, camera pose and 3D object bounding boxes. We also introduce a novel object mesh generator that strengthens the reconstruction robustness of GeoRec to indoor occlusion with geometry-enhanced implicit shape embedding. With the parsed scene semantics and geometries, the proposed GeoRec reconstructs an indoor scene by placing reconstructed object mesh models with 3D object detection results in the estimated layout cuboid. Extensive experiments conducted on two benchmark datasets show that the proposed GeoRec yields outstanding performance with 5.19×10-3 mean chamfer distance error for object reconstruction on the challenging Pix3D dataset, 70.45% mAP for 3D object detection and 77.1% 3D mIoU for layout estimation on the commonly-used SUN RGB-D dataset. Especially, the mesh reconstruction sub-network of GeoRec trained on Pix3D can be directly transferred to SUN RGB-D without any fine-tuning, manifesting a high generalization ability.
Deep Generative Models in the Industrial Internet of Things: A Survey

Advances in communication technologies and artificial intelligence are accelerating the paradigm of industrial Internet of Things (IIoT). With IIoT enabling continuous integration of sensors and controllers with the network, intelligent analysis of the generated Big Data is a critical requirement. Although IIoT is considered a subset of IoT, it has its own peculiarities in terms of higher levels of safety, security, and low-latency communication in an environment of critical real-time operations. Under these circumstances, discriminative deep learning (DL) algorithms are unsuitable due to their need for large amounts of labeled and balanced training data, uncertainty of inputs, etc. To overcome these issues, researchers have started using deep generative models (DGMs), which combine the flexibility of DL with the inference power of probabilistic modeling. In this article, we review the state of the art of DGMs and their applicability to IIoT, classifying the reviewed works into the IIoT application areas of anomaly detection, trust-boundary protection, network traffic prediction, and platform monitoring. Following an analysis of existing IIoT DGM implementations, we identify challenges (i.e., weak discriminative capability, insufficient interpretability, lack of generalization ability, generated data vulnerability, privacy concern, and data complexity) that need to be investigated in order to accelerate the adoption of DGMs in IIoT and also propose some potential research directions.
NoFED-Net: Nonlinear Fuzzy Ensemble of Deep Neural Networks for Human Activity Recognition

In the era of the Internet of Things (IoT), the need for human activity recognition (HAR) is growing, especially in smart-healthcare applications using on-body smart sensor devices. These devices amass data and employ various classification models to analyze and discern user activities. However, existing techniques that are susceptible to the data type, user inputs, and ensemble-based models lack the ability to correct a wrong classification made by a base classifier. Addressing the shortcomings, we propose a novel fuzzy ensemble of three deep neural networks, using three nonlinear functions to generate fuzzy scores. The proposed model works on sensor data and can adaptively penalize the activity classes when the classification is assumed to be incorrect. Besides, a novel rewarding technique is proposed that aids the ensemble to extract the correct class in adverse situations. The proposed model reports state-of-the-art accuracy when evaluated on four publicly available wearable sensor data sets. In addition, activities corresponding to real-time sensor data collected using a smartphone are predicted correctly by the proposed model, thereby establishing itself as a reliable smart-HAR model. We also discuss a possible future scope of implementing the model over the cloud for smart activity recognition.
Sample-Centric Feature Generation for Semi-Supervised Few-Shot Learning

Semi-supervised few-shot learning aims to improve the model generalization ability by means of both limited labeled data and widely-available unlabeled data. Previous works attempt to model the relations between the few-shot labeled data and extra unlabeled data, by performing a label propagation or pseudo-labeling process using an episodic training strategy. However, the feature distribution represented by the pseudo-labeled data itself is coarse-grained, meaning that there might be a large distribution gap between the pseudo-labeled data and the real query data. To this end, we propose a sample-centric feature generation (SFG) approach for semi-supervised few-shot image classification. Specifically, the few-shot labeled samples from different classes are initially trained to predict pseudo-labels for the potential unlabeled samples. Next, a semi-supervised meta-generator is utilized to produce derivative features centering around each pseudo-labeled sample, enriching the intra-class feature diversity. Meanwhile, the sample-centric generation constrains the generated features to be compact and close to the pseudo-labeled sample, ensuring the inter-class feature discriminability. Further, a reliability assessment (RA) metric is developed to weaken the influence of generated outliers on model learning. Extensive experiments validate the effectiveness of the proposed feature generation approach on challenging one- and few-shot image classification benchmarks.
The impact of intelligent personal assistants on adolescent EFL learners’ listening comprehension

Intelligent Personal Assistant (IPA) has emerged as a valuable tool for EFL learning by offering interactive authentic contexts. Although IPA is believed to be motivational and useful, empirical evidence is limited and contradictory, especially the efficacy of listening comprehension. Therefore, the study investigated the impact of IPA on adolescent EFL learners’ listening comprehension, particularly regarding the presentation mode of IPA responses and its interaction styles. Ninety-two ninth-grade EFL learners were recruited from three intact classes and randomly divided into two experimental groups (i.e. GA-Hub group, using Google Nest Hub with multimodal responses; GA-Mini group, using Google Nest Mini with audio responses) and one control group (non-GA group, using a CD player as they did in the conventional class). Both quantitative (English listening tests) and qualitative data (questionnaires and interviews) were collected and analyzed in the 10-week study. The results showed that Google Assistant, especially via Google Nest Hub, positively promoted the participants’ listening comprehension. Detailed analyses of interviews demonstrated that Google Assistant provided interactive listening, presented multimodal screen-based responses, encouraged peer collaboration, and offered edutainment (a combination of IPA- and game-based learning), which added authenticity, flexibility, and enjoyment for meaningful interaction and thus promoted EFL listening.
Decision-Based Adversarial Attack With Frequency Mixup

It has been widely observed that deep neural networks are highly vulnerable to adversarial examples. Decision-based attacks could generate adversarial examples based solely on top-1 labels returned by the target model. However, they typically make excessive queries and could not bypass detection effectively. To comprehensively assess a decision-based attack, besides its query efficiency, the performance against detection is also a concern. Considering that previous detections consume massive resources and always mistakenly recognize benign video frames as malicious attacks, we design a lightweight detection called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">boundary detection</i> to overcome the above limitations, whose success reveals serious limitations of existing decision-based attacks. To develop more powerful attacks, we first present <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">f-mixup</i> as a basic method to produce candidate adversarial examples in the frequency domain. Using <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">f-mixup</i> as the building block, we propose <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">f-attack</i> as a complete decision-based attack. With the help of several natural images, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">f-attack</i> could both work well with limited (hundreds of) queries and bypass detection effectively. Nevertheless, if the attacker could make relatively adequate (thousands of) queries and the target model is not equipped with detection, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">f-attack</i> will lag behind existing decision-based attacks. We additionally introduce <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">frequency binary search</i> based on <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">f-mixup</i> , which serves as a plug-and-play module for existing decision-based attacks to further improve their query efficiency. Experimental results verify the effectiveness of our proposed methods.
Multi-level Attention Map Network for Multimodal Sentiment Analysis

Multimodal sentiment analysis (MSA) is a very challenging task due to its complex and complementary interactions between multiple modalities, which can be widely applied into areas of product marketing, public opinion monitoring, and so on. However, previous works directly utilized the features extracted from multimodal data, in which the noise reduction within and among multiple modalities has been largely ignored before multimodal fusion. This paper proposes a multi-level attention map network (MAMN) to filter noise before multimodal fusion and capture the consistent and heterogeneous correlations among multi-granularity features for multimodal sentiment analysis. Architecturally, MAMN is comprised of three modules: multi-granularity feature extraction module, multi-level attention map generation module, and attention map fusion module. The first module is designed to sufficiently extract multi-granularity features from multimodal data. The second module is constructed to filter noise and enhance the representation ability for multi-granularity features before multimodal fusion. And the third module is built to extensibly mine the interactions among multi-level attention maps by the proposed extensible co-attention fusion method. Extensive experimental results on three public datasets show the proposed model is significantly superior to the state-of-the-art methods, and demonstrate its effectiveness on two tasks of document-based and aspect-based MSA tasks.
A Novel Multi-Sample Generation Method for Adversarial Attacks

Deep learning models are widely used in daily life, which bring great convenience to our lives, but they are vulnerable to attacks. How to build an attack system with strong generalization ability to test the robustness of deep learning systems is a hot issue in current research, among which the research on black-box attacks is extremely challenging. Most current research on black-box attacks assumes that the input dataset is known. However, in fact, it is difficult for us to obtain detailed information for those datasets. In order to solve the above challenges, we propose a multi-sample generation model for black-box model attacks, called MsGM. MsGM is mainly composed of three parts: multi-sample generation, substitute model training, and adversarial sample generation and attack. Firstly, we design a multi-task generation model to learn the distribution of the original dataset. The model first converts an arbitrary signal of a certain distribution into the shared features of the original dataset through deconvolution operations, and then according to different input conditions, multiple identical sub-networks generate the corresponding targeted samples. Secondly, the generated sample features achieve different outputs through querying the black-box model and training the substitute model, which are used to construct different loss functions to optimize and update the generator and substitute model. Finally, some common white-box attack methods are used to attack the substitute model to generate corresponding adversarial samples, which are utilized to attack the black-box model. We conducted a large number of experiments on the MNIST and CIFAR-10 datasets. The experimental results show that under the same settings and attack algorithms, MsGM achieves better performance than the based models.
An Intelligent handcrafted feature selection using Archimedes optimization algorithm for facial analysis

Human facial analysis (HFA) has recently become an attractive topic for computer vision research due to technological progress and mobile applications. HFA explores several issues as gender recognition (GR), facial expression, age, and race recognition for automatically understanding social life. This study explores HFA from the angle of recognizing a person's gender from their face. Several hard challenges are provoked, such as illumination, occlusion, facial emotions, quality, and angle of capture by cameras, making gender recognition more difficult for machines. The Archimedes optimization algorithm (AOA) was recently designed as a metaheuristic-based population optimization method, inspired by the Archimedes theory's physical notion. Compared to other swarm algorithms in the realm of optimization, this method promotes a good balance between exploration and exploitation. The convergence area is increased By incorporating extra data into the solution, such as volume and density. Because of the preceding benefits of AOA and the fact that it has not been used to choose the best area of the face, we propose utilizing a wrapper feature selection technique, which is a real motivation in the field of computer vision and machine learning. The paper's primary purpose is to automatically determine the optimal face area using AOA to recognize the gender of a human person categorized by two classes (Men and women). In this paper, the facial image is divided into several subregions (blocks), where each area provides a vector of characteristics using one method from handcrafted techniques as the local binary pattern (LBP), histogram-oriented gradient (HOG), or gray-level co-occurrence matrix (GLCM). Two experiments assess the proposed method (AOA): The first employs two benchmarking datasets: the Georgia Tech Face dataset (GT) and the Brazilian FEI dataset. The second experiment represents a more challenging large dataset that uses Gallagher's uncontrolled dataset. The experimental results show the good performance of AOA compared to other recent and competitive optimizers for all datasets. In terms of accuracy, the AOA-based LBP outperforms the state-of-the-art deep convolutional neural network (CNN) with 96.08% for the Gallagher's dataset.
CL3D: Camera-LiDAR 3D Object Detection With Point Feature Enhancement and Point-Guided Fusion

Camera-LiDAR 3D object detection has been extensively investigated due to its significance for many real-world applications. However, there are still of great challenges to address the intrinsic data difference and perform accurate feature fusion among two modalities. To these ends, we propose a two-stream architecture termed as CL3D, that integrates with point enhancement module, point-guided fusion module with IoU-aware head for cross-modal 3D object detection. Specifically, pseudo LiDAR is firstly generated from RGB image, and point enhancement module (PEM) is then designed to enhance the raw LiDAR with pseudo point. Moreover, point-guided fusion module (PFM) is developed to find image-point correspondence at different resolutions, and incorporate semantic with geometric features in a point-wise manner. We also investigate the inconsistency between localization confidence and classification score in 3D detection, and introduce IoU-aware prediction head (IoU Head) for accurate box regression. Comprehensive experiments are conducted on publicly available KITTI dataset, and CL3D reports the outstanding detection performance compared to both single- and multi-modal 3D detectors, demonstrating its effectiveness and competitiveness.
Graph-based sparse bayesian broad learning system for semi-supervised learning

During the past decades, semi-supervised learning in classification has been regarded as one of the most active research area due to the increasing physical demand. Generally, the semi-supervised learning model believes the unlabeled data could potential be helpful to achieve higher performance as long as scare labeled samples under either cluster assumption or manifold assumption. However, most of the semi-supervised classifiers directly incorporate all the unlabeled data without any selective admission, which contains unfavorable features and noise diminishing performance while resulting in inability to large-scale data. In this paper, we propose a graph-based semi-supervised learning algorithm named GSB2LS within Bayesian framework for classification. The algorithm can explore unlabeled data effectively by adopting the compound prior that consists of unlabeled manifold information and sparse Bayesian inference to the broad structure. In particular, GSB2LS takes advantage of the broad structure to search for more potential associations of features, the manifold regularization to capture beneficial interdependence of unlabeled samples, the Bayesian framework to maintain the universal sparsity, the fast marginal likelihood maximization to update the relevance set based on the defined contribution, which leads to the feasibility to process large-scale data in the inductive way. Moreover, the algorithm is capable of outputting the probabilistic estimation of prediction for further decision analysis. Extensive empirical results verifies the excellent performance of our algorithm with clearly superior efficiency and generalization compared to other state-of-the-art semi-supervised classifiers.
Forecasting of short-term photovoltaic power generation using combined interval type-2 Takagi-Sugeno-Kang fuzzy systems

For the purpose of environmental protection and economic development, photovoltaic (PV) power generation is becoming increasingly popular, but the intermittence and uncertainty in PV power generation make grid-connected PV systems face severe challenges, accurate forecast is an effective way to solve this problem. Motived by the expectancy to process uncertainty and improve the prediction accuracy, in this study, a new prediction model combines interval type-2 Takagi-Sugeno-Kang (TSK) fuzzy neural network (type-2 TSKFNN) model optimized by extended Kalman filter (EKF) and SOM is proposed. A possible clustering interval is first determined according to the distribution of PV power generation, then the meteorological data are clustered using SOM, and the optimal size of categories is determined by Davies-Bouldin index (DBI). In each data partition, the decoupled extended Kalman filter (DKEF) is used to optimize the parameters of type-2 TSKFNN, including two Gaussian models with uncertain mean and uncertain standard deviation, and the independent fuzzy prediction models are established. The actual prediction analysis verifies the performance of proposed SOM-EKTSK model by using the data collected from Yulara Solar System in Australia, and the comparisons with other fuzzy logic system (FLS) models are also completed. The results show that the proposed SOM-EKTSK model has the highest prediction accuracy. Compared with other contrast models, the root mean square error (RMSE) of forecasts is reduced by 16.47% in spring, 4.75% in summer, 23.15% in autumn and 44.36% in winter, on average. © 2017 Elsevier Inc. All rights reserved.
Sources of Risk of AI Systems

Artificial intelligence can be used to realise new types of protective devices and assistance systems, so their importance for occupational safety and health is continuously increasing. However, established risk mitigation measures in software development are only partially suitable for applications in AI systems, which only create new sources of risk. Risk management for systems that for systems using AI must therefore be adapted to the new problems. This work objects to contribute hereto by identifying relevant sources of risk for AI systems. For this purpose, the differences between AI systems, especially those based on modern machine learning methods, and classical software were analysed, and the current research fields of trustworthy AI were evaluated. On this basis, a taxonomy could be created that provides an overview of various AI-specific sources of risk. These new sources of risk should be taken into account in the overall risk assessment of a system based on AI technologies, examined for their criticality and managed accordingly at an early stage to prevent a later system failure.