Reviewerly/exHarmony huggingface dataset:
Domain-Private Factor Detachment Network for NIR-VIS Face Recognition

Near-InfraRed and VISual (NIR-VIS) face matching, as one of the most representative tasks in Heterogeneous Face Recognition (HFR), aims at retrieving a face image across different domains. With the development of deep learning and the growing demand for intelligent surveillance, it has aroused more and more research attention in the computer vision community. However, due to the dramatic modality gap between NIR and VIS images, the task of NIR-VIS face recognition becomes practically very challenging. In this paper, we propose a novel Domain-private Factor Detachment (DFD) network to disentangle domain-dependent factors and achieve identity information distillation. Our approach consists of three key components, including Domain-identity Representation Learning (DiRL), Cross-domain Factor Detachment (CdFD) and Cross-domain Aggregation Learning (CAL). Firstly, the proposed DiRL aims to achieve domain-specific information distillation and learn identity-related representations. Specifically, three sub-networks, i.e., NIR sub-Network (NIR-Net), VIS sub-Network (VIS-Net) and IDentity-dependent sub-Network (ID-Net) are designed to learn NIR facial representations, VIS facial representations and identity-dependent representations, respectively, and they can promote each other to facilitate the learning of identity-discriminative representations. Secondly, considering that the entangled modal components in face representations negatively affect the subsequent matching process, to reduce modality-related components, we model the cross-modal face matching problem into three parts, comprising Identity Variation (IV), Inter-Spectrum Variation (ISV) and Identity-Domain Variation (IDV). The CdFD is presented to eliminate ISV components and IDV components by introducing inter-spectrum invariant constraint and identity-domain invariant constraint, so that cross-modal face recognition can be performed under pure identity information differences without modal interference. Finally, the CAL is developed to learn modality-invariant yet discriminative representations by exploring within-class aggregation, negative pair separability and cross-domain positive pair compactness. Experimental results on multiple challenging databases demonstrate the effectiveness of the DFD approach.
MC-GCN: A Multi-Scale Contrastive Graph Convolutional Network for Unconstrained Face Recognition With Image Sets

In this paper, a Multi-scale Contrastive Graph Convolutional Network (MC-GCN) method is proposed for unconstrained face recognition with image sets, which takes a set of media (orderless images and videos) as a face subject instead of single media (an image or video). Due to factors such as illumination, posture, media source, etc., there are huge intra-set variances in a face set, and the importance of different face prototypes varies considerably. How to model the attention mechanism according to the relationship between prototypes or images in a set is the main content of this paper. In this work, we formulate a framework based on graph convolutional network (GCN), which considers face prototypes as nodes to build relations. Specifically, we first present a multi-scale graph module to learn the relationship between prototypes at multiple scales. Moreover, a Contrastive Graph Convolutional (CGC) block is introduced to build attention control model, which focuses on those frames with similar prototypes (contrastive information) between pair of sets instead of simply evaluating the frame quality. The experiments on IJB-A, YouTube Face, and an animal face dataset clearly demonstrate that our proposed MC-GCN outperforms the state-of-the-art methods significantly.
Apache Hadoop based effective sentiment analysis on demonetization and covid-19 tweets

In Recent, Twitter is the well-known public Network acquires a huge number of tweets. Sentiment analysis in twitter data are tremendously valuable in social media observing as it allows getting an overview of extensive global opinion in certain issue. This data are utilized for industrial, government, social and economic approaches by analyzing the tweets as per the requirement of the user. Processing and storing these data are more complicated to analyze. Hadoop is a distributed environment which process with Big and Huge variety of dataset which supports processing components that collectively called Hadoop Ecosystem. In this paper, regular tweets are analyzed by sentiment analysis technique in Hadoop Eco system. Dataset are taken from Kaggle data repository. This research has done by Apache Pig in Demonetization and Covid 19 twitter dataset.
E-commerce platform based on Machine Learning Recommendation System

Information overload is one of the potential setbacks to many e-commerce platform users. It is very important to filter the media and the choices that are overwhelming for internet users while making buying decisions using online stores. To solve this problem, recommendation systems are used widely. A recommender system helps users find a product of their own choice by filtering and prioritizing and effectively generating the relevant information to its users. The purpose of a recommender system is to save time and hassle of searching through the World Wide Web, instead it generates specific and relevant content that promotes online transaction and bring satisfaction to the users of e-commerce platforms. The proposed system is an e-commerce platform based on an apparel recommendation system that recommends products on the foundation of the user's preferences.
A New Neural Distinguisher Considering Features Derived From Multiple Ciphertext Pairs

Abstract Neural-aided cryptanalysis is a challenging topic, in which the neural distinguisher ($\mathcal{ND}$) is a core module. In this paper, we propose a new $\mathcal{ND}$ considering multiple ciphertext pairs simultaneously. Besides, multiple ciphertext pairs are constructed from different keys. The motivation is that the distinguishing accuracy can be improved by exploiting features derived from multiple ciphertext pairs. To verify this motivation, we have applied this new $\mathcal{ND}$ to five different ciphers. Experiments show that taking multiple ciphertext pairs as input indeed brings accuracy improvement. Then, we prove that our new $\mathcal{ND}$ applies to two different neural-aided key recovery attacks. Moreover, the accuracy improvement is helpful for reducing the data complexity of the neural-aided statistic attack. The code is available at https://github.com/AI-Lab-Y/ND_mc.
HarMI: Human Activity Recognition Via Multi-Modality Incremental Learning

Nowadays, with the development of various kinds of sensors in smartphones or wearable devices, human activity recognition (HAR) has been widely researched and has numerous applications in healthcare, smart city, etc. Many techniques based on hand-crafted feature engineering or deep neural network have been proposed for sensor based HAR. However, these existing methods usually recognize activities offline, which means the whole data should be collected before training, occupying large-capacity storage space. Moreover, once the offline model training finished, the trained model can't recognize new activities unless retraining from the start, thus with a high cost of time and space. In this paper, we propose a multi-modality incremental learning model, called HarMI, with continuous learning ability. The proposed HarMI model can start training quickly with little storage space and easily learn new activities without storing previous training data. In detail, we first adopt attention mechanism to align heterogeneous sensor data with different frequencies. In addition, to overcome catastrophic forgetting in incremental learning, HarMI utilizes the elastic weight consolidation and canonical correlation analysis from a multi-modality perspective. Extensive experiments based on two public datasets demonstrate that HarMI can achieve a superior performance compared with several state-of-the-arts.
Multi-Granularity Mutual Learning Network for Object Re-Identification

Object re-identification (re-ID), which is key and fundamental technology for intelligent transportation systems, is a challenging task including person re-ID and vehicle re-ID. It aims to retrieve a given target object from the gallery images captured by different cameras. In this task, it is necessary to extract fine-grained and discriminative features to deal with complex inter-class and intra-class variations caused by the changes of camera viewpoints and object poses. Existing methods focus on learning discriminative local features to improve the re-ID performance. Some state-of-the-art methods use key point detection model to locate local features, which also increases the additional computational cost as side effect. Another type of method focuses on how to learn features of different granularity from rigid stripes of different scales. However, there is little attention paid to how to effectively coalesce multi-granularity features without additional calculation cost. To tackle this issue, this paper proposes the Multi-granularity Mutual Learning Network (MMNet) and makes two contributions. 1) We introduce the multi-granularity jigsaw puzzle module into object re-ID to impel the network to learn local discriminative features from multiple visual granularities by breaking spatial correlation in original images. 2) We propose a parameter-free multi-scale feature reconstruction module to facilitate mutual learning of features at multiple grain levels, thereby both global features and local features have strong representation capabilities. Extensive experiments demonstrate the effectiveness of our proposed modules and the superiority of our method over various state-of-the-art methods on both person and vehicle re-ID benchmarks.
The heterogeneous Aquila optimization algorithm

A new swarm-based optimization algorithm called the Aquila optimizer (AO) was just proposed recently with promising better performance. However, as reported by the proposer, it almost remains unchanged for almost half of the convergence curves at the latter iterations. Considering the better performance and the lazy latter convergence rates of the AO algorithm in optimization, the multiple updating principle is introduced and the heterogeneous AO called HAO is proposed in this paper. Simulation experiments were carried out on both unimodal and multimodal benchmark functions, and comparison with other capable algorithms were also made, most of the results confirmed the better performance with better intensification and diversification capabilities, fast convergence rate, low residual errors, strong scalabilities, and convinced verification results. Further application in optimizing three benchmark real-world engineering problems were also carried out, the overall better performance in optimizing was confirmed without any other equations introduced for improvement.
Down-Sized Initialization for Optimization-Based Unstructured Trajectory Planning by Only Optimizing Critical Variables

Optimization-based trajectory planners for an autonomous vehicle have received increasing attention due to their convenience for handling various complex constraints. However, their formulated trajectory planning optimization problems in unstructured environments are highly non-convex and nonlinear, requiring a proper initial guess to help with convergence. Given the low efficiency of common sampling-and-search-based initialization strategies due to the dimensionality brought by complex vehicle dynamics and the maneuverable space narrowed by cluttered obstacles, the sequential optimization of a series of softened intermediate problems is emerging as a new initialization strategy. Unlike sequential optimization-based strategies, which optimize all decision variables (w.r.t. the positions, velocities, vehicle orientations, steering angles, and so on of all waypoints) for initialization, the proposed strategy only optimizes critical variables, whose values should help satisfy the inequalities strictly with a small equation violation. Thus, the original scheme for initialization is decoupled into two sub-tasks with down-sized variables. One reduces the nonlinear equation violation, and the other ensures strict inequality satisfaction. Experimental results show that the proposed initialization strategy can facilitate the solution of an optimization-based trajectory planner, thereby making the trajectory planner outperform the state-of-the-art competitors in this community.
Deep Learning-Based Sentiment Analysis on COVID-19 News Videos

Coronavirus disease (COVID-19) has adversely affected all walks of human life. The whole world is confronting this deadly virus, and no country in this world remains untouched during this pandemic. There are several online news videos related to COVID-19 that are shared on various online platforms such as YouTube, DailyMotion, and Vimeo. There were several arguments on the genuineness of the contents, people watch them, share them, and most importantly express their views and opinions as comments on those platforms. Analyzing these comments can unearth the patterns hidden in them to study people's responses to videos on COVID-19. This paper proposes a deep learning-based sentiment analysis approach to people's response toward online COVID-19 video news. This work implements different deep learning approaches such as LSTM, Bi-LSTM, CNN, and GRU to classify sentiment from the comments collected from YouTube.
Consistency-Induced Multiview Subspace Clustering

Multiview clustering has received great attention and numerous subspace clustering algorithms for multiview data have been presented. However, most of these algorithms do not effectively handle high-dimensional data and fail to exploit consistency for the number of the connected components in similarity matrices for different views. In this article, we propose a novel consistency-induced multiview subspace clustering (CiMSC) to tackle these issues, which is mainly composed of structural consistency (SC) and sample assignment consistency (SAC). To be specific, SC aims to learn a similarity matrix for each single view wherein the number of connected components equals to the cluster number of the dataset. SAC aims to minimize the discrepancy for the number of connected components in similarity matrices from different views based on the SAC assumption, that is, different views should produce the same number of connected components in similarity matrices. CiMSC also formulates cluster indicator matrices for different views, and shared similarity matrices simultaneously in an optimization framework. Since each column of similarity matrix can be used as a new representation of the data point, CiMSC can learn an effective subspace representation for the high-dimensional data, which is encoded into the latent representation by reconstruction in a nonlinear manner. We employ an alternating optimization scheme to solve the optimization problem. Experiments validate the advantage of CiMSC over 12 state-of-the-art multiview clustering approaches, for example, the accuracy of CiMSC is 98.06% on the BBCSport dataset.
Lifelong Learning with Monolithic 3D Ferroelectric Ternary Content-Addressable Memory

Lifelong learning at the edge requires on-the-fly learning from scarce data in one or few shots. Here, we present the array-level demonstration of few-shot learning using a first time fabricated monolithic 3D ternary content addressable memory (M3D-TCAM) using back-end-of-line (BEOL) ferroelectric FETs (FeFETs). The fabricated two-tier structure consists of two 10×10 sub-arrays in each tier and allows massively parallel search operation up to 20-bit long search vectors. We experimentally demonstrate: (a) record low write voltage of ± 1.6V with 20ns write latency for BEOL FeFETs in M3D-TCAM arrays, (b) in situ computation of Hamming distance (nearest neighbor match) between a 20-bit search vector and ten different stored vectors, (c) disturb-free write operation, and (d) high write endurance exceeding 10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">10</sup> cycles. We experimentally demonstrate a 3-way 3-shot learning with 20-bit feature vectors using Omniglot dataset and achieve an inference accuracy of 70% comparable to GPU accuracy of 72%. System-level benchmarking performed on a 64×512 M3D TCAM with 8 vertically stacked sub-arrays exhibit a 3.5x, 3.7x, 3.5x and 12x improvement in area, search energy, write energy and write latency, respectively, over 2D TCAM.
Intelligence Networking for Autonomous Driving in Beyond 5G Networks With Multi-Access Edge Computing

Artificial intelligence (AI)-powered autonomous vehicles (AVs) can integrate different machine learning (ML) techniques to build up a complex autonomous driving system. However, single AV intelligence is not enough to cope with ever-changing driving environments. The underlying reason is that, with current neural network design and training algorithms, it is challenging for the driving model to generalize to diverse driving environments all at once due to sample inefficiency and the curse of dimensionality. Powerful computing resources and massive amount of data can be used to train a good driving model offline. However, the driving model obtained offline might fail in corner case scenarios. In this paper, we propose an intelligence networking framework among AVs assisted by multi-access edge computing (MEC) with end-to-end learning for demonstration. In this framework, driving road is divided into segments and data is collected for each road segment separately. Assisted by MEC networks, a continuously updated driving model is produced in near real-time for each road segment when the environment changes. By dividing the road into segments, we aim to reduce the burden of generalization since a single model only needs to adapt to a specific road segment. Simulation results show that our solutions can produce updated driving model for each road segment to adapt to environmental changes better than the existing scheme. Upcoming AVs can then adapt to changing environments by downloading the updated driving model.
Grouping by Center: Predicting Centripetal Offsets for the Bottom-up Human Pose Estimation

We introduce Grouping by Center, a novel grouping approach for the bottom-up human pose estimation, which detects human joint first and then does grouping. The grouping strategy is the critical factor for the bottom-up pose estimation. To increase the conciseness and accuracy, we propose to use the center of the body as a grouping clue. More concretely, we predict the offsets from the keypoints to the body centers. Keypoints with aligned shifted results will be grouped as one person. However, the multi-scale variance of people can affect the prediction of the grouping clue, which has been neglected in previous research. To resolve the scale variance of the offset, we put forward a Multi-scale Translation Layer and an iterative refinement. Furthermore, we scheme a greedy grouping strategy with a dynamic threshold due to the various scales of instances. Through a comprehensive comparison, our framework is validated to be effective and practical. We also lay out the state-of-the-art performance revolving the bottom-up multi-person pose estimation on the MS-COCO dataset and the CrowdPose dataset.
From implicit to explicit feedback: A deep neural network for modeling sequential behaviours and long-short term preferences of online users

In this work, we examine the advantages of using multiple types of behaviours in recommendation systems. Intuitively, each user often takes some implicit actions (e.g., click) before making an explicit decision (e.g., purchase). Previous studies show that implicit and explicit feedback has different roles for a useful recommendation. However, these studies either exploit implicit and explicit behaviours separately or ignore the semantics of sequential interactions between users and items. In addition, we go from the hypothesis that a user's preferences at a time are combinations of long-term and short-term interests. In this paper, we propose some Deep Learning architectures. The first one is Implicit to Explicit (ITE), to exploit users' interests through the sequence of their actions. The second and third ones are two versions of ITE with Bidirectional Encoder Representations from Transformers based (BERT-based) architecture called BERT-ITE and BERT-ITE-Si, which combine users' long- and short-term preferences without and with side information to enhance users' representations. The experimental results show that our models outperform previous state-of-the-art ones and also demonstrate our views on the effectiveness of exploiting the implicit to explicit order as well as combining long- and short-term preferences in three large-scale datasets. The source code of our paper is available at: https://github.com/tranquyenbk173/BERT_ITE.
FAITH: A Fast Blockchain-Assisted Edge Computing Platform for Healthcare Applications

The Internet of Medical Things is developing rapidly in recent years. However, the timeliness and security of healthcare applications challenge its adoption. In this article, we propose a blockchain-assisted edge computing platform that timely and securely processes time-sensitive healthcare applications. We propose a blockchain-assisted framework that leverages distributed edge servers to achieve fast data processing. We design smart contracts to verify the identity and data credibility of network entities. We formulate the problem as a directed acyclic graph organized scheduling model and develop online orchestrating algorithms to meet the timeliness requirement. We implement the blockchain prototype and evaluate the performance of the proposed algorithm under extensive configurations. Results show that fast blockchain-assisted edge computing platform for healthcare achieves a significant timeliness guarantee, and at the same time outperforms conventional schemes from the latency perspective.
Exploiting Non-Local Priors via Self-Convolution for Highly-Efficient Image Restoration

Constructing effective priors is critical to solving ill-posed inverse problems in image processing and computational imaging. Recent works focused on exploiting non-local similarity by grouping similar patches for image modeling, and demonstrated state-of-the-art results in many image restoration applications. However, compared to classic methods based on filtering or sparsity, non-local algorithms are more time-consuming, mainly due to the highly inefficient block matching step, i.e., distance between every pair of overlapping patches needs to be computed. In this work, we propose a novel Self-Convolution operator to exploit image non-local properties in a unified framework. We prove that the proposed Self-Convolution based formulation can generalize the commonly-used non-local modeling methods, as well as produce results equivalent to standard methods, but with much cheaper computation. Furthermore, by applying Self-Convolution, we propose an effective multi-modality image restoration scheme, which is much more efficient than conventional block matching for non-local modeling. Experimental results demonstrate that (1) Self-Convolution with fast Fourier transform implementation can significantly speed up most of the popular non-local image restoration algorithms, with two-fold to nine-fold faster block matching, and (2) the proposed online multi-modality image restoration scheme achieves superior denoising results than competing methods in both efficiency and effectiveness on RGB-NIR images. The code for this work is publicly available at https://github.com/GuoLanqing/Self-Convolution.