Reviewerly/exHarmony huggingface dataset:
Group-Aware Long- and Short-Term Graph Representation Learning for Sequential Group Recommendation

Sequential recommendation and group recommendation are two important branches in the field of recommender system. While considerable efforts have been devoted to these two branches in an independent way, we combine them by proposing the novel sequential group recommendation problem which enables modeling group dynamic representations and is crucial for achieving better group recommendation performance. The major challenge of the problem is how to effectively learn dynamic group representations based on the sequential user-item interactions of group members in the past time frames. To address this, we devise a Group-aware Long- and Short-term Graph Representation Learning approach, namely GLS-GRL, for sequential group recommendation. Specifically, for a target group, we construct a group-aware long-term graph to capture user-item interactions and item-item co-occurrence in the whole history, and a group-aware short-term graph to contain the same information regarding only the current time frame. Based on the graphs, GLS-GRL performs graph representation learning to obtain long-term and short-term user representations, and further adaptively fuse them to gain integrated user representations. Finally, group representations are obtained by a constrained user-interacted attention mechanism which encodes the correlations between group members. Comprehensive experiments demonstrate that GLS-GRL achieves better performance than several strong alternatives coming from sequential recommendation and group recommendation methods, validating the effectiveness of the core components in GLS-GRL.
A novel technique for automated concealed face detection in surveillance videos

Face detection perceives great importance in surveillance paradigm and security paradigm areas. Face recognition is the technique to identify a person identity after face detection. Extensive research has been done on these topics. Another important research problem is to detect concealed faces, especially in high-security places like airports or crowded places like concerts and shopping centres, for they may prevail security threat. Also, in order to help effectively in preventing the spread of Coronavirus, people should wear masks during the pandemic especially in the entrance to hospitals and medical facilities. Surveillance systems in medical facilities should issue warnings against unmasked people. This paper presents a novel technique for concealed face detection based on complexion detection to challenge a concealed face assumption. The proposed algorithm first determine of the existence of a human being in the surveillance scene. Head and shoulder contour will be detected. The face will be clustered to cluster patches. Then determination of presence or absent of human skin will be determined. We proposed a hybrid approach that combines normalized RGB (rgb) and the YCbCr space color. This technique is tested on two datasets; the first one contains 650 images of skin patches. The second dataset contains 800 face images. The algorithm achieves an average detection rate of 97.51% for concealed faces. Also, it achieved a run time comparable with existing state-of-the-art concealed face detection systems that run in real time.
Exploiting Joint Robustness to Adversarial Perturbations

Recently, ensemble models have demonstrated empirical capabilities to alleviate the adversarial vulnerability. In this paper, we exploit first-order interactions within ensembles to formalize a reliable and practical defense. We introduce a scenario of interactions that certifiably improves the robustness according to the size of the ensemble, the diversity of the gradient directions, and the balance of the member's contribution to the robustness. We present a joint gradient phase and magnitude regularization (GPMR) as a vigorous approach to impose the desired scenario of interactions among members of the ensemble. Through extensive experiments, including gradient-based and gradient-free evaluations on several datasets and network architectures, we validate the practical effectiveness of the proposed approach compared to the previous methods. Furthermore, we demonstrate that GPMR is orthogonal to other defense strategies developed for single classifiers and their combination can further improve the robustness of ensembles.
How Much Time Do You Have? Modeling Multi-Duration Saliency

What jumps out in a single glance of an image is different than what you might notice after closer inspection. Yet conventional models of visual saliency produce predictions at an arbitrary, fixed viewing duration, offering a limited view of the rich interactions between image content and gaze location. In this paper we propose to capture gaze as a series of snapshots, by generating population-level saliency heatmaps for multiple viewing durations. We collect the CodeCharts1K dataset, which contains multiple distinct heatmaps per image corresponding to 0.5, 3, and 5 seconds of free-viewing. We develop an LSTM-based model of saliency that simultaneously trains on data from multiple viewing durations. Our Multi-Duration Saliency Excited Model (MD-SEM) achieves competitive performance on the LSUN 2017 Challenge with 57% fewer parameters than comparable architectures. It is the first model that produces heatmaps at multiple viewing durations, enabling applications where multi-duration saliency can be used to prioritize visual content to keep, transmit, and render.
MaHRL

As huge commercial value of the recommender system, there has been growing interest to improve its performance in recent years. The majority of existing methods have achieved great improvement on the metric of click, but perform poorly on the metric of conversion possibly due to its extremely sparse feedback signal. To track this challenge, we design a novel deep hierarchical reinforcement learning based recommendation framework to model consumers' hierarchical purchase interest. Specifically, the high-level agent catches long-term sparse conversion interest, and automatically sets abstract goals for low-level agent, while the low-level agent follows the abstract goals and catches short-term click interest via interacting with real-time environment. To solve the inherent problem in hierarchical reinforcement learning, we propose a novel multi-goals abstraction based deep hierarchical reinforcement learning algorithm (MaHRL). Our proposed algorithm contains three contributions: 1) the high-level agent generates multiple goals to guide the low-level agent in different sub-periods, which reduces the difficulty of approaching high-level goals; 2) different goals share the same state encoder structure and its parameters, which increases the update frequency of the high-level agent and thus accelerates the convergence of our proposed algorithm; 3) an appreciated reward assignment mechanism is designed to allocate rewards in each goal so as to coordinate different goals in a consistent direction. We evaluate our proposed algorithm based on a real-world e-commerce dataset and validate its effectiveness.
All-optical non-linear activation function for neuromorphic photonic computing using semiconductor Fano lasers

We predict that semiconductor Fano lasers can be used to realize an all-optical non-linear activation function for neuromorphic photonic computing. By exploiting optical control of a Fano mirror, the laser can generate optical pulses with low threshold energy, gigahertz repetition rates, and orders of magnitude suppression between the on- and off-states. Analytical estimates of the switching threshold energy, extinction ratio, and refractory period agree well with numerical results.
AnimalWeb: A Large-Scale Hierarchical Dataset of Annotated Animal Faces

Several studies show that animal needs are often expressed through their faces. Though remarkable progress has been made towards the automatic understanding of human faces, this has not been the case with animal faces. There exists significant room for algorithmic advances that could realize automatic systems for interpreting animal faces. Besides scientific value, resulting technology will foster better and cheaper animal care. We believe the underlying research progress is mainly obstructed by the lack of an adequately annotated dataset of animal faces, covering a wide spectrum of animal species. To this end, we introduce a large-scale, hierarchical annotated dataset of animal faces, featuring 22.4K faces from 350 diverse species and 21 animal orders across biological taxonomy. These faces are captured `in-the-wild' conditions and are consistently annotated with 9 landmarks on key facial features. The dataset is structured and scalable by design; its development underwent four systematic stages involving rigorous, overall effort of over 6K man-hours. We benchmark it for face alignment using the existing art under two new problem settings. Results showcase its challenging nature, unique attributes and present definite prospects for novel, adaptive, and generalized face-oriented CV algorithms. Further benchmarking the dataset across face detection and fine-grained recognition tasks demonstrates its multi-task applications and room for improvement. The dataset is available at: https://fdmaproject.wordpress.com/.
Noise-Aware Fully Webly Supervised Object Detection

We investigate the emerging task of learning object detectors with sole image-level labels on the web without requiring any other supervision like precise annotations or additional images from well-annotated benchmark datasets. Such a task, termed as fully webly supervised object detection, is extremely challenging, since image-level labels on the web are always noisy, leading to poor performance of the learned detectors. In this work, we propose an end-to-end framework to jointly learn webly supervised detectors and reduce the negative impact of noisy labels. Such noise is heterogeneous, which is further categorized into two types, namely background noise and foreground noise. Regarding the background noise, we propose a residual learning structure incorporated with weakly supervised detection, which decomposes background noise and models clean data. To explicitly learn the residual feature between clean data and noisy labels, we further propose a spatially-sensitive entropy criterion, which exploits the conditional distribution of detection results to estimate the confidence of background categories being noise. Regarding the foreground noise, a bagging-mixup learning is introduced, which suppresses foreground noisy signals from incorrectly labelled images, whilst maintaining the diversity of training data. We evaluate the proposed approach on popular benchmark datasets by training detectors on web images, which are retrieved by the corresponding category tags from photo-sharing sites. Extensive experiments show that our method achieves significant improvements over the state-of-the-art methods.
Path planning for indoor Mobile robot based on deep learning

Abstract This paper aims to give an optimal path planning of a mobile robot in a known indoor environment. An algorithm based on deep learning, ray tracing algorithm, waiting rule, and Rapidly-exploring Random Tree is proposed to solve the problem of obstacle avoidance and path planning. Firstly, GoogLeNet is used to classify obstacles. Here, it helps to distinguish between static and dynamic obstacles. Secondly, for the static obstacle avoidance, the ray tracing algorithm is proposed to avoid the obstacles which are identified by GoogLeNet. And for the dynamic obstacle avoidance, this paper proposed a waiting rule for dynamic obstacle avoidance. Thirdly, the RRT method plans a path from the start point to the goal point. The novelty of this paper is that the type of obstacles is distinguished by deep learning, and the two different kinds of obstacles used ray tracing algorithm and waiting rule to avoid obstacles collision, respectively. In experimental results, rapidly-exploring random tree is compared with genetic algorithm, and particle swarm optimization method in static environment, and it is compared with artificial potential field approach in dynamic environment. Experiments are carried out in the two-dimensional environment and successfully applied to the path planning of mobile robots in multi-obstacles environment, and they prove the feasibility of the algorithm.
Attention-Based Network For Low-Light Image Enhancement

The captured images under low-light conditions often suffer insufficient brightness and notorious noise. Hence, low-light image enhancement is a key challenging task in computer vision. A variety of methods have been proposed for this task, but these methods often failed in an extreme low-light environment and amplified the underlying noise in the input image. To address such a difficult problem, this paper presents a novel attention-based neural network to generate high-quality enhanced low-light images from the raw sensor data. Specifically, we first employ attention strategy (i.e. spatial attention and channel attention modules) to suppress undesired chromatic aberration and noise. The spatial attention module focuses on denoising by taking advantage of the non-local correlation in the image. The channel attention module guides the network to refine redundant colour features. Furthermore, we propose a new pooling layer, called inverted shuffle layer, which adaptively selects useful information from previous features. Extensive experiments demonstrate the superiority of the proposed network in terms of suppressing the chromatic aberration and noise artifacts in enhancement, especially when the low-light image has severe noise.
Towards a PDE-based large-scale decentralized solution for path planning of UAVs in shared airspace

Recently, there has been a tremendous increase of interest in utilizing Unmanned Aerial Vehicles (UAVs) for a number of civilian applications. With this increased interest, it is imperative that these UAVs are able to operate in shared airspace for enhanced efficiency. Multi-UAV systems are inherently safety-critical systems, which means that safety guarantees must be made to ensure no undesirable configurations, such as collisions, occur. This paper proposes a decentralized method based on a Partial Differential Equation (PDE) to generate collision-free 3D trajectories for multiple UAVs operating in a shared airspace. This method exploits the dynamical properties of multi-phase fluids flowing through a porous medium by modeling the porosity values as a function of the risk of collision. To highlight the feasibility for on-board implementation, we propose propose a machine learning technique for obtaining computationally efficient solutions of the PDE describing flow movements in porous medium. This method has been compared via a simulation study to two other path planning strategies, centralized and sequential planning, and the advantages of this method are presented. Furthermore, results from an experiment using three UAVs have been presented to demonstrate the applicability of the proposed method to real-world implementation.
Gaussian-response correlation filter for robust visual object tracking

This paper presents a novel correlation filter-based tracking method for robust visual object tracking in the presence of partial occlusion, large-scale variation and model drift. To do this, first, we develop a correlation filter for predicting the target location based on the distribution of correlation response. In this formulation, the correlation response of the target image follows Gaussian distribution to estimate the target location efficiently. Second, the constraints are derived using kernel ridge regression to mitigate the target failure in object tracking. Third, we propose an adaptive scale estimation method to detect the target scale changes during the tracking. In addition, two feature integration is elaborately designed to improve the discriminative strength of the correlation filter. Finally, extensive experimental results on OTB2013, OTB2015, TempleColor128 and UAV123 datasets demonstrate that the proposed method performs favourably against several state-of-the-art methods.
McFlow: Monte Carlo Flow Models for Data Imputation

We consider the topic of data imputation, a foundational task in machine learning that addresses issues with missing data. To that end, we propose MCFlow, a deep framework for imputation that leverages normalizing flow generative models and Monte Carlo sampling. We address the causality dilemma that arises when training models with incomplete data by introducing an iterative learning scheme which alternately updates the density estimate and the values of the missing entries in the training data. We provide extensive empirical validation of the effectiveness of the proposed method on standard multivariate and image datasets, and benchmark its performance against state-of-the-art alternatives. We demonstrate that MCFlow is superior to competing methods in terms of the quality of the imputed data, as well as with regards to its ability to preserve the semantic structure of the data.
Robot-Centric Perception of Human Groups

The robotics community continually strives to create robots that are deployable in real-world environments. Often, robots are expected to interact with human groups. To achieve this goal, we introduce a new method, the Robot-Centric Group Estimation Model (RoboGEM), which enables robots to detect groups of people. Much of the work reported in the literature focuses on dyadic interactions, leaving a gap in our understanding of how to build robots that can effectively team with larger groups of people. Moreover, many current methods rely on exocentric vision, where cameras and sensors are placed externally in the environment, rather than onboard the robot. Consequently, these methods are impractical for robots in unstructured, human-centric environments, which are novel and unpredictable. Furthermore, the majority of work on group perception is supervised, which can inhibit performance in real-world settings. RoboGEM addresses these gaps by being able to predict social groups solely from an egocentric perspective using color and depth (RGB-D) data. To achieve group predictions, RoboGEM leverages joint motion and proximity estimations. We evaluated RoboGEM against a challenging, egocentric, real-world dataset where both pedestrians and the robot are in motion simultaneously, and show RoboGEM outperformed two state-of-the-art supervised methods in detection accuracy by up to 30%, with a lower miss rate. Our work will be helpful to the robotics community, and serve as a milestone to building unsupervised systems that will enable robots to work with human groups in real-world environments.
Robust nonnegative matrix factorization with structure regularization

Nonnegative matrix factorization (NMF) has attracted more and more attention due to its wide applications in computer vision, information retrieval, and machine learning. In contrast to the original NMF and its variants, this paper proposes a novel unsupervised learning framework, called robust structured nonnegative matrix factorization (RSNMF) which respects both global and local structures of the data space. Specifically, to learn a discriminative representation, RSNMF explores both the global structure via considering the data variance and the local structure via exploiting the data neighborhood. To well address the problem of noise and outliers, it imposes joint L2,1-norm minimization on both the loss function of NMF and the regularization of the basis matrix. The geometric structure and the joint L2,1-norm are formulated as an optimization model, which is solved by the proposed iterative algorithm. Finally, the convergence of RSNMF is analyzed theoretically and empirically. The experimental results on real-world data sets show the effectiveness of our proposed algorithm in comparison to state-of-the-art algorithms.
Spatio-temporal visual attention modelling of standard biometry plane-finding navigation

We present a novel multi-task neural network called Temporal SonoEyeNet (TSEN) with a primary task to describe the visual navigation process of sonographers by learning to generate visual attention maps of ultrasound images around standard biometry planes of the fetal abdomen, head (trans-ventricular plane) and femur. TSEN has three components: a feature extractor, a temporal attention module (TAM), and an auxiliary video classification module (VCM). A soft dynamic time warping (sDTW) loss function is used to improve visual attention modelling. Variants of the model are trained on a dataset of 280 video clips, each containing one of the three biometry planes and lasting 3–7 seconds, with corresponding real-time recorded gaze tracking data of an experienced sonographer. We report the performances of the different variants of TSEN for visual attention prediction at standard biometry plane detection. The best model performance is achieved using bi-directional convolutional long-short term memory (biCLSTM) in both TAM and VCM, and it outperforms a previous spatial model on all static and dynamic saliency metrics. As an auxiliary task to validate the clinical relevance of the visual attention modelling, the predicted visual attention maps were used to guide standard biometry plane detection in consecutive US video frames. All spatio-temporal TSEN models achieve higher scores compared to a spatial-only baseline; the best performing TSEN model achieves F1 scores on these standard biometry planes of 83.7%, 89.9% and 81.1%, respectively.
Trusted UAV Network Coverage Using Blockchain, Machine Learning, and Auction Mechanisms

The UAV is emerging as one of the greatest technology developments for rapid network coverage provisioning at affordable cost. The aim of this paper is to outsource network coverage of a specific area according to a desired quality of service requirement and to enable various entities in the network to have intelligence to make autonomous decisions using blockchain and auction mechanisms. In this regard, by considering a multiple-UAV network where each UAV is associated to its own controlling operator, this paper addresses two major challenges: the selection of the UAV for the desired quality of network coverage and the development of a distributed and autonomous real-time monitoring framework for the enforcement of service level agreement (SLA). For a suitable UAV selection, we employ a reputation-based auction mechanism to model the interaction between the business agent who is interested in outsourcing the network coverage and the UAV operators serving in closeby areas. In addition, theoretical analysis is performed to show that the proposed auction mechanism attains a dominant strategy equilibrium. For the SLA enforcement and trust model, we propose a permissioned blockchain architecture considering Support Vector Machine (SVM) for real-time autonomous and distributed monitoring of UAV service. In particular, smart contract features of the blockchain are invoked for enforcing the SLA terms of payment and penalty, and for quantifying the UAV service reputation. Simulation results confirm the accuracy of theoretical analysis and efficacy of the proposed model.