Reviewerly/exHarmony huggingface dataset:
MESNet: A Convolutional Neural Network for Spotting Multi-Scale Micro-Expression Intervals in Long Videos

Micro-expression spotting is a fundamental step in the micro-expression analysis. This paper proposes a novel network based convolutional neural network (CNN) for spotting multi-scale spontaneous micro-expression intervals in long videos. We named the network as Micro-Expression Spotting Network (MESNet). It is composed of three modules. The first module is a 2+1D Spatiotemporal Convolutional Network, which uses 2D convolution to extract spatial features and 1D convolution to extract temporal features. The second module is a Clip Proposal Network, which gives some proposed micro-expression clips. The last module is a Classification Regression Network, which classifies the proposed clips to micro-expression or not, and further regresses their temporal boundaries. We also propose a novel evaluation metric for spotting micro-expression. Extensive experiments have been conducted on the two long video datasets: CAS(ME) <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> and SAMM, and the leave-one-subject-out cross-validation is used to evaluate the spotting performance. Results show that the proposed MESNet effectively enhances the F1-score metric. And comparative results show the proposed MESNet has achieved a good performance, which outperforms other state-of-the-art methods, especially in the SAMM dataset.
ResNeXt and Res2Net Structures for Speaker Verification

The ResNet-based architecture has been widely adopted to extract speaker embeddings for text-independent speaker verification systems. By introducing the residual connections to the CNN and standardizing the residual blocks, the ResNet structure is capable of training deep networks to achieve highly competitive recognition performance. However, when the input feature space becomes more complicated, simply increasing the depth and width <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> of the ResNet network may not fully realize its performance potential. In this paper, we present two extensions of the ResNet architecture, ResNeXt and Res2Net, for speaker verification. Originally proposed for image recognition, the ResNeXt and Res2Net introduce two more dimensions, cardinality and scale, in addition to depth and width, to improve the model's representation capacity. By increasing the scale dimension, the Res2Net model can represent multi-scale features with various granularities, which particularly facilitates speaker verification for short utterances. We evaluate our proposed systems on three speaker verification tasks. Experiments on the VoxCeleb test set demonstrated that the ResNeXt and Res2Net can significantly outperform the conventional ResNet model. The Res2Net model achieved superior performance by reducing the EER by 18.5% relative. Experiments on the other two internal test sets of mismatched conditions further confirmed the generalization of the ResNeXt and Res2Net architectures against noisy environment and segment length variations.
Embracing Four Tensions in Human-Computer Interaction Research with Marginalized People

Human-computer interaction has a long history of working with marginalized people. We sought to understand how HCI researchers navigate work that engages with marginalized people and considerations researchers might work through to expand benefits and mitigate potential harms. In total, 24 HCI researchers, located primarily in the United States, participated in an interview, survey, or both. Through a reflexive thematic analysis, we identified four tensions—exploitation, membership, disclosure, and allyship. We explore the complexity involved in each, demonstrating that an equitable endpoint may not be possible, but this work is still worth pursuing when researchers make certain considerations. We emphasize that researchers who work with marginalized people should account for each tension in their research approaches to move forward. Finally, we propose an allyship-oriented approach to research that draws inspiration from discourse occurring in tangential fields and activist spaces and pushes the field into a new paradigm of research with marginalized people.
Feature selection with kernelized multi-class support vector machine

Feature selection is an important procedure in machine learning because it can reduce the complexity of the final learning model and simplify the interpretation. In this paper, we propose a novel non-linear feature selection method that targets multi-class classification problems in the framework of support vector machines. The proposed method is achieved using a kernelized multi-class support vector machine with a fast version of recursive feature elimination. The proposed method selects features that work well for all classes, as the involved classifier simultaneously constructs multiple decision functions that separates each class from the others. We formulate the classifier as a large optimisation problem, and iteratively solve one decision function at a time, leading to a lower computational time complexity than when solving the large optimisation problem directly. The coefficients of the classifier are then used as a ranking criterion in the accelerated recursive feature elimination by adding batch elimination and a rechecking process. Experimental results on several datasets demonstrate the superior performance of the proposed feature selection method.
Unsupervised anomaly detection with LSTM autoencoders using statistical data-filtering

To address one of the most challenging industry problems, we develop an enhanced training algorithm for anomaly detection in unlabelled sequential data such as time-series. We propose the outputs of a well-designed system are drawn from an unknown probability distribution, U, in normal conditions. We introduce a probability criterion based on the classical central limit theorem that allows evaluation of the likelihood that a data-point is drawn from U. This enables the labelling of the data on the fly. Non-anomalous data is passed to train a deep Long Short-Term Memory (LSTM) autoencoder that distinguishes anomalies when the reconstruction error exceeds a threshold. To illustrate our algorithm’s efficacy, we consider two real industrial case studies where gradually-developing and abrupt anomalies occur. Moreover, we compare our algorithm’s performance with four of the recent and widely used algorithms in the domain. We show that our algorithm achieves considerably better results in that it timely detects anomalies while others either miss or lag in doing so.
SiamCorners: Siamese Corner Networks for Visual Tracking

The current Siamese network based on region proposal network (RPN) has attracted great attention in visual tracking due to its excellent accuracy and high efficiency. However, the design of the RPN involves the selection of the number, scale, and aspect ratios of anchor boxes, which will affect the applicability and convenience of the model. Furthermore, these anchor boxes require complicated calculations, such as calculating their intersection-over-union (IoU) with ground truth bounding boxes. Due to the problems related to anchor boxes, we propose a simple yet effective anchor-free tracker (named Siamese corner networks, SiamCorners), which is end-to-end trained offline on large-scale image pairs. Specifically, we introduce a modified corner pooling layer to convert the bounding box estimate of the target into a pair of corner predictions (the bottom-right and the top-left corners). By tracking a target as a pair of corners, we avoid the need to design the anchor boxes. This will make the entire tracking algorithm more flexible and simple than anchor-based trackers. In our network design, we further introduce a layer-wise feature aggregation strategy that enables the corner pooling module to predict multiple corners for a tracking target in deep networks. We then introduce a new penalty term that is used to select an optimal tracking box in these candidate corners. Finally, SiamCorners achieves experimental results that are comparable to the state-of-art tracker while maintaining a high running speed. In particular, SiamCorners achieves a 53.7&#x0025; AUC on NFS30 and a 61.4&#x0025; AUC on UAV123, while still running at 42 frames per second (FPS).
Self-Training and Pre-Training are Complementary for Speech Recognition

Self-training and unsupervised pre-training have emerged as effective approaches to improve speech recognition systems using unlabeled data. However, it is not clear whether they learn similar patterns or if they can be effectively combined. In this paper, we show that pseudo-labeling and pre-training with wav2vec 2.0 are complementary in a variety of labeled data setups. Using just 10 minutes of labeled data from Libri-light as well as 53k hours of unlabeled data from LibriVox achieves word error rates (WER) of 2.8%/4.8% on the clean and other test sets of Librispeech – rivaling the best published systems trained on 960 hours of labeled data only a year ago. Training on all labeled data of Librispeech achieves WERs of 1.5%/3.1%.
Ultimately Bounded Filtering Subject to Impulsive Measurement Outliers

This article is concerned with the ultimately bounded filtering problem for a class of linear time-delay systems subject to norm-bounded disturbances and impulsive measurement outliers (IMOs). The considered IMOs are modeled by a sequence of impulsive signals with certain known minimum norm (i.e., the minimum of the norms of all impulsive signals). In order to characterize the occasional occurrence of IMOs, a sequence of independent and identically distributed random variables is introduced to depict the interval lengths (i.e., the durations between two adjacent IMOs) of the outliers. In order to achieve satisfactory filtering performance, a novel parameter-dependent filtering approach is proposed to protect the filtering performance from IMOs by using a special outlier detection scheme, which is developed based on a particular input–output model. First, the ultimate boundedness (in mean square) of the filtering error is investigated by using the stochastic analysis technique and the Lyapunov-functional-like method. Then, the desired filter gain matrix is derived through solving a constrained optimization problem. Furthermore, the designed filtering scheme is applied to the case where the statistical properties about the interval lengths of outliers are completely unknown. Finally, a simulation example is provided to demonstrate the effectiveness of our proposed filtering strategy.
Online social networks security and privacy: comprehensive review and analysis

Abstract With fast-growing technology, online social networks (OSNs) have exploded in popularity over the past few years. The pivotal reason behind this phenomenon happens to be the ability of OSNs to provide a platform for users to connect with their family, friends, and colleagues. The information shared in social network and media spreads very fast, almost instantaneously which makes it attractive for attackers to gain information. Secrecy and surety of OSNs need to be inquired from various positions. There are numerous security and privacy issues related to the user’s shared information especially when a user uploads personal content such as photos, videos, and audios. The attacker can maliciously use shared information for illegitimate purposes. The risks are even higher if children are targeted. To address these issues, this paper presents a thorough review of different security and privacy threats and existing solutions that can provide security to social network users. We have also discussed OSN attacks on various OSN web applications by citing some statistics reports. In addition to this, we have discussed numerous defensive approaches to OSN security. Finally, this survey discusses open issues, challenges, and relevant security guidelines to achieve trustworthiness in online social networks.
Image Inpainting Guided by Coherence Priors of Semantics and Textures

Existing inpainting methods have achieved promising performance in recovering defective images of specific scenes. However, filling holes involving multiple semantic categories remains challenging due to the obscure se-mantic boundaries and the mixture of different semantic textures. In this paper, we introduce coherence priors between the semantics and textures which make it possible to concentrate on completing separate textures in a semantic-wise manner. Specifically, we adopt a multi-scale joint optimization framework to first model the coherence priors and then accordingly interleaving optimize image inpainting and semantic segmentation in a coarse-to-fine manner. A Semantic-Wise Attention Propagation (SWAP) module is devised to refine completed image textures across scales by exploring non-local semantic coherence, which effectively mitigates the mix-up of textures. We also propose two coherence losses to constrain the consistency between the semantics and the inpainted image in terms of the overall structure and detailed textures. Experimental results demonstrate the superiority of our proposed method for challenging cases with complex holes.
Deep Dual Consecutive Network for Human Pose Estimation

Multi-frame human pose estimation in complicated situations is challenging. Although state-of-the-art human joints detectors have demonstrated remarkable results for static images, their performances come short when we apply these models to video sequences. Prevalent shortcomings include the failure to handle motion blur, video defocus, or pose occlusions, arising from the inability in capturing the temporal dependency among video frames. On the other hand, directly employing conventional recurrent neural networks incurs empirical difficulties in modeling spatial contexts, especially for dealing with pose occlusions. In this paper, we propose a novel multi-frame human pose estimation framework, leveraging abundant temporal cues between video frames to facilitate keypoint detection. Three modular components are designed in our framework. A Pose Temporal Merger encodes keypoint spatiotemporal context to generate effective searching scopes while a Pose Residual Fusion module computes weighted pose residuals in dual directions. These are then processed via our Pose Correction Network for efficient refining of pose estimations. Our method ranks No.1 in the Multi-frame Person Pose Estimation Challenge on the large-scale benchmark datasets PoseTrack2017 and PoseTrack2018. We have released our code, hoping to inspire future research.
Interventional Video Grounding with Dual Contrastive Learning

Video grounding aims to localize a moment from an untrimmed video for a given textual query. Existing approaches focus more on the alignment of visual and language stimuli with various likelihood-based matching or regression strategies, i.e., P(Y |X). Consequently, these models may suffer from spurious correlations between the language and video features due to the selection bias of the dataset. 1) To uncover the causality behind the model and data, we first propose a novel paradigm from the perspective of the causal inference, i.e., interventional video grounding (IVG) that leverages backdoor adjustment to deconfound the selection bias based on structured causal model (SCM) and do-calculus P(Y |do(X)). Then, we present a simple yet effective method to approximate the unobserved confounder as it cannot be directly sampled from the dataset. 2) Meanwhile, we introduce a dual contrastive learning approach (DCL) to better align the text and video by maximizing the mutual information (MI) between query and video clips, and the MI between start/end frames of a target moment and the others within a video to learn more informative visual representations. Experiments on three standard benchmarks show the effectiveness of our approaches.
ST3D: Self-training for Unsupervised Domain Adaptation on 3D Object Detection

We present a new domain adaptive self-training pipeline, named ST3D, for unsupervised domain adaptation on 3D object detection from point clouds. First, we pre-train the 3D detector on the source domain with our proposed random object scaling strategy for mitigating the negative effects of source domain bias. Then, the detector is iteratively improved on the target domain by alternatively conducting two steps, which are the pseudo label updating with the developed quality-aware triplet memory bank and the model training with curriculum data augmentation. These specific designs for 3D object detection enable the detector to be trained with consistent and high-quality pseudo labels and to avoid overfitting to the large number of easy examples in pseudo labeled data. Our ST3D achieves state-of-the-art performance on all evaluated datasets and even surpasses fully supervised results on KITTI 3D object detection benchmark. Code will be available at https://github.com/CVMI-Lab/ST3D.
Learning a Proposal Classifier for Multiple Object Tracking

The recent trend in multiple object tracking (MOT) is heading towards leveraging deep learning to boost the tracking performance. However, it is not trivial to solve the data-association problem in an end-to-end fashion. In this paper, we propose a novel proposal-based learnable framework, which models MOT as a proposal generation, proposal scoring and trajectory inference paradigm on an affinity graph. This framework is similar to the two-stage object detector Faster RCNN, and can solve the MOT problem in a data-driven way. For proposal generation, we propose an iterative graph clustering method to reduce the computational cost while maintaining the quality of the generated proposals. For proposal scoring, we deploy a trainable graph-convolutional-network (GCN) to learn the structural patterns of the generated proposals and rank them according to the estimated quality scores. For trajectory inference, a simple deoverlapping strategy is adopted to generate tracking output while complying with the constraints that no detection can be assigned to more than one track. We experimentally demonstrate that the proposed method achieves a clear performance improvement in both MOTA and IDF1 with respect to previous state-of-the-art on two public benchmarks. Our code is available at https://github.com/daip13/LPC_MOT.git.
Parser-Free Virtual Try-on via Distilling Appearance Flows

Image virtual try-on aims to fit a garment image (target clothes) to a person image. Prior methods are heavily based on human parsing. However, slightly-wrong segmentation results would lead to unrealistic try-on images with large artifacts. A recent pioneering work employed knowledge distillation to reduce the dependency of human parsing, where the try-on images produced by a parser-based method are used as supervisions to train a "student" network without relying on segmentation, making the student mimic the try-on ability of the parser-based model. However, the image quality of the student is bounded by the parser-based model. To address this problem, we propose a novel approach, "teacher-tutor-student" knowledge distillation, which is able to produce highly photo-realistic images without human parsing, possessing several appealing advantages compared to prior arts. (1) Unlike existing work, our approach treats the fake images produced by the parser-based method as "tutor knowledge", where the artifacts can be corrected by real "teacher knowledge", which is extracted from the real person images in a self-supervised way. (2) Other than using real images as supervisions, we formulate knowledge distillation in the try-on problem as distilling the appearance flows between the person image and the garment image, enabling us to find accurate dense correspondences between them to produce high-quality results. (3) Extensive evaluations show large superiority of our method (see Fig. 1).
Artificial intelligence and human workers interaction at team level: a conceptual assessment of the challenges and potential HRM strategies

Purpose This paper aims to specifically focus on the challenges that human resource management (HRM) leaders and departments in contemporary organisations face due to close interaction between artificial intelligence (AI) (primarily robots) and human workers especially at the team level. It further discusses important potential strategies, which can be useful to overcome these challenges based on a conceptual review of extant research. Design/methodology/approach The current paper undertakes a conceptual work where multiple streams of literature are integrated to present a rather holistic yet critical overview of the relationship between AI (particularly robots) and HRM in contemporary organisations. Findings We highlight that interaction and collaboration between human workers and robots is visible in a range of industries and organisational functions, where both are working as team members. This gives rise to unique challenges for HRM function in contemporary organisations where they need to address workers' fear of working with AI, especially in relation to future job loss and difficult dynamics associated with building trust between human workers and AI-enabled robots as team members. Along with these, human workers' task fulfilment expectations with their AI-enabled robot colleagues need to be carefully communicated and managed by HRM staff to maintain the collaborative spirit, as well as future performance evaluations of employees. The authors found that organisational support mechanisms such as facilitating environment, training opportunities and ensuring a viable technological competence level before organising human workers in teams with robots are important. Finally, we found that one of the toughest challenges for HRM relates to performance evaluation in teams where both humans and AI (including robots) work side by side. We referred to the lack of existing frameworks to guide HRM managers in this concern and stressed the possibility of taking insights from the computer gaming literature, where performance evaluation models have been developed to analyse humans and AI interactions while keeping the context and limitations of both in view. Originality/value Our paper is one of the few studies that go beyond a rather general or functional analysis of AI in the HRM context. It specifically focusses on the teamwork dimension, where human workers and AI-powered machines (robots) work together and offer insights and suggestions for such teams' smooth functioning.
Anthropomorphism and customers’ willingness to use artificial intelligence service agents

Previous studies provide inconsistent evidence regarding the effect of anthropomorphism on customers’ willingness to use AI service agents. This paper explains the reason by introducing service context. Two situational experiments are used to demonstrate that under the context of high perceived control, customers expect AI service agents with more anthropomorphic designs to perform better and prefer highly human-like AI service agents. However, under the context of low perceived control, customers perceive stronger threat in facing AI service agents with more anthropomorphic designs and prefer less human-like AI service agents. Moreover, we find that this effect is significant only in social scenarios. These findings provide new insights into previous inconsistent evidence regarding anthropomorphic design’s influence on customers’ willingness to use AI service agents. Our findings also have important implications for AI service agents design in different service contexts and advance the literature on human–robot interaction and marketing.