Reviewerly/exHarmony huggingface dataset:
Penalized multiple distribution selection method for imbalanced data classification

In reality, the amount of data from different categories varies significantly, which results in learning bias towards prominent classes, hindering the overall classification performance. In this paper, by proving that traditional classification methods that use single softmax distribution are limited for modeling complex and imbalanced data, we propose a general Multiple Distribution Selection (MDS) method for imbalanced data classification. MDS employs a mixture distribution that is composed of a single softmax distribution and a set of degenerate distributions to model imbalanced data. Furthermore, a dynamic distribution selection method, based on L1 regularization, is also proposed to automatically determine the weights of distributions. In addition, the corresponding two-stage optimization algorithm is designed to estimate the parameters of models. Extensive experiments conducted on three widely used benchmark datasets (IMDB, ACE2005, 20NewsGroups) show that our proposed mixture method outperforms previous methods. Moreover, under highly imbalanced setting, our method achieves up to a 4.1 absolute F1 gain over high-performing baselines.
Privacy of Dependent Users Against Statistical Matching

Modern applications significantly enhance user experience by adapting to each user's individual condition and/or preferences. While this adaptation can greatly improve a user's experience or be essential for the application to work, the exposure of user data to the application presents a significant privacy threat to the users-even when the traces are anonymized-since the statistical matching of an anonymized trace to prior user behavior can identify a user and their habits. Because of the current and growing algorithmic and computational capabilities of adversaries, provable privacy guarantees as a function of the degree of anonymization and obfuscation of the traces are necessary. Our previous work has established the requirements on anonymization and obfuscation in the case that data traces are independent between users. However, the data traces of different users will be dependent in many applications, and an adversary can potentially exploit such. In this paper, we consider the negative impact of dependency between user traces on their privacy. First, we demonstrate that the adversary can readily identify the association graph of the obfuscated and anonymized version of the data, revealing which user data traces are dependent. Next, we demonstrate that the adversary can use this association graph to break user privacy with significantly shorter traces than in the case of independent users, and that obfuscating data traces independently across users is often insufficient to remedy such leakage. In other words, we have shown that inter-user dependency is disastrous to privacy, and any non-negligible dependency between users significantly reduces the effectiveness of anonymization and obfuscation schemes. Finally, we discuss how users can improve privacy by employing joint obfuscation that removes or reduces the data dependency.
Human activity recognition in RGB-D videos by dynamic images

Human Activity Recognition in RGB-D videos has been an active research topic during the last decade. However, only a few efforts have been made, for recognizing human activity in RGB-D videos where several performers are performing simultaneously. In this paper we introduce such a challenging dataset with several performers performing the activities simultaniously. We present a novel method for recognizing human activities performed simultaniously in the same videos. The proposed method aims in capturing the motion information of the whole video by producing a dynamic image corresponding to the input video. We use two parallel ResNet-101 architectures to produce the dynamic images for the RGB video and depth video separately. The dynamic images contain only the motion information of the whole frame, which is the main cue for analyzing the motion of the performer during action. Hence, dynamic images help recognizing human action by concentrating only on the motion information appeared on the frame. We send the two dynamic images through a fully connected layer for classification of activity. The proposed dynamic image reduces the complexity of the recognition process by extracting a sparse matrix from a video, while preserving the motion information required for activity recognition, and produces comparable results with respect to the state-of-the-art.
A Customized Convolutional Neural Network Model Integrated With Acceleration-Based Smart Insole Toward Personalized Foot Gesture Recognition

Foot gestures play an important role in human machine interface and also indicate the lower-limb motor functionalities. Lately, machine learning models have been integrated with different kinds of wearable devices for foot gesture recognition. However, most existing wearable devices lack of robustness and facileness and existing machine learning models suffer from inconsistent recognition accuracy due to limited and unbalanced training datasets and, therefore, do not work well for new users. To address this, a customized convolutional neural network model architecture is designed, and strategies for achieving personalized models are presented, considering potential fairness sensitive attributes of a diverse training group, including age, gender, body mass index (defined by height and weight), shoe size, and health conditions. The proposed model to be embodied into a flexible smart insole with only two accelerometers aims to recognize four types of foot gestures-toe tapping, heel tapping, foot kicking, and foot stepping-in a sitting posture at high accuracies. After the hyperparameters are well-tuned, average recognition accuracy reaches 92% in training sets, 88.05% in validation sets, 85.27% in test sets, and 83.09% in fivefold cross-validation. The evaluation results from both in-group and leave-one-subject-out show that our unified model is easily transformable for individual uses by further fine-tuning the key hyperparameters.
JavaScript API Deprecation in the Wild: A First Assessment

Building an application using third-party libraries is a common practice in software development. As any other software system, code libraries and their APIs evolve over time. In order to help version migration and ensure backward compatibility, a recommended practice during development is to deprecate API. Although studies have been conducted to investigate deprecation in some programming languages, such as Java and C#, there are no detailed studies on API deprecation in the JavaScript ecosystem. This paper provides an initial assessment of API deprecation in JavaScript by analyzing 50 popular software projects. Initial results suggest that the use of deprecation mechanisms in JavaScript packages is low. However, we find five different ways that developers use to deprecate API in the studied projects. Among these solutions, deprecation utility (i.e., any sort of function specially written to aid deprecation) and code comments are the most common practices in JavaScript. Finally, we find that the rate of helpful message is high: 67% of the deprecations have replacement messages to support developers when migrating APIs.
CNN and Stacked LSTM Model for Indian Sign Language Recognition

In this paper, we propose a deep learning for sign language recognition using convolutional neural network (CNN) and long short term memory (LSTM). The architecture used CNN as a pretrained model for feature extraction and is passed to the LSTM for capturing spatio-temporal information. One more LSTM is stacked for increasing the accuracy. Deep learning model which captures temporal information is less. There is only less papers which deals with sign language recognition by using the deep learning architectures such as CNN and LSTM. The algorithm was tested in Indian sign language (ISL) dataset. We have presented the performance evaluation after testing with ISL dataset. Literature shows that deep learning models capturing temporal information is still an open research problem.
The Role of Flag Emoji in Online Political Communication

Flags are important national symbols that have transcended into the digital world with inclusion in the Unicode character set. Despite their significance, there is little information about their role in online communication. This article examines the role of flag emoji in political communication online by analyzing 640,676 tweets by the most important political parties and Members of Parliament in Germany and the United States. We find that national flags are frequently used in political communication and are mostly used in-line with political ideology. As off-line, flag emoji usage in online communication is associated with external events of national importance. This association is stronger in the United States than in Germany. The results also reveal that the presence of the national flag emoji is associated with significantly higher engagement in Germany irrespective of party, whereas it is associated with slightly higher engagement for politicians of the Republican party and slightly lower engagement for Democrats in the United States. Implications of the results and future research directions are discussed.
Blind image deblurring using group sparse representation

We propose a blind image deblurring model based on the group sparse representation (GSR) prior. The motivation for this work is that adjacent similar patches of images and blur kernels favor clear images and ground-truth kernels over blurred images and poor kernels. Therefore, we impose GSR constraints on similar patches based on images and kernels to ensure the sparsity of the intermediate latent images and kernels. In addition, to effectively preserve the main edges in the image, we impose an L0-regularized gradient prior on image restoration. Moreover, a flexible non-convex weighted Lp-norm minimization is applied to the GSR and an adaptive dictionary is used for each similar group. To optimize our model, an effective optimization algorithm based on a generalized soft-thresholding algorithm and half-quadratic splitting strategy is developed. Several analyses verify the validity of the GSR prior. Experiments show that our method achieves satisfactory deblurring effects in several specific domains and is robust in both deblurring and denoising tasks. The results of qualitative and quantitative experiments indicate that our algorithm performs favorably with respect to state-of-the-art algorithms.
Multi-View Vehicle Detection Based on Fusion Part Model With Active Learning

Computer vision-based vehicle detection techniques are widely used in real-world applications. However, most of these techniques aim to detect only single-view vehicles, and their performances are easily affected by partial occlusion. Therefore, this paper proposes a novel multi-view vehicle detection system that uses a part model to address the partial occlusion problem and the high variance between all types of vehicles. There are three features in this paper; firstly, different from Deformable Part Model, the construction of part models in this paper is visual and can be replaced at any time. Secondly, this paper proposes some new part models for detection of vehicles according to the appearance analysis of a large number of modern vehicles by the active learning algorithm. Finally, this paper proposes the method that contains color transformation along with the Bayesian rule to filter out the background to accelerate the detection time and increase accuracy. The proposed method outperforms other methods on given dataset.
Learning to Detect Keyword Parts and Whole by Smoothed Max Pooling

We propose smoothed max pooling loss and its application to keyword spotting systems. The proposed approach jointly trains an encoder (to detect keyword parts) and a decoder (to detect whole keyword) in a semi-supervised manner. The proposed new loss function allows training a model to detect parts and whole of a keyword, without strictly depending on frame-level labeling from LVCSR (Large vocabulary continuous speech recognition), making further optimization possible. The proposed system outperforms the baseline keyword spotting model in [1] due to increased optimizability. Further, it can be more easily adapted for on-device learning applications due to reduced dependency on LVCSR.
Trajectory-Based Air-Writing Character Recognition Using Convolutional Neural Network

Writing in the air can be defined as to write digit or character in a 3D space by using a finger or marker movement. It is different from the traditional writing style. However, it has become easier to track finger and joint precisely due to the extensive improvement of sensor technologies. In this research, we proposed a trajectory-based air-writing character recognition system using a convolutional neural network (CNN). The trajectories were collected using a depth camera as a three-dimensional (3D) sequence. We used 10-fold cross-validation to validate the model. The accuracy of the proposed model was 97.29%. Also, we have collected an air-writing dataset containing 26,000 characters for training and validation, and another 4,000 for testing the model. The recognition time was 14ms per character which is fast enough to implement in a real-time system.
MSPNET: Multi-Supervised Parallel Network for Crowd Counting

Crowd counting has a wide range of applications such as video surveillance and public safety. Many existing methods only focus on improving the accuracy of counting but ignore the importance of density maps. It's no doubt that a high-quality density map contains more information such as localization and movement of the crowd. In this paper, we propose a multi-supervised parallel network (MSPNet) to achieve high accuracy of crowd counting and generate high-quality density maps. We conduct multiple supervisions in the training process, which can supplement the details lost in pooling and up-sampling operations to improve the quality of density maps. In addition, to reduce the impact of background noise, the attention mechanism is employed to help the network focus on the crowd. Extensive experiments on two mainstream benchmarks show that MSPNet achieves significantly improvement over the state-of-the-art in terms of counting accuracy and the quality of density maps.
DMGAN: Adversarial Learning-Based Decision Making for Human-Level Plant-Wide Operation of Process Industries Under Uncertainties

To achieve plant-wide operational optimization and dynamic adjustment of operational index for an industrial process, knowledge-based methods have been widely employed over the past years. However, the extraction of knowledge base is a bottleneck for most existing approaches. To address this problem, we propose a novel framework based on the generative adversarial networks (GANs), termed as decision-making GAN (DMGAN), which directly learns from operational data and performs human-level decision making of the operational indices for plant-wide operation. In the proposed DMGAN, two adversarial criteria and three cycle consistency criteria are incorporated to encourage efficient posterior inference. To improve the generalization power of a generator with an increasing complexity of the industrial processes, a reinforced U-Net (RU-Net) is presented that improves the traditional U-Net by providing a more general combinator, a building block design, and drop-level regularization. In this article, we also propose three quantitative metrics for assessing the plant-wide operation performance. A case study based on the largest mineral processing factory in Western China is carried out, and the experimental results demonstrate the promising performance of the proposed DMGAN when compared with decision-making based on domain experts.
DNN-Based Speech Recognition for Globalphone Languages

This paper describes new reference benchmark results based on hybrid Hidden Markov Model and Deep Neural Networks (HMM-DNN) for the GlobalPhone (GP) multilingual text and speech database. GP is a multilingual database of high-quality read speech with corresponding transcriptions and pronunciation dictionaries in more than 20 languages. Moreover, we provide new results for five additional languages, namely, Amharic, Oromo, Tigrigna, Wolaytta, and Uyghur. Across the 22 languages considered, the hybrid HMM-DNN models outperform the HMM-GMM based models regardless of the size of the training speech used. Overall, we achieved relative improvements that range from 7.14% to 59.43%.
G2G: TTS-Driven Pronunciation Learning for Graphemic Hybrid ASR

Grapheme-based acoustic modeling has recently been shown to outperform phoneme-based approaches in both hybrid and end-to-end automatic speech recognition (ASR), even on non-phonemic languages like English. However, graphemic ASR still has problems with low-frequency words that do not follow the standard spelling conventions seen in training, such as entity names. In this work, we present a novel method to train a statistical grapheme-to-grapheme (G2G) model on text-to-speech data that can rewrite an arbitrary character sequence into more phonetically consistent forms. We show that using G2G to provide alternative pronunciations during decoding reduces Word Error Rate by 3% to 11% relative over a strong graphemic baseline and bridges the gap on rare name recognition with an equivalent phonetic setup. Unlike many previously proposed methods, our method does not require any change to the acoustic model training procedure. This work reaffirms the efficacy of grapheme-based modeling and shows that specialized linguistic knowledge, when available, can be leveraged to improve graphemic ASR.
Reinforced Depth-Aware Deep Learning for Single Image Dehazing

Image dehazing continues to be one of the most challenging inverse problems. Deep learning methods have emerged to complement traditional model-based methods and have helped define a new state of the art in achievable dehazed image quality. However, most deep learning-based methods usually design a regression network as a black-box tool to either estimate the dehazed image and/or the physical parameters in the haze model, i.e. ambient light (A) and transmission map (t). The inverse haze model may then be used to estimate the dehazed image. In this work, we proposed a Depth-aware Dehazing using Reinforcement Learning system, denoted as DDRL. DDRL generates the dehazed image in a near-to-far progressive manner by utilizing the depth-information from the scene. This contrasts with the most recent learning-based methods that estimate these parameters in one pass. In particular, DDRL exploits the fact that the haze is less dense near the camera and gets increasingly denser as the scene moves farther away from the camera. DDRL consists of a policy network and a dehazing (regression) network. The policy network estimates the current depth for the dehazing network to use. A novel policy regularization term is introduced for the policy network to generate the policy sequence following the near-to-far order. Based on extensive tests over three benchmark test sets, DDRL demonstrates vastly enhanced dehazing results, particularly when training is limited.
The Method for Selecting the Urban Infrastructure Objects Contours

The necessity of distinguishing elements of the urban infrastructure on the optoelectronic images of remote sensing of the Earth is substantiated. Known methods for the selection of contours and their main disadvantages are analyzed. A method for isolating the contours of urban infrastructure objects on optoelectronic images of remote sensing of the Earth is proposed. The method includes two stages. The first stage is the selection of contours by the Canny edge detection algorithm. The second stage is the selection of geometric primitives using the Hough transform. The color image of the onboard Earth remote sensing system was processed. The results of the method at each intermediate stage are given. The assessment of the quality of the selection of the contours of urban infrastructure in the image carried out by visual means.
A Video Analytics Based Solution for Detecting the Attention Level of the Students in Class Rooms

Classroom surveillance, using video cameras, affords enhanced understanding of student behavior. This paper proposes a new algorithmic framework to evaluate the attention level of students, from classroom videos. The live video of a class room, when a teacher is delivering the lecture, is the input to the proposed framework. This framework identifies the key frames from the video and then detects the attention level of a particular student. The paper perused the Structural Similarity Index Method (SSIM) to discern key frames in a video. Detection of drowsiness is then performed to deduce whether or not the student is sleepy. Scrutiny of facial expressions is carried out, to perceive the psychological state of the student in the classroom. Finally, detection of gaze is carried out to examine whether or not the student's attention is on the black board. The algorithmic design for the proposed approach, the results obtained and the sample test cases are presented in this paper.
Attention-guided dual spatial-temporal non-local network for video super-resolution

In this paper, we propose an attention-guided dual spatial-temporal non-local network for video super-resolution (ADNLVSR). We integrate temporal and spatial non-local self-similar contexts from continuous video frames after motion compensation, and merge the features of different levels discriminatively with channel attention mechanism for target frame. During motion compensation, unlike previous methods directly stacking input images or features for merging, we use learnable attention mechanism to guide the merging, which suppresses undesired components caused by misalignment and enhances desirable fine details. During feature fusion, in contrast to most previous approaches where global-level non-local self-similarity existing in space or time is usually considered, we propose region-level spatial and temporal non-local operations for exploiting temporal correlations and enhancing similar spatial structures. The proposed modules can effectively avoid the computational burden caused by existing global-level non-local operations based on our analysis, and enhance correlated structure information. In addition, we propose a channel attention-guided residual dense block (CRDB), in which a second-order channel attention mechanism is applied to adaptively rescale the channel-wise features for more discriminative representations. Extensive experiments on different datasets demonstrate superior performance to state-of-the-art published methods on video super-resolution.
Improving End-to-End Speech Synthesis with Local Recurrent Neural Network Enhanced Transformer

Although Transformer based neural end-to-end TTS model has demonstrated extreme effectiveness in capturing long-term dependencies and achieved state-of-the-art performance, it still suffers from two problems. 1) limited ability to model sequential and local structures in sequences; 2) heavily rely on position embeddings that have limited effect but require an amount of design efforts. In this paper, we introduce local recurrent neural network (Local-RNN) into Transformer to make full use of the advantages of both RNN and Transformer while mitigating their drawbacks. The sequential and local structures could be effectively modeled by Local-RNN, while the long-term dependencies could be captured by Transformer without any use of position embeddings. Subjective evaluation results show our proposed model outperforms baseline (Transformer) with a gap of 0.12 in MOS and achieves close to human quality (4.34 vs. 4.45 in MOS) on general test. Case level intelligibility test also show an absolute improvement of 6.5% in case level intelligibility rate over the baseline on a challenging test.