Reviewerly/exHarmony huggingface dataset:
Subband Aware CNN for Cell-Phone Recognition

Identifying the model of a cell-phone with which an audio recording is made serves an important forensic purpose. In this paper, we propose a channel attention mechanism based on subband awareness to focus on the most relevant parts of the frequency bands to produce more efficient feature representations for cell-phone recognition task. A multi-stream network is introduced to fully exploit difference among frequency bands that provide critical clues to identify the fingerprints from the built-in microphones, and thus is able to recognize cell-phones from different manufacturers and even different models from the same manufacturer. The effectiveness of the proposed design is validated by a collection of speaker-independent audio recordings from 20 models of cell-phones made by 5 major manufacturers. In particular, the fusion of data augmentation and attention strategy greatly increases the robustness of the scheme when additive noise is present in the recorded audios. Finally, the salient regions infer the critical subbands for the recognition of different cell-phone models.
Counterfactual Online Learning to Rank

Exploiting users' implicit feedback, such as clicks, to learn rankers is attractive as it does not require editorial labelling effort, and adapts to users' changing preferences, among other benefits. However, directly learning a ranker from implicit data is challenging, as users' implicit feedback usually contains bias (e.g., position bias, selection bias) and noise (e.g., clicking on irrelevant but attractive snippets, adversarial clicks). Two main methods have arisen for optimizing rankers based on implicit feedback: counterfactual learning to rank (CLTR), which learns a ranker from the historical click-through data collected from a deployed, logging ranker; and online learning to rank (OLTR), where a ranker is updated by recording user interaction with a result list produced by multiple rankers (usually via interleaving). In this paper, we propose a counterfactual online learning to rank algorithm (COLTR) that combines the key components of both CLTR and OLTR. It does so by replacing the online evaluation required by traditional OLTR methods with the counterfactual evaluation common in CLTR. Compared to traditional OLTR approaches based on interleaving, COLTR can evaluate a large number of candidate rankers in a more efficient manner. Our empirical results show that COLTR significantly outperforms traditional OLTR methods. Furthermore, COLTR can reach the same effectiveness of the current state-of-the-art, under noisy click settings, and has room for future extensions.
A Hybrid Network Based on Dense Connection and Weighted Feature Aggregation for Human Activity Recognition

Human activity recognition (HAR) using body-worn sensors is an active research area in human-computer interaction and human activity analysis. The traditional methods use hand-crafted features to classify multiple activities, which is both heavily dependent on human domain knowledge and results in shallow feature extraction. Rapid developments in deep learning have caused most researchers to switch to deep learning methods, which extract features from raw data automatically. Most of the existing works on human activity recognition tasks involve multimodal sensor data, and these networks mainly focus on the top representation extracted from bottom-up feedforward process without reusing other features from bottom layers. In this paper, we present a novel hybrid deep learning network for human activity recognition that also employs multimodal sensor data; however, our proposed model is a ConvLSTM pipeline that makes full use of the information in each layer extracted along the temporal domain. Thus, we propose a dense connection module (DCM) to ensure maximum information flow between the network layers. Furthermore, we employ a multilayer feature aggregation module (MFAM) to extract features along the spatial domain, and we aggregate the features obtained from every convolutional layer according to the importance of features in different spatial locations. The output of the MFAM is input into two LSTM layers to further model the temporal dependencies. Finally, a fully connected layer and a softmax function are used to compute the probability of each class. We demonstrate the effectiveness of our proposed model on two benchmark datasets: Opportunity and UniMiB-SHAR. The results illustrate that our designed network outperforms the state-of-the-art models. We also conduct experiments on efficiency, multimodal fusion and different hyperparameters to analyze our proposed network. Finally, we carry out ablation and visualization experiments to reveal the effectiveness of the two proposed modules.
Aerial Multi-Object Tracking by Detection Using Deep Association Networks

A lot a research is focused on object detection and it has achieved significant advances with deep learning techniques in recent years. Inspite of the existing research, these algorithms are not usually optimal for dealing with sequences or images captured by drone-based platforms, due to various challenges such as view point change, scales, density of object distribution and occlusion. In this paper, we develop a model for detection of objects in drone images using the VisDrone2019 DET dataset. Using the RetinaNet model as our base, we modify the anchor scales to better handle the detection of dense distribution and small size of the objects. We explicitly model the channel interdependencies by using “Squeeze-and-Excitation” (SE) blocks that adaptively recalibrates channel-wise feature responses. This helps to bring significant improvements in performance at a slight additional computational cost. Using this architecture for object detection, we build a custom DeepSORT network for object detection on the VisDrone2019 MOT dataset by training a custom Deep Association network for the algorithm.
Digital Image Analysis of Heterogeneous Tuberculosis Pulmonary Pathology in Non-Clinical Animal Models using Deep Convolutional Neural Networks

Efforts to develop effective and safe drugs for treatment of tuberculosis require preclinical evaluation in animal models. Alongside efficacy testing of novel therapies, effects on pulmonary pathology and disease progression are monitored by using histopathology images from these infected animals. To compare the severity of disease across treatment cohorts, pathologists have historically assigned a semi-quantitative histopathology score that may be subjective in terms of their training, experience, and personal bias. Manual histopathology therefore has limitations regarding reproducibility between studies and pathologists, potentially masking successful treatments. This report describes a pathologist-assistive software tool that reduces these user limitations, while providing a rapid, quantitative scoring system for digital histopathology image analysis. The software, called 'Lesion Image Recognition and Analysis' (LIRA), employs convolutional neural networks to classify seven different pathology features, including three different lesion types from pulmonary tissues of the C3HeB/FeJ tuberculosis mouse model. LIRA was developed to improve the efficiency of histopathology analysis for mouse tuberculosis infection models, this approach has also broader applications to other disease models and tissues. The full source code and documentation is available from https://Github.com/TB-imaging/LIRA.
Digital Signal Modulation Recognition Algorithm Based on VGGNet Model

Aiming at the problem that feature extraction in traditional modulation recognition relies on manual experience and the poor performance of traditional methods in low signal-to-noise ratio (SNR), a deep learning intelligent modulation recognition algorithm based on VGG convolution neural network model is proposed in this paper, which replaces manual design features and improves the recognition performance of digital signals in low SNR. The algorithm first converts the sampled data of communication signals into gray images, then trains the VGGNet model built under PyTorch to extract and select the features of six kinds of digital modulation signals automatically, thus realizing the automatic recognition of digital modulation signals. The simulation results show that the recognition rate can reach over 98% when SNR is -2dB, which is better than the recognition results using support vector machine and other algorithms, thus verifying the validity of this method for digital modulation signal recognition at low SNR.
Text classification with semantically enriched word embeddings

Abstract The recent breakthroughs in deep neural architectures across multiple machine learning fields have led to the widespread use of deep neural models. These learners are often applied as black-box models that ignore or insufficiently utilize a wealth of preexisting semantic information. In this study, we focus on the text classification task, investigating methods for augmenting the input to deep neural networks (DNNs) with semantic information. We extract semantics for the words in the preprocessed text from the WordNet semantic graph, in the form of weighted concept terms that form a semantic frequency vector. Concepts are selected via a variety of semantic disambiguation techniques, including a basic, a part-of-speech-based, and a semantic embedding projection method. Additionally, we consider a weight propagation mechanism that exploits semantic relationships in the concept graph and conveys a spreading activation component. We enrich word2vec embeddings with the resulting semantic vector through concatenation or replacement and apply the semantically augmented word embeddings on the classification task via a DNN. Experimental results over established datasets demonstrate that our approach of semantic augmentation in the input space boosts classification performance significantly, with concatenation offering the best performance. We also note additional interesting findings produced by our approach regarding the behavior of term frequency - inverse document frequency normalization on semantic vectors, along with the radical dimensionality reduction potential with negligible performance loss.
Gaussian Processes Over Graphs

Kernel Regression over Graphs (KRG) was recently proposed for predicting graph signals in a supervised learning setting, where the inputs are agnostic to the graph. KRG model predicts targets that are smooth graph signals as over the given graph, given the input when all the signals are deterministic. In this work, we consider the development of a stochastic or Bayesian variant of KRG. Using priors and likelihood functions, our goal is to systematically derive a predictive distribution for the smooth graph signal target given the training data and a new input. We show that this naturally results in a Gaussian process formulation which we call Gaussian Processes over Graphs (GPG). Experiments with real-world datasets show that the performance of GPG is superior to a conventional Gaussian Process (without the graph-structure) for small training data sizes and under noisy training.
Energy-Efficient Workflow Scheduling Using Container-Based Virtualization in Software-Defined Data Centers

Workflow scheduling is one of the most difficult tasks due to the variation in the traffic flows generated from diverse cloud applications. Hence, in this article, a container-based virtualization is used to design an energy-efficient workflow scheduling in software-defined data centers. The containers provide the flexibility to the applications to access the underlying resource as per their requirements. Moreover, a runtime scheduler is responsible to handle all the scheduling decisions in the proposed workflow scheduling scheme. Even more, a doubly linked list-based access mechanism is used to provide access to the servers and virtual machines by traversing both ways. Finally, a hashing scheme is used to select an ideal location for the allocation of the containers. The proposed scheme is evaluated with respect to different performance metrics (makespan, execution time, fault tolerance, energy consumption, etc.) on the real data traces. The results obtained depict the superiority of the proposed scheme in comparison to the other existing schemes of its category.
Emotional Voice Conversion Using Multitask Learning with Text-To-Speech

Voice conversion (VC) is a task that alters the voice of a person to suit different styles while conserving the linguistic content. Previous state-of-the-art technology used in VC was based on the sequence-to-sequence (seq2seq) model, which could lose linguistic information. There was an attempt to overcome this problem using textual supervision; however, this required explicit alignment, and therefore the benefit of using seq2seq model was lost. In this study, a voice converter that utilizes multitask learning with text-to-speech (TTS) is presented. By using multitask learning, VC is expected to capture linguistic information and preserve the training stability. This method does not require explicit alignment for capturing abundant text information. Experiments on VC were performed on a male-Korean-emotional-text-speech dataset to convert the neutral voice to emotional voice. It was shown that multitask learning helps to preserve the linguistic content.
A Multi-layer Bidirectional Transformer Encoder for Pre-trained Word Embedding: A Survey of BERT

Language modeling is the task of assigning a probability distribution over sequences of words that matches the distribution of a language. A language model is required to represent the text to a form understandable from the machine point of view. A language model is capable to predict the probability of a word occurring in the context-related text. Although it sounds formidable, in the existing research, most of the language models are based on unidirectional training. In this paper, we have investigated a bi-directional training model-BERT (Bidirectional Encoder Representations from Transformers). BERT builds on top of the bidirectional idea as compared to other word embedding models (like Elmo). It practices the comparatively new transformer encoder-based architecture to compute word embedding. In this paper, it has been described that how this model is to be producing or achieving state-of-the-art results on various NLP tasks. BERT has the capability to train the model in bi-directional over a large corpus. All the existing methods are based on unidirectional training (either the left or the right). This bi-directionality of the language model helps to obtain better results in the context-related classification tasks in which the word(s) was used as input vectors. Additionally, BERT is outlined to do multi-task learning using context-related datasets. It can perform different NLP tasks simultaneously. This survey focuses on the detailed representation of the BERT- based technique for word embedding, its architecture, and the importance of this model for pre-training purposes using a large corpus.
Attention-Based ASR with Lightweight and Dynamic Convolutions

End-to-end (E2E) automatic speech recognition (ASR) with sequence-to-sequence models has gained attention because of its simple model training compared with conventional hidden Markov model based ASR. Recently, several studies report the state-of-the-art E2E ASR results obtained by Transformer. Compared to recurrent neural network (RNN) based E2E models, training of Transformer is more efficient and also achieves better performance on various tasks. However, self-attention used in Transformer requires computation quadratic in its input length. In this paper, we propose to apply lightweight and dynamic convolution to E2E ASR as an alternative architecture to the self-attention to make the computational order linear. We also propose joint training with connectionist temporal classification, convolution on the frequency axis, and combination with self-attention. With these techniques, the proposed architectures achieve better performance than RNN-based E2E model and performance competitive to state-of-the-art Transformer on various ASR benchmarks including noisy/reverberant tasks.
Selective residual learning for Visual Question Answering

Visual Question Answering (VQA) aims to reason an answer, given a textual question and image pair. VQA methods are required to learn the relationship between image region features. These methods have the limitation of inefficient learning that can produce a performance drop. It is because current intra-relationship methods are trying to learn all the intra-relationships, regardless of their importance. In this paper, a novel self-attention based VQA module named Selective Residual learning (SelRes) is proposed. SelRes processes the residual learning selectively in self-attention networks. It measures the importance of the input vectors by the attention map and limits residual learning, except in the selected regions which related to the correct answer. Selective masking is also proposed, which can ensure that the selection in SelRes is preserved in the multi-stack structure of the VQA network. Our full model achieves new state-of-the-art performances on both from-scratch and fine-tuning models.
Application of Support Vector Machine for Arabic Sentiment Classification Using Twitter-Based Dataset

Sentiment classification is the process of classifying emotions and opinions in texts. In this study, the problem of Arabic sentiment analysis was addressed. A support vector machine (SVM) model was proposed to classify opinions in Arabic micro-texts as being positive or negative. To evaluate the performance of the SVM model, a dataset was built from tweets discussing several social issues in Saudi Arabia. These issues include changes that were implemented by the country as part of a newly established vision, known as Saudi Arabia Vision 2030. The constructed dataset was manually annotated according to the sentiment conveyed in the text. To achieve the best sentiment classification accuracy, several procedures were implemented within the proposed framework including light stemming, feature extraction (Ngrams, emoji and tweet-topic features), parameter optimisation and feature-set reduction. The experimental results revealed excellent outcomes. An accuracy of 89.83% was achieved using the proposed SVM model.
Pre-Training for Query Rewriting in a Spoken Language Understanding System

Query rewriting (QR) is an increasingly important technique to reduce customer friction caused by errors in a spoken language understanding pipeline, where the errors originate from various sources such as speech recognition errors, language understanding errors or entity resolution errors. In this work, we first propose a neural-retrieval based approach for query rewriting. Then, inspired by the wide success of pre-trained contextual language embeddings, and also as a way to compensate for insufficient QR training data, we propose a language-modeling (LM) based approach to pre-train query embeddings on historical user conversation data with a voice assistant. In addition, we propose to use the NLU hypotheses generated by the language understanding system to augment the pre-training. Our experiments show pre-training provides rich prior information and help the QR task achieve strong performance. We also show joint pre-training with NLU hypotheses has further benefit. Finally, after pre-training, we find a small set of rewrite pairs is enough to fine-tune the QR model to outperform a strong baseline by full training on all QR training data.
Multiple-target tracking in human and machine vision

Humans are able to track multiple objects at any given time in their daily activities-for example, we can drive a car while monitoring obstacles, pedestrians, and other vehicles. Several past studies have examined how humans track targets simultaneously and what underlying behavioral and neural mechanisms they use. At the same time, computer-vision researchers have proposed different algorithms to track multiple targets automatically. These algorithms are useful for video surveillance, team-sport analysis, video analysis, video summarization, and human-computer interaction. Although there are several efficient biologically inspired algorithms in artificial intelligence, the human multiple-target tracking (MTT) ability is rarely imitated in computer-vision algorithms. In this paper, we review MTT studies in neuroscience and biologically inspired MTT methods in computer vision and discuss the ways in which they can be seen as complementary.
Effectiveness of Random Deep Feature Selection for Securing Image Manipulation Detectors Against Adversarial Examples

We investigate if the random feature selection approach proposed in [1] to improve the robustness of forensic detectors to targeted attacks, can be extended to detectors based on deep learning features. In particular, we study the transferability of adversarial examples targeting an original CNN image manipulation detector to other detectors (a fully connected neural network and a linear SVM) that rely on a random subset of the features extracted from the flatten layer of the original network. The results we got by considering three image manipulation detection tasks (resizing, median filtering and adaptive histogram equalization), two original network architectures and three classes of attacks, show that feature randomization helps to hinder attack transferability, even if, in some cases, simply changing the architecture of the detector, or even retraining the detector is enough to prevent the transferability of the attacks.
Cosine adapted modified whale optimization algorithm for control of switched reluctance motor

Abstract Whale optimization algorithm (WOA) imitates social conduct of humpback whales which is inspired by bubble net hunting strategy of humpback whales. In the present study, Cosine adapted modified whale optimization algorithm (CamWOA) which is a modified version of WOA, has been proposed where cosine function is incorporated for the selection of control parameter “ d ” which governs the position of whales during optimization process. Also, correction factors are employed to modify the movement of search agents during the search process. These changes provide a proper balance between exploration and exploitation phases in CamWOA technique. The performance of CamWOA is analyzed by testing on a set of benchmark functions and compared with other state‐of‐the‐art algorithms. It is observed that CamWOA outperforms other state‐of‐the‐art metaheuristic algorithms in majority of benchmark functions. The efficiency of CamWOA is also evaluated by solving a multiobjective engineering problem pertaining to control of switched reluctance motor. The simulation results confirm that CamWOA yields very promising and competitive results compared to that of WOA and other metaheuristic optimization algorithms.
Detection of Adversarial Attacks and Characterization of Adversarial Subspace

Adversarial attacks have always been a serious threat for any data-driven model. In this paper, we explore subspaces of adversarial examples in unitary vector domain, and we propose a novel detector for defending our models trained for environmental sound classification. We measure chordal distance between legitimate and malicious representation of sounds in unitary space of generalized Schur decomposition and show that their manifolds lie far from each other. Our front-end detector is a regularized logistic regression which discriminates eigenvalues of legitimate and adversarial spectrograms. The experimental results on three benchmarking datasets of environmental sounds represented by spectrograms reveal high detection rate of the proposed detector for eight types of adversarial attacks and it also outperforms other detection approaches.