Reviewerly/exHarmony huggingface dataset:
Text-Independent Speaker Identification Through Feature Fusion and Deep Neural Network

Speaker identification refers to the process of recognizing human voice using artificial intelligence techniques. Speaker identification technologies are widely applied in voice authentication, security and surveillance, electronic voice eavesdropping, and identity verification. In the speaker identification process, extracting discriminative and salient features from speaker utterances is an important task to accurately identify speakers. Various features for speaker identification have been recently proposed by researchers. Most studies on speaker identification have utilized short-time features, such as perceptual linear predictive (PLP) coefficients and Mel frequency cepstral coefficients (MFCC), due to their capability to capture the repetitive nature and efficiency of signals. Various studies have shown the effectiveness of MFCC features in correctly identifying speakers. However, the performances of these features degrade on complex speech datasets, and therefore, these features fail to accurately identify speaker characteristics. To address this problem, this study proposes a novel fusion of MFCC and time-based features (MFCCT), which combines the effectiveness of MFCC and time-domain features to improve the accuracy of text-independent speaker identification (SI) systems. The extracted MFCCT features were fed as input to a deep neural network (DNN) to construct the speaker identification model. Results showed that the proposed MFCCT features coupled with DNN outperformed existing baseline MFCC and time-domain features on the LibriSpeech dataset. In addition, DNN obtained better classification results compared with five machine learning algorithms that were recently utilized in speaker recognition. Moreover, this study evaluated the effectiveness of one-level and two-level classification methods for speaker identification. The experimental results showed that two-level classification presented better results than one-level classification. The proposed features and classification model for identifying a speaker can be widely applied to different types of speaker datasets.
Two-Step CNN Framework for Text Line Recognition in Camera-Captured Images

In this paper, we introduce an “on the device” text line recognition framework that is designed for mobile or embedded systems. We consider per-character segmentation as a language-independent problem and individual character recognition as a language-dependent one. Thus, the proposed solution is based on two separate artificial neural networks (ANN) and dynamic programming instead of employing image processing methods for the segmentation step or end-to-end ANN. To satisfy the tight constraints on memory size imposed by embedded systems and to avoid overfitting, we employ ANNs with a small number of trainable parameters. The primary purpose of our framework is the recognition of low-quality images of identity documents with complex backgrounds and a variety of languages and fonts. We demonstrate that our solution shows high recognition accuracy on natural datasets even being trained on purely synthetic data. We use MIDV-500 and Census 1961 Project datasets for text line recognition. The proposed method considerably surpasses the algorithmic method implemented in Tesseract 3.05, the LSTM method (Tesseract 4.00), and unpublished method used in the ABBYY FineReader 15 system. Also, our framework is faster than other compared solutions. We show the language-independence of our segmenter with the experiment with Cyrillic, Armenian, and Chinese text lines.
Data Sampling Methods to Deal With the Big Data Multi-Class Imbalance Problem

The class imbalance problem has been a hot topic in the machine learning community in recent years. Nowadays, in the time of big data and deep learning, this problem remains in force. Much work has been performed to deal to the class imbalance problem, the random sampling methods (over and under sampling) being the most widely employed approaches. Moreover, sophisticated sampling methods have been developed, including the Synthetic Minority Over-sampling Technique (SMOTE), and also they have been combined with cleaning techniques such as Editing Nearest Neighbor or Tomek’s Links (SMOTE+ENN and SMOTE+TL, respectively). In the big data context, it is noticeable that the class imbalance problem has been addressed by adaptation of traditional techniques, relatively ignoring intelligent approaches. Thus, the capabilities and possibilities of heuristic sampling methods on deep learning neural networks in big data domain are analyzed in this work, and the cleaning strategies are particularly analyzed. This study is developed on big data, multi-class imbalanced datasets obtained from hyper-spectral remote sensing images. The effectiveness of a hybrid approach on these datasets is analyzed, in which the dataset is cleaned by SMOTE followed by the training of an Artificial Neural Network (ANN) with those data, while the neural network output noise is processed with ENN to eliminate output noise; after that, the ANN is trained again with the resultant dataset. Obtained results suggest that best classification outcome is achieved when the cleaning strategies are applied on an ANN output instead of input feature space only. Consequently, the need to consider the classifier’s nature when the classical class imbalance approaches are adapted in deep learning and big data scenarios is clear.
Locate, Size and Count: Accurately Resolving People in Dense Crowds via Detection

We introduce a detection framework for dense crowd counting and eliminate the need for the prevalent density regression paradigm. Typical counting models predict crowd density for an image as opposed to detecting every person. These regression methods, in general, fail to localize persons accurate enough for most applications other than counting. Hence, we adopt an architecture that locates every person in the crowd, sizes the spotted heads with bounding box and then counts them. Compared to normal object or face detectors, there exist certain unique challenges in designing such a detection system. Some of them are direct consequences of the huge diversity in dense crowds along with the need to predict boxes contiguously. We solve these issues and develop our LSC-CNN model, which can reliably detect heads of people across sparse to dense crowds. LSC-CNN employs a multi-column architecture with top-down feature modulation to better resolve persons and produce refined predictions at multiple resolutions. Interestingly, the proposed training regime requires only point head annotation, but can estimate approximate size information of heads. We show that LSC-CNN not only has superior localization than existing density regressors, but outperforms in counting as well. The code for our approach is available at https://github.com/val-iisc/lsc-cnn.
Attention Based On-Device Streaming Speech Recognition with Large Speech Corpus

In this paper, we present a new on-device automatic speech recognition (ASR) system based on monotonic chunk-wise attention (MoChA) models trained with large (> 10K hours) corpus. We attained around 90% of a word recognition rate for general domain mainly by using joint training of connectionist temporal classifier (CTC) and cross entropy (CE) losses, minimum word error rate (MWER) training, layer-wise pretraining and data augmentation methods. In addition, we compressed our models by more than 3.4 times smaller using an iterative hyper low-rank approximation (LRA) method while minimizing the degradation in recognition accuracy. The memory footprint was further reduced with 8-bit quantization to bring down the final model size to lower than 39 MB. For on-demand adaptation, we fused the MoChA models with statistical n-gram models, and we could achieve a relatively 36% improvement on average in word error rate (WER) for target domains including the general domain.
Twitter Bot Detection Using Bidirectional Long Short-Term Memory Neural Networks and Word Embeddings

Twitter is a web application playing dual roles of online social networking and micro-blogging. The popularity and open structure of Twitter have attracted a large number of automated programs, known as bots. Legitimate bots generate a large amount of benign contextual content, i.e., tweets delivering news and updating feeds, while malicious bots spread spam or malicious contents. To assist human users in identifying who they are interacting with, this paper focuses on the classification of human and spambot accounts on Twitter, by employing recurrent neural networks, specifically bidirectional Long Short-term Memory (BiLSTM), to efficiently capture features across tweets. To the best of our knowledge, our work is the first that develops a recurrent neural model with word embeddings to distinguish Twitter bots from human accounts, that requires no prior knowledge or assumption about users' profiles, friendship networks, or historical behavior on the target account. Moreover, our model does not require any handcrafted features. The preliminary simulation results are very encouraging. Experiments on the cresci-2017 dataset show that our approach can achieve competitive performance compared with existing state-of-the-art bot detection systems.
Deep sentiments in Roman Urdu text using Recurrent Convolutional Neural Network model

Although over 64 million people worldwide speak Urdu language and are well aware of its Roman script, limited research and efforts have been made to carry out sentiment analysis and build language resources for the Roman Urdu language. This article proposes a deep learning model to mine the emotions and attitudes of people expressed in Roman Urdu - consisting of 10,021 sentences from 566 online threads belonging to the following genres: Sports; Software; Food & Recipes; Drama; and Politics. The objectives of this research are twofold: (1) to develop a human-annotated benchmark corpus for the under-resourced Roman Urdu language for the sentiment analysis; and (2) to evaluate sentiment analysis techniques using the Rule-based, N-gram, and Recurrent Convolutional Neural Network (RCNN) models. Using Corpus, annotated by three experts to be positive, negative, and neutral with 0.557 Cohen's Kappa score, we run two sets of tests, i.e., binary classification (positive and negative) and tertiary classification (positive, negative and neutral). Finally, the results of the RCNN model are analyzed by comparing it with the outcome of the Rule-based and N-gram models. We show that the RCNN model outperforms baseline models in terms of accuracy of 0.652 for binary classification and 0.572 for tertiary classification.
Characteristics of a Blockchain Ecosystem for Secure and Sharable Electronic Medical Records

Blockchain technology is a secure and distributed information accounting, storage, and retrieval modality which has the ability to disrupt and revolutionize business practices. One such disruption resides in the capability of Blockchain to serve as a secure method for storing and sharing electronic medical records in new and innovative ways. Current medical record storage and transmission methods are proprietary and have interoperability and security problems. To that end, secure, effective, and interoperable electronic records options are highly prized. This article uses a grounded theory approach to qualitative analysis of electronic medical records users in the United States to develop a perspective from industry and scholarly practice on the suitability of Blockchain technologies for electronic medical records, security, and storage.
A survey of security and privacy issues in the Internet of Things from the layered context

Summary Internet of Things (IoT) is a novel paradigm, which not only facilitates a large number of devices to be ubiquitously connected over the Internet but also provides a mechanism to remotely control these devices. The IoT is pervasive and is almost an integral part of our daily life. These connected devices often obtain user's personal data and store it online. The security of collected data is a big concern in recent times. As devices are becoming increasingly connected, privacy and security issues become more and more critical and these need to be addressed on an urgent basis. IoT implementations and devices are eminently prone to threats that could compromise the security and privacy of the consumers, which, in turn, could influence its practical deployment. In recent past, some research has been carried out to secure IoT devices with an intention to alleviate the security concerns of users. There have been research on blockchain technologies to tackle the privacy and security issues of the collected data in IoT. The purpose of this paper is to highlight the security and privacy issues in IoT systems. To this effect, the paper examines the security issues at each layer in the IoT protocol stack, identifies the under‐lying challenges and key security requirements and provides a brief overview of existing security solutions to safeguard the IoT from the layered context.
Student teachers’ responsible use of ICT: Examining two samples in Spain and Norway

Information and communication technology (ICT) has become an important component of initial teacher education (ITE) in Europe and in the continuous professional development of practicing teachers. The development of professional digital competence (PDC) is emerging as an essential part of teacher education. Due to the increasing use of ICT and the growing number of online teaching and learning resources, the responsible use of ICT has become one of the key aspects of PDC. For the purpose of this paper, the responsible use of ICT includes privacy issues, cyberbullying and the ability to evaluate digital content. We examine Spanish and Norwegian student teachers' perceived competence in privacy issues and in handling cyberbullying and their ability to evaluate digital content. In a survey conducted in autumn 2017, 681 Spanish and 563 Norwegian first-year student teachers in Spain and Norway answered questions on the responsible use of ICT. The findings show that in both countries the three concepts are recognised as distinct and that there is a positive relationship between student teachers' perceived understanding of the concepts. This implies that these concepts should be taught as separate components of PDC. However, it is challenging to compare student teachers’ perceived knowledge of the concepts across two countries and to create an integration model that fit both countries. This is partly due to cultural and language differences. The study provides a baseline in terms of knowledge about responsible use at the participating universities. It also details general implications for policy, practice and ITE programmes.
Plaintext-related image encryption algorithm based on perceptron-like network

A perceptron-like network based on integer domain was proposed, and then a new image cryptosystem based on the network was proposed in this paper. Different from the structure of classical image cryptosystem, the proposed image cryptosystem employed the perceptron-like network to achieve the diffusion of image information. It firstly generated the pseudo-random key streams from the secret key through chaotic system and S-box of AES. Then it obscured the sequence of plain image by XOR operation. Next, it used two perceptron-like networks to fulfill the information memory and diffusion of the image sequence. Finally, it obscured the diffused sequence using the equivalent keys to obtain the cipher image sequence. Performance analysis and comparison analysis show that the proposed image cryptosystem has the advantages of fast processing speed, strong system sensitivity and high security, and can be applied to the actual image communications.
Qsparse-Local-SGD: Distributed SGD With Quantization, Sparsification, and Local Computations

Communication bottleneck has been identified as a significant issue in distributed optimization of large-scale learning models. Recently, several approaches to mitigate this problem have been proposed, including different forms of gradient compression or computing local models and mixing them iteratively. In this paper, we propose Qsparse-local-SGD algorithm, which combines aggressive sparsification with quantization and local computation along with error compensation, by keeping track of the difference between the true and compressed gradients. We propose both synchronous and asynchronous implementations of Qsparse-local-SGD. We analyze convergence for Qsparse-local-SGD in the distributed setting for smooth non-convex and convex objective functions. We demonstrate that Qsparse-local-SGD converges at the same rate as vanilla distributed SGD for many important classes of sparsifiers and quantizers. We use Qsparse-local-SGD to train ResNet-50 on ImageNet and show that it results in significant savings over the state-of-the-art, in the number of bits transmitted to reach target accuracy.
GazeR: A Package for Processing Gaze Position and Pupil Size Data

Eye-tracking is widely used throughout the scientific community, from vision science and psycholinguistics to marketing and human-computer interaction. Surprisingly, there is little consistency and transparency in preprocessing steps, making replicability and reproducibility difficult. To increase replicability, reproducibility, and transparency, a package in R (a free and widely used statistical programming environment) called gazeR was created to read and preprocess two types of data: gaze position and pupil size. For gaze position data, gazeR has functions for reading in raw eye-tracking data, formatting it for analysis, converting from gaze coordinates to areas of interest, and binning and aggregating data. For data from pupillometry studies, the gazeR package has functions for reading in and merging multiple raw pupil data files, removing observations with too much missing data, eliminating artifacts, blink identification and interpolation, subtractive baseline correction, and binning and aggregating data. The package is open-source and freely available for download and installation: https://github.com/dmirman/gazer . We provide step-by-step analyses of data from two tasks exemplifying the package's capabilities.
Human Actions Tracking and Recognition Based on Body Parts Detection via Artificial Neural Network

Human body action recognition has drawn a good deal of interest in the community of computer vision, owing to its wide range of applications. Recently, the video / image sequence base action recognition techniques are believed to be ideal for its efficiency and lower cost compared to other techniques such as the ambient sensor and the wearable sensor. However, given to a large amount of variation in human pose and image quality, reliable detection of human action is still a very challenging job for scientists. In this document, we used linear discriminant analysis for the generation of features from the body parts detected. The primary goal of this study is to combine linear discriminant analysis with an artificial neural network for precise human action detection and recognition. Our proposed mechanism detects complicated human actions in two state-of-the-art datasets, i.e. KTH-dataset and Weizmann Human Action. We obtained multidimensional features from twelve body parts, which are estimated from body models. These multidimensional characteristics are used as inputs for the artificial neural network. To access the efficiency of our suggested method, we compared the outcomes with other state-of-the-art classifiers. Experimental results show that our proposed technique is reliable and applicable in health exercise systems, smart surveillance, e-learning, abnormal behavioral detection, protection for child abuse, care of the elderly people, virtual reality, intelligent image retrieval and human computer interaction.
Semantic Image Manipulation Using Scene Graphs

Image manipulation can be considered a special case of image generation where the image to be produced is a modification of an existing image. Image generation and manipulation have been, for the most part, tasks that operate on raw pixels. However, the remarkable progress in learning rich image and object representations has opened the way for tasks such as text-to-image or layout-to-image generation that are mainly driven by semantics. In our work, we address the novel problem of image manipulation from scene graphs, in which a user can edit images by merely applying changes in the nodes or edges of a semantic graph that is generated from the image. Our goal is to encode image information in a given constellation and from there on generate new constellations, such as replacing objects or even changing relationships between objects, while respecting the semantics and style from the original image. We introduce a spatio-semantic scene graph network that does not require direct supervision for constellation changes or image edits. This makes it possible to train the system from existing real-world datasets with no additional annotation effort.
The survey: Text generation models in deep learning

Deep learning methods possess many processing layers to understand the stratified representation of data and have achieved state-of-art results in several domains. Recently, deep learning model designs and architectures have unfolded in the context of Natural Language Processing (NLP). This survey presents a brief description of the advances that have occurred in the area of Deep Generative modeling. This work considers most of the papers from 2015 onwards. In this paper, we review many deep learning models that have been used for the generation of text. We also summarize the various models and have put forward the detailed understanding of past, present, and future of text generation models in deep learning. Furthermore, DL approaches that have been explored and evaluated in different application domains in NLP are included in this survey.
FedOpt: Towards Communication Efficiency and Privacy Preservation in Federated Learning

Artificial Intelligence (AI) has been applied to solve various challenges of real-world problems in recent years. However, the emergence of new AI technologies has brought several problems, especially with regard to communication efficiency, security threats and privacy violations. Towards this end, Federated Learning (FL) has received widespread attention due to its ability to facilitate the collaborative training of local learning models without compromising the privacy of data. However, recent studies have shown that FL still consumes considerable amounts of communication resources. These communication resources are vital for updating the learning models. In addition, the privacy of data could still be compromised once sharing the parameters of the local learning models in order to update the global model. Towards this end, we propose a new approach, namely, Federated Optimisation (FedOpt) in order to promote communication efficiency and privacy preservation in FL. In order to implement FedOpt, we design a novel compression algorithm, namely, Sparse Compression Algorithm (SCA) for efficient communication, and then integrate the additively homomorphic encryption with differential privacy to prevent data from being leaked. Thus, the proposed FedOpt smoothly trade-offs communication efficiency and privacy preservation in order to adopt the learning task. The experimental results demonstrate that FedOpt outperforms the state-of-the-art FL approaches. In particular, we consider three different evaluation criteria; model accuracy, communication efficiency and computation overhead. Then, we compare the proposed FedOpt with the baseline configurations and the state-of-the-art approaches, i.e., Federated Averaging (FedAvg) and the paillier-encryption based privacy-preserving deep learning (PPDL) on all these three evaluation criteria. The experimental results show that FedOpt is able to converge within fewer training epochs and a smaller privacy budget.