Reviewerly/exHarmony huggingface dataset:
Machine Learning Aided Minimal Sensor based Hand Gesture Character Recognition

Hand gesture recognition is the process of detecting the hand movements via sensor measurements for detecting an activity, such as writing a letter or a number. Recognising the handwritten characters using wearable devices enables machine-human interaction to occur without the need for a communication method. An intelligent automated framework is required to accurately detect the handwritten characters using wrist worn sensor signals, in particular, with minimal number of sensors. Moreover, the system developed needs to have the capacity to recognise the characters written in different sizes. In order to address these, we analyse performance of several machine learning models using single/multiple sensors namely, accelerometer or/and gyroscope, for recognising hand gesture characters including alphabet and numbers of varying sizes. We formulate a set of features that enable robust and accurate detection of the characters.We performed novel data collection using an off-the-shelf wrist-worn sensor based device, and evaluated our framework to detect the different characters effectively. The maximum accuracy (90.40%) was achieved using both sensors and Random Forest (RF) model. This was dropped to 82.51% for the same model using accelerometer sensor alone. Using the gyroscope sensor, an overall average accuracy of 80.16% was achieved with the Forward Neural Network (FNN) model. Although the model based on both sensors showed the best performance, our evaluation reveals that it is feasible to develop a machine learning model using single sensor to detect hand gesture characters of varying sizes with reasonable (≥ 80%) accuracy.
Improving Graph Neural Network with Learnable Permutation Pooling

Graph neural networks (GNN) have achieved great success in various graph-related applications. Most existing graph neural network models follow the message-passing neural network (MPNN) paradigm where the graph pooling function forms a critical component that directly determines the model effectiveness. In this paper, we propose PermPool, a new graph pooling function that provably improves the GNN model expressiveness. The method is based on the insight that the distribution of node permuations, when defined properly, forms characteristic encoding of graphs. We propose to express graph representations as the expectation of node permutations with a general pooling function. We show that the graph representation remains invariant to node-reordering and has strong expressive power than MPNN models. In addition, we propose novel permutation modeling and sampling techniques that integrate PermPool into the differentiable neural network models. Empirical results show that our method outperformed other pooling methods in benchmark graph classification tasks.
Low Pass Filtering and Bandwidth Extension for Robust Anti-spoofing Countermeasure Against Codec Variabilities

A reliable voice anti-spoofing countermeasure system needs to robustly protect automatic speaker verification (ASV) systems in various kinds of spoofing scenarios. However, the performance of countermeasure systems could be degraded by channel effects and codecs. In this paper, we show that using the low-frequency subbands of signals as input can mitigate the negative impact introduced by codecs on the countermeasure systems. To validate this, two types of low-pass filters with different cut-off frequencies are applied to countermeasure systems, and the equal error rate (EER) is reduced by up to 25% relatively. In addition, we propose a deep learning based bandwidth extension approach to further improve the detection accuracy. Recent studies show that the error rate of countermeasure systems increase dramatically when the silence part is removed by Voice Activity Detection (VAD), our experimental results show that the filtering and bandwidth extension approaches are also effective under the codec condition when VAD is applied.
A Paraphrase Identification Approach in Paragraph Length Texts

Measuring the semantic similarity of natural language is a fundamental issue in many tasks, such as paraphrase identification (PI) and plagiarism detection (PD) which are intended to solve maj or issues in education. Various approaches that have been suggested to tackle this issue, including machine learning (ML) and deep learning (DL) methods. Unlike prior research, where the focus has been on detecting paraphrases in short and sentence-level texts, we focus on the not yet explored area of paraphrase detection in paragraphs. We consider that the meaning of a piece of text can be broken into more than one sentence, which is over and beyond sentences as extracted from two benchmark datasets (Webis-CPC-ll and MSRP). We use TF-IDF, Bleu metric, N-gram overlap, and Word2vec as features, and then invoke SVM as a classifier. The contribution of this paper clearly indicates that, on a commonly used evaluation set, considering text at the length of a paragraph is more appropriate than considering short or long text for ML and DL approaches. Additionally, our method outperforms the existing work done on the Webis-CPC-ll dataset.
Power of Explanations: Towards automatic debiasing in hate speech detection

Hate speech detection is a common downstream application of natural language processing (NLP) in the real world. In spite of the increasing accuracy, current data-driven approaches could easily learn biases from the imbalanced data distributions originating from humans. The deployment of biased models could further enhance the existing social biases. But unlike handling tabular data, defining and mitigating biases in text classifiers, which deal with unstructured data, are more challenging. A popular solution for improving machine learning fairness in NLP is to conduct the debiasing process with a list of potentially discriminated words given by human annotators. In addition to suffering from the risks of overlooking the biased terms, exhaustively identifying bias with human annotators are unsustainable since discrimination is variable among different datasets and may evolve over time. To this end, we propose an automatic misuse detector (MiD) relying on an explanation method for detecting potential bias. And built upon that, an end-to-end debiasing framework with the proposed staged correction is designed for text classifiers without any external resources required.
Scalable N-Queens Solving on GPGPUs via Interwarp Collaborations

In this paper we present how recent hardware revisions and newly introduced approaches to thread collaboration in NVIDIA GPUs and the CUDA toolkit can be used to design an extensible, scalable GPU-based solver for the N-Queens problem. We discuss various design choices ranging from memory structure, to low-level optimisations on newer GPU hardware that result in strong performance when solving the N-Queens problem using an optimised solving algorithm that can be applied to other similar in nature problems.
A Probabilistic Framework for Adapting to Changing and Recurring Concepts in Data Streams

The distribution of streaming data often changes over time as conditions change, a phenomenon known as concept drift. Only a subset of previous experience, collected in similar conditions, is relevant to learning an accurate classifier for current data. Learning from irrelevant experience describing a different concept can degrade performance. A system learning from streaming data must identify which recent experience is irrelevant when conditions change and which past experience is relevant when concepts reoccur, e.g., when weather events or financial patterns repeat. Existing streaming approaches either do not consider experience to change in relevance over time and thus cannot handle concept drift, or only consider the recency of experience and thus cannot handle recurring concepts, or only sparsely evaluate relevance and thus fail when concept drift is missed. To enable learning in changing conditions, we propose SELeCT, a probabilistic method for continuously evaluating the relevance of past experience. SELeCT maintains a distinct internal state for each concept, representing relevant experience with a unique classifier. We propose a Bayesian algorithm for estimating state relevance, combining the likelihood of drawing recent observations from a given state with a transition pattern prior based on the system’s current state. The current state is continuously maintained using a Hoeffding bound based algorithm, which unlike existing methods, guarantees that every observation is classified using the state estimated as the most relevant, while also maintaining temporal stability. We find SELeCT is able to choose experience relevant to ground truth concepts with recall and precision above 0.9, significantly outperforming existing methods and close to a theoretical optimum, leading to significantly higher accuracy and enabling new opportunities for learning in complex changing conditions.
Deep Heterogeneous Graph Neural Networks via Similarity Regularization Loss and Hierarchical Fusion

Recently, Graph Neural Networks (GNNs) have emerged as a promising and powerful method for tackling graph-structured data. However, most real-world graph-structured data contains distinct types of objects (nodes) and links (edges), which is called heterogeneous graph. The heterogeneity and rich semantic information indeed increase the difficulties in handling heterogeneous graph. Most of the current heterogeneous graph neural networks (HeteGNNs) can only build on a very shallow structure. This is caused by a phenomenon called semantic confusion, where the node embeddings become indistinguishable with the growth of model depth, leading to the degradation of the model performance. In this paper, we address this problem by proposing a similarity regularization loss and hierarchical fusion based heterogeneous graph neural networks (SHGNN). The hierarchical fusion strategy is utilized to fuse the features of the node embeddings at each layer, which can improve the expressive power of the model, and then a similarity regularization loss is introduced, by which the problem of indistinguishability among nodes can be alleviated. Our approach is flexible to combine various HeteGNNs effectively. Experimental results on real-world heterogeneous graph-structured data demonstrate the state-of-the-art performance of the proposed approach, which can efficiently mitigate the semantic confusion problem.
Discovering Periodicity in Locally Repeating Patterns

Analysing and learning from sequentially ordered symbolic data is increasingly important in applications such as finance and biology. Finding patterns that show interesting behaviour, such as regularly repeating occurrences within a time interval, can provide useful insight. In this paper we address the problem of efficiently identifying such behaviours. Existing approaches often require a target period to be specified, which will limit possible patterns to those approximating the specified periodicity, such as daily, monthly, quarterly and so on. In this paper we extend one such approach, derived from frequent pattern mining, to operate without the need for user-specified periodicity. Our new algorithms can identify the time interval and periodicity, or frequency of occurrence, of all periodically occurring patterns within a certain used-specified tolerance. Experimental results of our implementation show that the new approach can identify many more patterns in a real-world financial dataset, while on other sequential datasets it finds similar numbers of patterns without significant reduction in efficiency compared to existing approaches. We also verify the algorithm’s ability to recover recurring patterns in controlled experiments on synthetic data.
A Malay Language Cyberbullying Detection Model on Twitter using Supervised Machine Learning

This research detects cyberbullying for the Malay language using supervised machine learning (ML) and Natural Language Processing (NLP). Due to the high number of cyberbullying cases in Malaysia over the years and the belief that there is an increased number of unreported cyberbullying cases, there needs an intelligent way to detect cyberbullying on social media. Thus, this research explores how supervised ML and NLP can help detect cyberbullying incidents for the Malay language on social media. The dataset was collected from Twitter by scrapping tweets based on some common Malay words used in cyberbullying incidents before being labelled into six cyberbullying classes: appearance, intellectual, political, racial, sexual, and non-abusive. The resulting cyberbullying dataset is an imbalanced dataset with 45,580 tweets. The model is then built using Logistic Regression (LR), Naïve Bayes (NB), Support Vector Machine (SVM) and Random Forest (RF) algorithms combined with three different feature extraction techniques, that is Bag of Words (BoW), Term Frequency-Inverse Document Frequency (TF-IDF) and Word2Vec. The result indicates that the best model uses LR combined with the TF-IDF feature extraction technique. The model was improved further by using an oversampling technique (Synthetic Minority Oversampling Technique, SMOTE) to deal with the imbalanced dataset and tuning the model hyperparameters. The F-Score of the optimised TF-IDF – LR is 0.46.
Diagonally Colorized iVAT Images for Labeled Data

The iVAT (improved Visual Assessment of cluster Tendency) image is a useful tool for assessing possible cluster structure in an unlabeled, numerical data set. If labeled data are available then it is sometimes helpful to determine how closely the (unlabeled) data clusters agree with the data partitioning based on the labels. In this note the DCiVAT (Diagonally Colorized iVAT) image is introduced for the case of labeled data. It incorporates all available data and label information into a single colorized iVAT image so that it is possible to visually assess the degree to which data clusters are aligned with label categories. The new approach is illustrated with several examples.
Evidence Based Pipeline for Explaining Artificial Intelligence Algorithms with Interactions

Artificial intelligence (AI) enables machines to learn from human experience, adjust to new inputs, and perform intelligent tasks without human intervention. AI is progressing rapidly and is transforming the way businesses operate, from process automation to cognitive augmentation of tasks and intelligent process/data analytics. However, the main challenge for the AI system users is to comprehend and trust the result of AI algorithms and methods. To address this challenge, we first study the recent techniques in the area of eXplainable Artificial Intelligence (XAI). Then, we introduce a novel XAI process to facilitate producing explainable models while maintaining a high level of learning performance. We present an interactive evidence-based approach to assist the users in comprehending and trusting the results and outputs generated by AI-enabled algorithms, resulting in developing a digital dashboard to facilitate inter-acting with the algorithm. Lastly, we discuss how the proposed XAI method can significantly improve the confidence of data scientists in understanding the result of AI-enabled algorithms with an application in the banking domain for analyzing customer transactions.
Efficient Conformer-Based CTC Model for Intelligent Cockpit Speech Recognition

In this paper, we discuss the rationale of our work for automatic speech recognition (ASR) in the ISCSLP 2022 Intelligent Cockpit Speech Recognition Challenge and provide a detailed description. We proposed a combination of intermediate CTC-based loss regularization(Inter-CTC) with self-conditioned folded encoders, also deployed with TLG attention-rescore-based acoustic model. This combination finds good solutions for Intelligent Cockpit Speech recognition, as it makes accuracy improvements. The Character Error Rate (CER) of the model in Track II (Unlimited model size track) decreased by 38.12% compared to the baseline model, achieving second place in the ranking period with a CER of 9.86%. For Track I (Limited model size track), we apply knowledge distillation to train the Teacher-Student model from the unlimited track, with a combining Track I and Track II Kloss. The CER of the Track II model drops by 40.18% from the baseline model, achieving third place in the ranking period with a CER of 13.39%.
LeVoice ASR Systems for the ISCSLP 2022 Intelligent Cockpit Speech Recognition Challenge

This paper describes LeVoice automatic speech recognition systems to track2 of intelligent cockpit speech recognition challenge 2022. Track2 is a speech recognition task without limits on the scope of model size. Our main points include deep learning based speech enhancement, text-to-speech based speech generation, training data augmentation via various techniques and speech recognition model fusion. We compared and fused the hybrid architecture and two kinds of end-toend architecture. For end-to-end modeling, we used models based on connectionist temporal classification/attention-based encoder-decoder architecture and recurrent neural network transducer/attention-based encoder-decoder architecture. The performance of these models is evaluated with an additional language model to reduce word error rates. As a result, our system achieved 10.2% character error rate on the challenge test set data and ranked third place among the submitted systems in the challenge.
Audio Splicing Localization: Can We Accurately Locate the Splicing Tampering?

Audio splicing is a low-cost and straightforward tampered form of audio fraud. Splicing localization is to locate where the splicing tampering happens. Although there are a few studies on audio splicing localization, most of them use a much smaller dataset and locate the splicing points within a long segment, which can not accurately reflect the real model performance in the real-world application. In this study, we revisit audio splicing localization by using a large-scale splicing dataset and how accurate the model can be if we narrow down the localization range. We employ a ResNet-based model for localization and develop a large-scale dataset from LibriSpeech for experiments. The dataset consists of more than 130k utterances, including insertion, deletion and substitution splicing tampering.
Adaptive Attention Network with Domain Adversarial Training for Multi-Accent Speech Recognition

Spoken accents severely degrade the performance of automatic speech recognition (ASR) systems. Domain adversarial training (DAT) is widely adopted for generating domain-invariant features to reduce the influence of accents. However, the generated features trained by DAT are still maintaining some accent discrimination information, limiting the ASR performance. In addition, the features generated by DAT of each accent have different degrees of residual accent discriminant information. In this paper, we propose an adaptive attention network with DAT to further eliminate the influence of retaining accent information in features generated by DAT. We employ the adaptive attention module to transform the encoder output to a more general representation. Experiments on the AESRC2020 dataset show that the proposed method can achieve satisfactory performance improvements on seen and unseen accent when the correct accent information is still preserved in the output of the encoder.
Label-free Knowledge Distillation with Contrastive Loss for Light-weight Speaker Recognition

Very deep models for speaker recognition (SR) have demonstrated remarkable performance improvement in recent research. However, it is impractical to deploy these models for on-device applications with constrained computational resources. On the other hand, light-weight models are highly desired in practice despite their sub-optimal performance. This research aims to improve light-weight SR models through large-scale label-free knowledge distillation (KD). Existing KD approaches for SR typically require speaker labels to learn task-specific knowledge, due to the inefficiency of conventional loss for distillation. To address the inefficiency problem and achieve label-free KD, we propose to employ the contrastive loss from self-supervised learning for distillation. Extensive experiments are conducted on a collection of public speech datasets from diverse sources. Results on light-weight SR models show that the proposed approach of label-free KD with contrastive loss consistently outperforms both conventional distillation methods and self-supervised learning methods by a significant margin.
Enabling Synthetic Data adoption in regulated domains

The switch from a Model-Centric to a Data-Centric mindset is putting emphasis on data and its quality rather than algorithms, bringing forward new challenges. In particular, the sensitive nature of the information in highly regulated scenarios needs to be accounted for. Specific approaches to address the privacy issue have been developed, as Privacy Enhancing Technologies. However, they frequently cause loss of information, putting forward a crucial trade-off among data quality and privacy. A clever way to bypass such a conundrum relies on Synthetic Data: data obtained from a generative process, learning the real data properties. Both Academia and Industry realized the importance of evaluating synthetic data quality: without all-round reliable metrics, the innovative data generation task has no proper objective function to maximize. Despite that, the topic remains under-explored. For this reason, we systematically catalog the important traits of synthetic data quality and privacy, and devise a specific methodology to test them. The result is DAISYnt (aDoption of Artificial Intelligence SYnthesis): a comprehensive suite of advanced tests, which sets a de facto standard for synthetic data evaluation. As a practical use-case, a variety of generative algorithms have been trained on real-world Credit Bureau Data. The best model has been assessed, using DAISYnt on the different synthetic replicas. Further potential uses, among others, entail auditing and fine-tuning of generative models or ensuring high quality of a given synthetic dataset. From a prescriptive viewpoint, eventually, DAISYnt may pave the way to synthetic data adoption in highly regulated domains, ranging from Finance to Healthcare, through Insurance and Education.
Fully Subliminal-Free Schnorr Signature for Nonce

Subliminal channel is a kind of covert channels that allows sender and receiver to transfer information without being realized by others. It has been pointed out that some digital signatures can be used as subliminal channel by embedding information into a nonce used in the signature generation algorithm. Subliminal-free signatures preventing subliminal channel have been studied extensively so far. There are two major approaches to make a digital signature subliminal-free. The first one modifies both the signature generation and signature verification algorithm to detect subliminal channels, and the second one only modifies signature generation. The latter approach is more challenging. In fact, there exists no scheme that guarantees fully subliminal-freeness against adversaries tries to embedded information into nonce, and allows adversaries to embed small amount of information into nonce. In this paper, we presents two subliminal-free signature based on Schnorr signature. The first one which we call PSF-Schnorr is partially subliminal-free. Based on PSF-Shcnorr signature, we presents SF-Schnorr signature that achieves fully subliminal-freeness. To make PSF-Schnorr signature subliminal-free, it is required that the signer can compute the hash value without knowing the input. In the proposed scheme, we employs fully homomorphic encryption (FHE) in computing hash value to fulfill the above-mentioned requirement, which will be of independent interest in the sense that the proposed scheme shows a novel application of FHE.