Reviewerly/exHarmony huggingface dataset:
Explaining Human Activities Instances Using Deep Learning Classifiers

The recognition of human activities in sensorized smart-home environments enables a wide variety of healthcare applications, including the detection of early symptoms of cognitive decline. The most effective Human Activity Recognition (HAR) methods are based on supervised Deep Learning classifiers. Those models are usually considered as black boxes, and the rationale behind their decisions is difficult to understand for human beings. The recent advances in eXplainable Artificial Intelligence (XAI) offer promising tools to make HAR models more transparent. The state-of-the-art explainable HAR methods provide explanations for the output of classifiers that periodically predict the performed activity on short time windows (usually in the range of 15-60 seconds). However, non-technical users may be more interested in investigating explanations associated with complete activity instances (e.g., an instance of the cooking activity may last 30 minutes). Unfortunately, temporally extending the time window harms the recognition rate of HAR classifiers. In this paper, we propose DeXAR++: a novel method that generates explanations for human activity instances based on deep learning classifiers. The sensor data time windows used for classification are encoded as images. DeXAR++ aggregates the explanations generated by a computer-vision XAI approach on each time window to obtain a single explanation for approximated activity instances. Moreover, DeXAR++ includes a novel visualization approach particularly suitable for non-expert users. We evaluate DeXAR++ with both automatic and user-based evaluation methodologies on a public dataset of activities performed in smart-home environments, showing that our results outperform the ones obtained by state-of-the-art methods.
Interpreting Categorical Data Classifiers using Explanation-based Locality

Local surrogate explanation methods are a popular class of post-hoc interpretability approaches that explain the rationale of machine learning models in the locality of every particular instance. Fidelity, which refers to the accuracy of explanation methods in imitating the actual behavior of a model, is highly affected by their strategy for identifying the locality of instances. To find the locality of an instance, we need to calculate the distance between the instance and perturbed data points concerning categorical and numerical features. While the distance of numerical features can be measured precisely, the existing works usually adopt a coarse-grained or imprecise approach for comparing categorical features. This is especially problematic in the categorical data setting, where defining a representative locality demands fine-grained semantic similarity information between categories. In this paper, we propose a locality generation approach for categorical data classifiers that makes no assumption about domain knowledge and infers categorical similarities by relying on the model's explanations. Further, we devise a multi-centered sampling approach based on the derived similarity information that, compared to the conventional instance-centered technique, captures the local behavior of the model more effectively. Moreover, we develop a knowledge-based locality generation approach based on knowledge graphs to benchmark our explanation-based method against a scenario where the similarity information is provided by a domain expert. The experiments conducted on various data sets demonstrate the efficacy of our approach in generating faithful explanations.
Federated Deep Recommendation System Based on Multi-View Feature Embedding

The application of recommendation systems online services is becoming more and more extensive. However, most existing recommendation algorithms centralize multi-party information into a central processor, which may lead to the risk of privacy leakage. And many enterprises or institutions still have the problem that data cannot be shared. Federated learning has been introduced into recommendation algorithms for privacy- aware distributed learning. A typical federated learning is that each client uses local data to train a shared model, the server uses their gradient information to form a global model, and then each client updates. In this paper, we propose a federated deep recommendation algorithm called FedHe-mlp that applies a federated deep learning for data privacy protection, and combines heterogeneous information network (HIN) and matrix factorization technique for better prediction performance. First, each client obtains heterogeneous information through meta- paths, then we combine matrix factorization and heterogeneous information to mine the latent features and heterogeneous features of each client. Finally, We propose a deep neural network that considers features from multiple views. Extensive experiments on three public datasets demonstrate that FedHe- mlp can provide excellent convergence speed, recommendation accuracy, and communication efficiency while preserving data privacy.
Research on Lightweight Pedestrian Detection Model in Complex Background

Pedestrian detection under complex background is a hot research content in the field of Computer Vision. Pedestrian detection in a real and complex environment is extremely important for Human Action Recognition, Intelligent Transportation, Video Surveillance Security and other security aspects. This paper proposes a lightweight model based on the YOLO algorithm for pedestrian detection in a complex background. First, we build a YOLO v3 model based on Darknet-53. Then in order to simplify the pedestrian detection model and ensure the performance of the model, this paper adopts the method of convolution channel pruning to lighten the model. This method realizes rapid and accurate detection of pedestrians in complex backgrounds. In order to verify the performance of the proposed model, this paper constructs a scientific and reasonable small dataset sample based on the YOLO model. The dataset contains 1505 images, covering various complex scenes such as intersections with crowded people and traffic at night, and commercial streets with complex buildings. The test results show that for a complex background dataset with a small number of samples, the model's mAP reached 87.89%, precision was 90.37%, and recall was 77.05%. In addition, this paper also compares the performance of YOLO v3 and YOLO v4-tiny models. The results show that the lightweight models proposed in this paper have strong robustness. It is feasible to apply this method to real-time and accurate detection of pedestrians in complex backgrounds.
J-TranPSP: A Joint Transition-based Model for Prosodic Structure Prediction, Word Segmentation and PoS Tagging

Text analysis is usually the first step in speech synthesis. It provides linguistic features as the inputs of the state-of-the-art acoustic models to generate speech. The linguistic features include word boundaries, part-of-speech (PoS) tags, prosodic phrase boundaries and phoneme sequences. Usually all acoustic models from different speakers share the same linguistic features. So the correctness of the linguistic features will profoundly affect the synthesized speech. In this paper, we first time applied transition-based model on prosodic structure prediction with two actions read and break. Moreover we proposed a simple but effective joint model on top of the ELECTRA encoder that simultaneously predicts word boundaries and PoS tags. Our unified model achieves comparable or better F1-scores than the state-of-the-art models on the three tasks.
Graph Summarization as Vertex Classification Task using Graph Neural Networks vs. Bloom Filter

The goal of graph summarization is to represent large graphs in a structured and compact way. A graph summary based on equivalence classes preserves predefined features of each vertex within a k-hop neighborhood, such as the vertex and edge labels. Based on these neighborhood characteristics, the vertex is assigned to an equivalence class. The calculation of the assigned equivalence class must be a permutation invariant operation on the predefined features. This is typically achieved by sorting on the feature values, which is computationally expensive, and subsequently hashing the result. Graph Neural Networks (GNNs) fulfill the permutation invariance requirement. We formulate the problem of graph summarization as a subgraph classification task on the root vertex of the k-hop neighborhood. We adapt different GNN architectures, both based on the popular message-passing protocol and alternative approaches, to perform the structural graph summarization task. We compare different GNNs with a standard multi-layer perceptron (MLP) and Bloom filter as a non-neural method. We consider four popular graph summary models on a large web graph. This resembles challenging multi-class vertex classification tasks with the numbers of classes ranging from 576 to hundreds of thousands. Our results show that the performance of GNNs are close to each other. In three out of four experiments, the non-message-passing Graph-MLP model outperforms the other GNNs. The performance of the standard MLP is extraordinarily good, especially in the presence of many classes. Finally, the Bloom filter outperforms all neural architectures by a large margin, except for the dataset with the fewest number (576) of classes. This is an interesting result, since it sheds light on how well and in which contexts GNNs are suited for graph summarization. Furthermore, it demonstrates the need for considering strong non-neural baselines for standard GNN tasks such as vertex classification.
Transform Domain and Singular Value Decomposition for Robust and Imperceptible Image Watermarking

As technology advances at an ever-increasing rate, data is disseminated online through multimedia, particularly images, which are relatively simple to access. However, this situation is prone to theft of digital data and copyright infraction. A method to preserve and maintain multimedia data information, particularly images, is the robust watermarking approach. The implanted watermark can be recovered even after being altered by numerous techniques and will be difficult to erase. The host image is altered by redundant wavelets, and the suggested watermarking approach incorporates watermarks situated in the low and high-frequency sub-bands of the curvelet transform to provide a fair trade-off between robustness and imperceptibility. Applying a discrete cosine transform to the watermark causes it to be integrated into the image. Then, using the singular value decomposition approach, the watermark’s unique value is added to the host image’s unique value. The watermarked image is extracted on the receiving side in order to return the watermark to its original state. According to the study’s findings, the scheme has a good imperceptibility and is resilient with a maximum peak signal to noise ratio value of 71.7630 dB, SSIM 1, NC 1, and BER 0. The suggested approach is resistant to attacks from Gaussian filter, rotation, and JPEG compression with noise that is (Gaussian, salt-and-pepper, speckle).
Malaysian Banknotes Counterfeit Detection Algorithm for Fifty Ringgit and One-hundred Ringgit

With the advent of high-definition color printing technology, the counterfeiting of Ringgit has become a formidable obstacle. Therefore, image processing techniques based on watermarks are vital for detecting counterfeit Ringgit. The fifty Ringgit and one hundred Ringgit counterfeit detection algorithms proposed in this work use fuzzy logic and Ultraviolet image processing techniques. The algorithm will first determine the inserted banknote’s currency values, perform banknote position detection and re-adjustment, locate Ultraviolet watermarks (Rectangle Text "BNM50" or "BNM100", Logo "Stag Motif", and Security Fibers) and then employ fuzzy IF-THEN conditional statements to infer and decide whether the slotted in banknote is a real fifty Ringgit, a real one hundred Ringgit, or neither.
End-to-end speech topic classification based on pre-trained model Wavlm

Speech topic classification (STC) is the task of automatically classifying audio segments into predefined categories, and has an increasingly wide application in the field of speech indexing, retrieval, surveillance, etc. Currently, the typical STC is a pipeline method consisting of automatic speech recognition (ASR), possible machine translation (MT), and text classification (TC). Although each component in the pipeline has a clear function and mature solutions, it suffers from error propagation and scarcity of annotated training data. To solve it, we propose a monolithic network based on pre-trained models to accomplish the speech topic classification task. The end-to-end training strategy based on the unified network structure avoids error propagation. And the pre-trained models reduce the requirements for a large amount of annotated data. Besides, the proposed method can take advantage of the intrinsic semantic feature of the speech for better performance. Our method carried out a series of experiments on the Fisher dataset. Compared with the traditional pipeline method, we also achieved an accuracy of 7 percentage points better than the traditional method without a large number of voice annotation data, so our method has huge advantages.
Domain Adaptation Based Multi-Source Data Fusion for Pattern Classification

Multi-source Unsupervised Domain Adaptation (MUDA) is an important and challenging research topic for target classification with the assistance of labeled data in source domains. When we have several labeled source domains, it is hard to map all source domains and target domain into a common feature space for well classifying the targets. A new Progressive Multi-Source Domain Adaptation Network (PMSDAN) is proposed to further improve the classification performance. PMSDAN mainly consists of two steps for distribution alignment. Firstly, the multiple source domains are integrated as one auxiliary domain to match the distribution with the target domain. In order to mine as much as possible assistance knowledge from each source domain, the distribution of the target domain will be separately aligned with that of each source domain. Finally, a weighted fusion method is employed to combine the multiple classification results for making the final classification decision. In the optimization of domain adaption, Weighted Hybrid Maximum Mean Discrepancy (WHMMD) is proposed, and it considers both the inter-class and intra-class Discrepancy. The experiments show that the proposed method can obtain remarkable results for MUDA on public benchmark dataset compared to some state-of-the-art methods.
Joint Detection and State Estimation of Doppler Radar

In order to improve the performance of Doppler echo signal's detection and parameter estimation under low signal-to-noise ratio (SNR), a joint detection and state estimation algorithm based on random finite set is proposed. Under the framework of formal Bernoulli filter, the target state model and observation model are designed at the Doppler echo signal level, which overcomes the problem that the traditional Detect-Before-track (DBT) method fails due to the loss of target information in the case of low SNR. The time update and observation update of state vector are realized by sequential Monte Carlo (SMC), and the joint detection and state parameter estimation of single target are realized. Simulation results show that the algorithm can effectively improve the performance of detection and state estimation.
Scene and Texture Based Feature Set for DeepFake Video Detection

The existence of fake videos is a problem that is challenging today's social media-enabled world. There are many classifications for fake videos with one of the most popular being DeepFakes. Detecting such fake videos is a challenging issue. This research attempts to comprehend the characteristics that belong to DeepFake videos. In attempting to understand DeepFake videos this work investigates the characteristics of the video that make them unique. As such this research uses scene and texture detection to develop a unique feature set containing 19 data features which is capable of detecting whether a video is a DeepFake or not. This study validates the feature set using a standard dataset of the features relating to the characteristics of the video. These features are analysed using a classification machine learning model. The results of these experiments are examined using four evaluation methodologies. The analysis reveals positive performance with the use of the ML method and the feature set. From these results, it can be ascertained that using the proposed feature set, a video can be predicted as a DeepFake or not and as such prove the hypothesis that there exists a correlation between the characteristics of a video and its genuineness, i.e., whether or not a video is a DeepFake.
LSP: Acceleration of Graph Neural Networks via Locality Sensitive Pruning of Graphs

Graph Neural Networks (GNNs) have emerged as highly successful tools for graph-related tasks. However, real- world problems involve very large graphs, and the compute resources needed to fit GNNs to those problems grow rapidly. Moreover, the noisy nature and size of real-world graphs cause GNNs to over-fit if not regularized properly. Surprisingly, recent works show that large graphs often involve many redundant components that can be removed without compromising the per- formance too much. This includes node or edge removals during inference through GNNs layers or as a pre-processing step that sparsifies the input graph. This intriguing phenomenon enables the development of state-of-the-art GNNs that are both efficient and accurate. In this paper, we take a further step towards demystifying this phenomenon and propose a systematic method called Locality-Sensitive Pruning (LSP) for graph pruning based on Locality-Sensitive Hashing. We aim to sparsify a graph so that similar local environments of the original graph result in similar environments in the resulting sparsified graph, which is an essential feature for graph-related tasks. To justify the application of pruning based on local graph properties, we exemplify the advantage of applying pruning based on locality properties over other pruning strategies in various scenarios. Extensive experiments on synthetic and real-world datasets demonstrate the superiority of LSP, which removes a significant amount of edges from large graphs without compromising the performance, accompanied by a considerable acceleration.
Model Personalization with Static and Dynamic Patients' Data

Recent progress in machine learning has been ac-companied by the availability of large amount of data in many do-mains. The clinical domain is no exception, comprehensive patient information covering their whole treatment journeys is stored in Electronic Health Records (EHRs). Still, there is a strong heterogeneity among patients that is difficult to capture using standard supervised machine learning tools. Few-shot learning methods, developed for limited data regimes, are foremost trained to quickly adapt to new tasks. In the clinical setting a task is a patient, adapting to the task encompasses optimizing predictions for the patient at hand, and thus personalizing predictions. In this work, we utilize MAML, a meta-learning approach for few-shot learning, for personalizing models in a challenging setting. We assume patients' data to comprise a static part and a dynamic part. Our goal is to investigate how well models trained to quickly adapt to specific patients perform compared to models trained in a conventional manner in this setting. Additionally, we study how static patients' data can be leveraged to acquire personalized models early on, and how these models adapt as more dynamic data are available. Our experiments show that models learned by MAML do indeed make better predictions when adapted to specific patients, and that leveraging static attributes for personalization does lead to an improved performance at best and does not hurt performance at worst. Our findings suggest that few-shot learning has a promising potential in challenging clinical settings, where only limited information are available about patients and where personalized medicine is beneficial.
DeepDive: Deep Latent Factor Model for Enhancing Diversity in Recommender Systems

Most collaborative filtering techniques concentrate on increasing the accuracy of business-to-customer recommender systems. Emphasis on accuracy alone leads to repetitive recommendations based on user's past preferences; such predictions pose a problem from both business and user's perspective as they fail to recommend niche items and maintain the user's interest. Incorporating diversity in recommendations can overcome these issues. Most prior studies include diversity by randomizing the item-set predicted by the collaborating filtering technique. These techniques do not have control over the accuracy vs. diversity trade-off; one needs to be mindful that a drastic loss in accuracy is not acceptable from the recommender system. Our work proposes a deep latent factor model with a diversity cost/penalty that allows us to control the trade-off between diversity and accuracy. Experimental results obtained with the Movielens dataset demonstrate the superior performance of our proposed method in providing relevant, novel, and diverse recommendations compared to state-of-the-art techniques; with a slight drop in accuracy, our proposed method provides an improvement in different established measures of diversity.
A Mandarin Prosodic Boundary Prediction Model Based on Multi-Source Semi-Supervision

High-quality prosodic boundary prediction plays an important role in enhancing speech naturalness and intelligibility in Mandarin text-to-speech tasks. However traditional methods usually require a large amount of token-level labels, which can hardly be applied in low-resource scenarios. In this paper, to solve this problem, we propose a multi-source semi-supervised model using an HMM to assist BERT-based prosody prediction. Our proposed model implements an alternate training mechanism combining BERT-Prosody and HMM, where BERT takes denoised labels from HMM, providing updated character embedding and weak labels for the latter to form a training cycle. Experimental results show that, compared with baseline methods, the F1 score of our model is raised by 1.01%/8.25% respectively at prosodic word/phrase level, approaching the performance of supervised models.
Constraint-based Sequential Rule Mining

Sequential rule mining (SRM) is an alternative to sequential pattern mining (SPM) when dealing with sequence data. SRM has a wide range of applications in numerous data analysis scenarios. Existing SRM algorithms usually discover the entire set of rules in the databases, which makes it not only difficult to analyze results because the discovered set is too large, but also does not consider the user’s expectations and background knowledge. To tackle this problem, researchers have explored related algorithms with different constraints according to their requirements. In this paper, we propose a flexible constraint-based SRM algorithm called ConSRM for discovering only the sequential rules within user-specified time bounds in a sequence database. This algorithm uses an efficient rule-growth method and develops corresponding constraints and pruning strategies to reduce the search space and speed up calculation. Comprehensive experiments were carried out on four real datasets to evaluate the performance (both effectiveness and efficiency) of ConSRM.