Reviewerly/exHarmony huggingface dataset:
Wavelet Dual-Stream Network for Brain MR Image Super-Resolution

High-resolution (HR) magnetic resonance (MR) images provide more detailed information for reliable diagnoses and quantitative medical image analyses. Deep convolutional neural networks (CNNs) have demonstrated their ability to effectively retrieve HR MR images from low-resolution (LR) MR images. However, most CNN-based super-resolution (SR) algorithms treat content and background information equally, ignoring the unique properties of MR images, such as low contrast, intricate tissue textures, and sparse backgrounds. We present a Wavelet Dual- Stream Network (WDN) for accurate MR SR that addresses the issues raised above. First, a wavelet transform is leveraged at the network's beginning to decouple the inputs, which are divided into high-frequency and low-frequency sub-bands. The high-frequency sub-bands relate to content information with larger frequency changes, and the low-frequency sub-bands correspond to background information. In addition, we devise a two-branch structure to reconstruct the high-frequency and low-frequency features separately. On the one hand, we design the U-Net Attention (U-A) mechanism for focusing the network's attention on regions with critical information. On the other hand, due to the correlation between high-frequency and low- frequency branches, We establish the Cross Attention Block (CAB) to accomplish the interaction between two branches. CAB takes advantage of the redundancy of information between different branches to distill the information of the current branch. Finally, the inverse wavelet transform is utilized to couple the modified high-frequency and low-frequency sub-bands as SR. Extensive experiments confirm the effectiveness of the WDN, which provides a clear improvement over the state-of-the-art method in both subjective and objective evaluations.
Dual Hypergraph Features for Path Inference in Wikipedia Links

Here, path extrapolation is assessed in a graph by extracting the most informative features from nodes and edges. Wikipedia articles are the graph nodes, and the links between articles are the graph edges. A graph neural network called GRETEL is used as baseline. New features are extracted by exploiting the information from the data. By employing the dual hypergraph transformation the structural role of nodes and edges is interchanged, enabling more complex relationships to be captured. Experimental evidence is disclosed demonstrating that the features extracted from the dual hypergraph are more de-scriptive than those extracted from the original graph, improving GRETEL performance.
A Reference-Less Quality Metric for Automatic Speech Recognition Via Contrastive-Learning of A Multi-Language Model with Self-Supervision

The common standard for quality evaluation of automatic speech recognition (ASR) systems is reference-based metrics such as the Word Error Rate (WER), computed using manual ground-truth transcriptions that are time-consuming and expensive to obtain. This work proposes a multi-language referenceless quality metric, which allows comparing the performance of different ASR models on a speech dataset without ground truth transcriptions. To estimate the quality of ASR hypotheses, a pre-trained language model (LM) is fine-tuned with contrastive learning in a self-supervised learning manner. In experiments conducted on several unseen test datasets consisting of outputs from top commercial ASR engines in various languages, the proposed referenceless metric obtains a much higher correlation with WER scores and their ranks than the perplexity metric from the state-of-art multi-lingual LM in all experiments, and also reduces WER by more than 7% when used for ensembling hypotheses. The fine-tuned model and experiments are made available for the reproducibility: https://github.com/aixplain/NoRefER
Combining Multi-Head Attention and Sparse Multi-Head Attention Networks for Session-Based Recommendation

The core of session-based recommendation is to predict the next interactive item based on a set of anonymous user temporal or specified behavior sequences (e.g., click, browse or purchase item sequence), which is a key task of many online services today. Recently, self-attention networks have achieved remarkable success in the task of session-based recommendation. However, in session-based recommendation, some items may be clicked by mistake, and most of the current attention mechanisms assign weights to these items, resulting in the disadvantage of distraction. Although sparse attention networks can address the aforementioned issues, solely relying on sparse attention may in turn reduce the weight of some real-intent clicked items. Therefore, this paper proposes a model that combines multi-headed attention network and sparse multi-headed attention network, referred to as CMAN, which solves the drawback of assigning weights to items clicked by mistake in the traditional attention mechanism. And also prevents the drawback of reducing the weights of items that are truly clicked by some users brought by using sparse attention mechanism alone to some extent. Experiments on two real datasets show that the model outperforms some state-of-the-art models.
Improving Dino-Based Self-Supervised Speaker Verification with Progressive Cluster-Aware Training

Self-supervised contrastive learning has recently emerged as one of the promising approaches in speaker verification task, due to its independence from labeled data. Among them, the DINO-based self-supervised framework, trained without exploiting negative pairs, is very popular and achieves excellent performance in the speaker verification task. However, limited by the duration of utterance, there exist many overlaps which may mislead the model to pay attention to irrelevant information. To tackle this problem, we propose a cluster-aware (CA) training strategy to make the model crop positive segments from several utterances in the same cluster rather than from a single utterance. Besides, in the clustering stage, we also investigate strategies of fixed number clustering as well as progressive clustering. With these strategies, our CA-DINO achieves the state-of-the-art result on Vox-O test set. Finally, we explore the effect of fine-tuning CA-DINO with a small amount of labeled data. Our proposed model with only 10% labeled data outperforms the fully supervised system trained on all data.
Specialized Semantic Enrichment of Speech Representations

SSL is now commonly used to capture multilingual speech representation, by exploiting huge audio speech data in several languages. In parallel, some text-based large neural models trained on huge multilingual textual documents have been introduced in order to capture the general semantics of a sentence, independently of the language, and to represent it under the form of a sentence embedding. Very recently, an approach has been introduced that takes benefit of such sentence embedding in order to continue the training of an SSL speech model in order to inject some multilingual semantic information. In a previous work, we made a layer-wise analysis in order to better understand how this semantic information is integrated into a wav2vec2.0 model. In this new study, we show how this semantic information can be specialized to a targeted downstream task dedicated to a task-oriented spoken language understanding by exploiting a small amount of transcribed data. We also show that the use of in-domain data from a close language can also be very beneficial in order to make the semantic representation captured by this enriched SSL model more accurate.
State-of-the-Art in Nudity Classification: A Comparative Analysis

This paper presents a comparative analysis of existing nudity classification techniques for classifying images based on the presence of nudity, with a focus on their application in content moderation. The evaluation focuses on CNN-based models, vision transformer, and popular open-source safety checkers from Stable Diffusion and Large-scale Artificial Intelligence Open Network (LAION). The study identifies the limitations of current evaluation datasets and highlights the need for more diverse and challenging datasets. The paper discusses the potential implications of these findings for developing more accurate and effective image classification systems on online platforms. Overall, the study emphasizes the importance of continually improving image classification models to ensure the safety and well-being of platform users. The project page, including the demonstrations and results is publicly available at https://github.com/fcakyon/contentmoderation-deep-learning.
Assisted Labeling Visualizer (ALVI): A Semi-Automatic Labeling System For Time-Series Data

Machine learning applications can significantly benefit from large amounts of labeled data, although the task of labeling data is notoriously challenging and time-consuming. This is particularly evident in domains involving human subjects, where labeling time-series signals often necessitates trained professionals. In this work, we introduce the Assisted Labeling Visualizer (ALVI), a system that simplifies the process of labeling data by offering an interactive user interface that visualizes synchronized video, feature-map representations, and raw time-series signals. ALVI also leverages deep learning and self-supervised learning techniques to facilitate the semi-automatic labeling of large amounts of unlabeled data. We demonstrate the capabilities of ALVI on a human activity recognition dataset to showcase its potential for enhancing the labeling process of time-series sensor data.
Blockchain-Oriented Software Testing: A Preliminary Literature Review

Blockchain-oriented software (BOS) is emerging rapidly, however, tailored software engineering techniques to assist software testing is one of the important and necessary phases of software development that plays a vital role in software quality assurance. Blockchain-oriented software testing (BOST) is another area that is still emerging. This paper conducts a preliminary literature review to assess the type of research published, where this research can be found, and the main themes of existing research in BOST. This study also introduces a framework for conducting an empirical study on BOS available on GitHub and proposes future work to expand the knowledge in this domain.
Prediction of Driver’s Stress Affection in Simulated Autonomous Driving Scenarios

We investigate the task of predicting stress affection from physiological data of users experiencing simulations of autonomous driving. We approach this task on two levels of granularity, depending on whether the prediction is performed at the end of the simulation, or along the simulation. In the former, denoted as coarse-grained prediction, we employed Decision Trees. In the latter, denoted as fine-grained prediction, we employed Echo State Networks, a Recurrent Neural Network that allows efficient learning from temporal data and hence is suitable for pervasive environments. We conduct experiments on a private dataset of physiological data from people participating in multiple driving scenarios simulating different stress-inducing events. The results show that the proposed model is capable of detecting event-related stress reactions, proving the existence of a correlation between stress-inducing events and the physiological data.
Human Activity Recognition From Motion and Acoustic Sensors Using Contrastive Learning

In this paper, we formulate human activity recognition as a downstream task of pretrained multimodal contrastive learning (MCL) models and break the convention of the one-modality-to-one-modality contrastive paradigm by allowing the models to have more than one source modality. Different from the prevailing assumption in MCL that one source modality and one target modality are the counterparts of each other, this work considers the possibility where it takes multiple source modalities with complementary information to match up to a target modality. In particular, we leverage a large-scale pretrained audio-language contrastive model and extend it to accepting IMU and audio input. The experiment results indicate the superiority of using complementary source modalities over using any source modality alone with 10.3% to 35.0% performance gain.
Failure Prediction in 2D Document Information Extraction with Calibrated Confidence Scores

Modern machine learning models can achieve impressive results in many tasks, but often fail to express reliably how confident they are with their predictions. In an industrial setting, the end goal is usually not a prediction of a model, but a decision based on that prediction. It is often not sufficient to generate high-accuracy predictions on average. One also needs to estimate the uncertainty and risks involved when making related decisions. Thus, having reliable and calibrated uncertainty estimates is highly useful for any model used in automated decision-making.In this paper, we present a case study, where we propose a novel method to improve the uncertainty estimates of an in-production machine learning model operating in an industrial setting with real-life data. This model is used by Basware, a Finnish software company, to extract information from invoices in the form of machine-readable PDFs. The solution we propose is shown to produce calibrated confidence estimates, which outperform legacy estimates on several relevant metrics, increasing coverage of automated invoices from 65.6% to 73.2% with no increase in error rate.
Detecting Cyber Attacks in Industrial Control Systems Using Spatio-Temporal Autoencoder

Industrial control systems (ICSs) are widely used in various industries. These systems have become prime targets for cyber and physical attacks. The attacks, which have an impact on the physical processes of an ICS, often lead to system misbehaviors through data contamination. Recent research has shown that network-based detection methods cannot monitor the physical level activities well enough to mitigate hybrid cyber attacks and cannot entirely protect ICSs. To protect ICSs from such threats, we propose a Spatio-Temporal Autoencoder (STA) with a Dynamic Thresholding Mechanism. The STA learns the normal physical behaviors of the system by capturing deep spatio-temporal dependencies to form a unified representation of the system state. The unified representation is decoded to reconstruct the input features. Then, the dynamic threshold is used to detect and locate the anomalies. We validate the STA using data set from a real water treatment plant testbed, SWaT. Evaluation results indicate the superior performance of the STA compared with six state-of-the-art methods, achieving an average improvement of 5.8% in the F-Score.
Real-world Performance Estimation of Liquid State Machines for Spoken Digit Classification

Liquid State Machine (LSM) is a brain-inspired neural network architecture for solving temporal classification problems like speech recognition. The simple structure of LSM with a reservoir and single-layer classifier is attractive from a hardware implementation perspective. When the LSM is considered for low-power hardware implementation in real-world command word recognition tasks, challenges like nonidealities in sensor filter response and ambient noise become critical concerns. In this work, we evaluate the performance of LSM based on two aspects (1) ambient noise and (2) sensor/preprocessing circuit nonidealities. For Ambient noise, we use additive white gaussian noise (AWGN) and ambient noise using the iNoise Indian Noise dataset that covers various natural indoor, outdoor, and travel-related environmental sounds. To understand the impact of input hardware nonidealities, we analyzed the impact of the audio preprocessing filter's quality factor, order, center frequency variations, and output nonlinearity on LSM performance. We use the spoken digits classification in the TI-46 dataset. This paper's findings present design guidelines for the system designers intending to use liquid-state machines for speech classification tasks. In terms of filter design, first, there is a broad Q, order space for filter design where performance is high. We use the hardware-friendly parallel 4th order Butterworth bandpass filter model to provide a baseline 98% accuracy in speech classification tasks. Second, the performance of LSM degrades proportionally to the variation in the center frequency of the bandpass filters in the filter bank. Third, nonlinearity with the third-order harmonic of 50 dBc can be tolerated. Regarding ambient noise, our study shows that a 40 dB SNR for AWGN is sufficient for ideal performance. Second, the best case of “home” noise leads to a performance of 91.4%. Outdoor and travel noise reduce the classification performance to 78.8% and 62.4%, respectively. However, ideal performance is recovered if the signal to noise ratio (SNR) is increased, particularly by 10 dB in indoor conditions and 30 dB in outdoor conditions. Thus, our study presents an engineering evaluation for real-world spoken digit recognition using LSMs.
A fast reservoir computing based image encryption algorithm

Reservoir computing has gained a lot of momentum in recent years, in part due to its high-speed performance. The algorithm presented here deliberately exploits this property for efficient encryption and decryption of image data. It is demonstrated that its performance is a factor of 1000 faster than another encryption algorithm based on reservoir computing. We further extend the presented method for applicability to high-resolution and color images by exploiting the computational speed of small reservoir networks and the property that reservoir computers are generally able to reproduce their training data with high accuracy. To that, security and performance aspects are discussed.
Integrating Multiple Visual Attention Mechanisms in Deep Neural Networks

Inspired by the success of various visual attention techniques in computer vision, we introduce a novel method for integrating multiple attention mechanisms to boost model performance. Our approach involves augmenting a base model with a Parallel Visual Attention Encoder (PVAE) branch, which concurrently employs two different attention modules (modified large kernel attention and modified convolutional block attention) to capture essential visual features. To reduce the training cost incurred by these additional components, we apply an encoder for efficient feature extraction and dimensionality reduction before applying the attention modules. The proposed PVAE architecture can be combined with cutting-edge models (e.g., EfficientNet, ResNet, DenseNet, etc.) to create a Parallel Visual Attention Network (PVAN). We evaluate the efficacy of our approach by devising a PVAN with EfficientNet as the base model for the task of classifying dog breeds. Our experimental results demonstrate the effectiveness of the proposed hybrid visual attention architecture, which achieves superior performance compared to the base model and models with a single attention mechanism. We further present an interactive web application developed for the general public to identify dog breeds using their photographs to test our model’s performance in real-life scenarios.
Prototypical Contrastive Learning for Domain Adaptive Semantic Segmentation

The goal of domain adaptive semantic segmentation is to train a model using labeled source domain data and produce accurate dense predictions on the unlabeled target domain. Previous methods adopt self-training, where reliable target domain predictions are used as pseudo labels for training. However, intra-class variations across domains, such as the varying visual appearance in each category, have not been fully explored, leading to misalignment in feature distribution between the source and target domains. In this paper, we propose to optimize the feature space with representative prototypes shared across domains. Specifically, we first adopt the non-parametric clustering to model multiple prototypes for each category feature space. Then, category-discriminative feature space is obtained via pixel-to-prototype contrastive learning. Through extensive experiments, our proposed method demonstrates competitive performance on GTA5→Cityscapes and Synthia→Cityscapes benchmark. It is noteworthy that our method is compatible with the existing UDA methods.
MemAU-Net: Memory-Enhanced Attention U-Net for Medical Image Forgery Localization

Medical image forgery has become an urgent issue in academia and medicine. Unlike natural images, images in the medical field are so sensitive that even minor manipulation can produce severe consequences. In view of the specificities of medical images, natural image forgery localization methods are difficult to generalize. While the unsatisfactory performance of existing medical image forgery localization methods, we propose MemAU-Net: a copy-move and splicing resistant medical image forgery localization network. We propose a novel attention gate named Memory-Enhanced Attention Gate (MAG), which effectively fuses shallow-deep features and improves feature representation to make attention better suit for medical image forgery localization tasks. To minimize holes and edge serrations caused by the delicate and blurred textures in medical images, we use dense CRF to smooth the boundaries, reduce false alarms and missed detection. Since there is no medical image forgery dataset publicly available, by using the copy-move and splicing forgery operations, we manually tamper and annotate two forged medical image datasets: OIAT (eye) and COVIDLT (lung) to verify the generality of the proposed model. The dataset includes the images from funds and different views of the lungs, corresponding to the specificity of medical images with flat grayscale changes and complex textures. The results show that by using MemAU-Net alone, we can improve the F-measure by 2.33% and 4.61% over the state-of-the-art baseline on these two datasets, respectively. Moreover, the raised precisions and enhanced visualizations with the addition of dense CRF indicate that it effectively removes false positives, fills holes and smooths edges. A good proof of the superiority of MemAU-Net's medical image forgery localization function is provided by these results. In addition, the proposed method achieves promising robustness to forgery post-processing such as rotation, scaling and anti-forensic attacks like noise.
Hybrid Clustering Solutions Fusion based on Gated Three-way Decision

Consensus clustering methods provide better performance by fusing multiple clustering solutions in terms of accuracy, robustness and stability. However, most current methods suffer from different challenges: i) the high-dimensional problem; ii) limitations of single clustering method; iii) the optimal number of clusters selecting for a certain validity measure; iv) redundant clustering candidate attributes. To overcome the above limitations, we propose a hybrid clustering solutions fusion method based on gated three-way decision (HCFG) for data analysis. By integrating multiple clustering solutions and executing information fusion, HCFG enjoys four properties: (1) multiple random subspace generation strategy is utilized to generate diverse low-dimensional subspaces effectively; (2) a fusion framework that considers characteristics of both the soft clustering and hard clustering methods is designed, in which potential boundary of feature attribute sets is explored; (3) the optimal number of clusters is set by utilizing multiple clustering validity indices; (4) clustering solutions is considered as attributes and a gated three-way decision method is proposed to adaptively conduct attribute reductions. Extensive comparative experiments on 24 real-world data sets demonstrates the effectiveness and superiority of HCFG. Moreover, nonparametric tests are conducted to compare HCFG with multiple consensus clustering methods.