Reviewerly/exHarmony huggingface dataset:
BTS-E: Audio Deepfake Detection Using Breathing-Talking-Silence Encoder

Voice phishing (vishing) is increasingly popular due to the development of speech synthesis technology. In particular, the use of deep learning to generate an arbitrary-content audio clip simulating the victim’s voice makes it difficult not only for humans but also for automatic speaker verification (ASV) systems to distinguish. Countermeasure (CM) systems have been developed recently to help ASV combat synthetic speech. In this work, we propose BTS-E, a framework to evaluate the correlation between Breathing, Talking (speech), and Silence sounds in an audio clip, then use this information for deepfake detection tasks. We argue that natural human sounds, such as breathing, are hard to synthesize by Text-to-speech (TTS) system. We conducted a large-scale evaluation using ASVspoof 2019 and 2021 evaluation set to validate our hypothesis. The experiment results show the applicability of the breathing sound feature in detecting deepfake voices. In general, the proposed system significantly increases the performance of the classifier by up to 46%.
An Application of Quantum Mechanics to Attention Methods in Computer Vision

This work proposes the quantum-state-based mapping (QSM) for machine learning. QSM uses wave functions that describe microscopic particle systems as mappings. By QSM, original inputs or features extracted by neural networks are processed as quantum states to train wave function parameters. QSM has a low computational cost, almost no additional parameters, and is easy to integrate with other modules. We demonstrate the simplest form of the wave function as a mapping, that is, when a one-dimensional particle is in an infinitely deep potential well, in combination with advanced attention modules. Experiments show that QSM significantly improves the feature recalibration ability of attention module in transfer learning tasks. Then, we tried to analyze the effectiveness of QSM. This work indicates that QSM has an important application value in interdisciplinary machine learning.
Latent Domain Generation for Unsupervised Domain Adaptation Object Counting

Unsupervised cross-domain crowd counting has recently received great attention in computer vision, which generalizes the model from the source domain to the unlabeled target domain. However, it is an extremely challenging task because only unlabeled data is available from the target domain and the domain gap between two domains is implicit in crowd counting. In this paper, we propose a latent domain generation method to improve the generalization ability of unsupervised domain adaptation crowd counting by generating a latent domain. To this end, we propose a domain generator with random perturbations to learn a new latent distribution derived from the original source distribution. The latent domain generator can extract target information sampled in its stochastic latent representation, which preserves the original target information and enhances the variational ability. Meanwhile, to ensure that the generated latent domain is consistent with the source domain in counting performance, we introduce a consistency loss to encourage similar output from latent and source domains. Moreover, to enhance the adaptation ability of the generated latent domain, we apply the adversarial loss to achieve alignment between the latent and target domains. The domain generator with the adversarial loss and consistency loss ensures that the generated domain is aligned to the target while also improving the robustness of the original source domain model. The experiment indicates that our framework can effortlessly extend to scenarios with different objects (crowd, cars). The experiments also demonstrate the effectiveness of our method on unsupervised realistic-to-realistic crowd counting problems.
Rethinking Pseudo-Labeling for Semi-Supervised Facial Expression Recognition With Contrastive Self-Supervised Learning

Self-supervised learning for semi-supervised facial expression recognition aims to avoid the need to collect expensive labeled facial expression data. Existing methods demonstrate an impressive performance boost, but they artificially assume that small labeled facial expression data and large unlabeled facial expression data are from the same class distribution. In a more realistic scenario, when utilizing facial expression data from a large face recognition database as unlabeled data, there will be a mismatch distribution between the two sets of data. This often results in severe performance degradation due to incorrect propagation of unlabeled data from unrelated sources. In this paper, we address the class distribution mismatch problem in deep semi-supervised learning-based facial expression recognition. Specifically, we propose a silhouette coefficient-based contrast clustering algorithm, which determines the degree of separation between clusters by examining the intra-cluster and inter-cluster distances to accurately detect out-of-distribution data. In addition, we propose a pseudo-labeling rethinking strategy that matches the soft pseudo-labels estimated from a fine-tuned network to the contrast clustering to produce reliable pseudo-labels. Experiments on three in-the-wild datasets, RAF-DB, FERPlus and AffectNet, demonstrate the effectiveness of our method, and our approach performs well compared to state-of-the-art methods.
Arbitrary Style Transfer With Fused Convolutional Block Attention Modules

The advancement of deep learning has rendered image style transfer a progressively intricate subject matter. The proposed solution aims to tackle the limitations of current methods in retaining the content image object contours and avoiding blurred image boundaries and mismatched color matching after stylization. To achieve this, an arbitrary-style transfer network is introduced, which leverages the attention mechanism. The network comprises an encoder-decoder module, a convolutional block attention module (CBAM), and an adaptive attention normalization network (AdaAttN) module. The CBAM attention mechanism is presented as an extension of the AdaAttN network, with the aim of producing stylized images that exhibit both global and local style coordination. This is achieved by leveraging long-range dependencies in the image. Additionally, a novel loss function, referred to as the structural similarity loss, is proposed to enhance the consistency of the generated images with respect to the underlying content structure. Finally, a new local feature loss is introduced to further enhance the visual quality of the stylized images at a local level. The study involved conducting style transfer training on a dataset comprising 82,783 real images and 81,446 artistic images. Furthermore, an additional set of 1,000 resultant images, generated from 100 real photos and 10 artistic portraits, was utilized for testing purposes. The study compares the experimental outcomes with four contemporary-style transfer techniques. Additionally, the efficacy of the CBAM module and SSIM loss function is demonstrated through ablation experiments. The findings of the experiment demonstrate that the network proposed has the ability to effectively adapt to the local style and can adeptly correspond the semantically proximate style features to the content features, thereby preserving superior spatial consistency.
Marigold: Efficient<i>k</i>-Means Clustering in High Dimensions

How can we efficiently and scalably cluster high-dimensional data? The k -means algorithm clusters data by iteratively reducing intra-cluster Euclidean distances until convergence. While it finds applications from recommendation engines to image segmentation, its application to high-dimensional data is hindered by the need to repeatedly compute Euclidean distances among points and centroids. In this paper, we propose Marigold ( k -means for high-dimensional data), a scalable algorithm for k -means clustering in high dimensions. Marigold prunes distance calculations by means of (i) a tight distance-bounding scheme; (ii) a stepwise calculation over a multiresolution transform; and (iii) exploiting the triangle inequality. To our knowledge, such an arsenal of pruning techniques has not been hitherto applied to k -means. Our work is motivated by time-critical Angle-Resolved Photoemission Spectroscopy (ARPES) experiments, where it is vital to detect clusters among high-dimensional spectra in real time. In a thorough experimental study with real-world data sets we demonstrate that Marigold efficiently clusters high-dimensional data, achieving approximately one order of magnitude improvement over prior art.
Addressing Information Inequality for Text-Based Person Search via Pedestrian-Centric Visual Denoising and Bias-Aware Alignments

Text-based person search is an important task in video surveillance, which aims to retrieve the corresponding pedestrian images with a given description. In this fine-grained retrieval task, accurate cross-modal information matching is an essential yet challenging problem. However, existing methods usually ignore the information inequality between modalities, which could introduce great difficulties to cross-modal matching. Specifically, in this task, the images inevitably contain some pedestrian-irrelevant noise like background and occlusion, and the descriptions could be biased to partial pedestrian content in images. With that in mind, in this paper, we propose a Text-Guided Denoising and Alignment (TGDA) model to alleviate the information inequality and realize effective cross-modal matching. In TGDA, we first design a prototype-based denoising module, which integrates pedestrian knowledge from textual features into a prototype vector and uses it as guidance to filter out pedestrian-irrelevant noise from visual features. Thereafter, a bias-aware alignment module is introduced, which guides our model to focus on the description-biased pedestrian content in cross-modal features consistently. Through extensive experiments, the effectiveness of both modules has been validated. Besides, our TGDA achieves state-of-the-art performance on various related benchmarks.
Reinforcement Learning on Graphs: A Survey

Graph mining tasks arise from many different application domains, including social networks, biological networks, transportation, and E-commerce, which have been receiving great attention from the theoretical and algorithmic design communities in recent years, and there has been some pioneering work employing the research-rich Reinforcement Learning (RL) techniques to address graph mining tasks. However, these fusion works are dispersed in different research domains, which makes them difficult to compare. In this survey, we provide a comprehensive overview of these fusion works and generalize these works to Graph Reinforcement Learning (GRL) as a unified formulation. We further discuss the applications of GRL methods across various domains, and simultaneously propose the key challenges and advantages of integrating graph mining and RL methods. Furthermore, we propose important directions and challenges to be solved in the future. To our knowledge, this is the latest work on a comprehensive survey of GRL, this work provides a global view and a learning resource for scholars. Based on our review, we create a collection of papers for both interested scholars who want to enter this rapidly developing domain and experts who would like to compare GRL methods.
RAPP: Reversible Privacy Preservation for Various Face Attributes

The tremendous progress in deep learning has enabled to extract soft-biometric attributes from faces, which raises privacy concerns over images collected for face recognition. Advances toward attribute privacy have been able to conceal multiple attributes while preserving identity information but suffer from limitations: they 1) only consider a few soft-biometric attributes and 2) fail to support reversibility for attribute privacy preservation. To break these limitations, we design a reversible privacy-preserving scheme for various face attributes, called reversible attribute privacy preservation (RAPP). RAPP benefits from two modules: 1) The attribute obfuscator introduces a stream cipher to determine that special attributes have to be concealed with the user-defined password, which also supports recovering original attributes. 2) The attribute adversarial network is proposed to generate perturbed images that conceal various attributes while retaining the utility of face verification. In addition, when a wrong password is provided, the returned image with wrong attribute classification results still keeps realistic, which confuses an attacker to know whether the recovery is correct. Extensive experiments demonstrate that RAPP enables to conceal various attributes and recover original images while facilitating face verification.
Energy-Efficient Federated Learning Over Cell-Free IoT Networks: Modeling and Optimization

To leverage massive distributed data and computation resources in the Internet-of-Things (IoT) networks, federated learning (FL) is considered to be a promising technique with benefits of improved data privacy and communication efficiency. Meanwhile, cell-free massive multiple-input–multiple-output (cell-free massive MIMO) is a promising technology to enable the IoT networks to support FL. By deploying the access points (APs) closer to the IoT devices, path loss attenuation can be reduced. However, the performance of FL is still constrained by the limited power resources of IoT devices. To address this issue, we design an energy-efficient FL scheme over cell-free IoT networks by formulating an optimization problem to minimize the total energy consumption of the IoT devices participating in the FL process. To solve the intractable problem in hand, by exploiting its unique structure, we decompose it into three subproblems that facilitate the development of the proposed scheme. First, we derive the optimal central processing unit (CPU) operating frequency for IoT devices. Then, we design an optimal power allocation scheme to mitigate the straggler effect. Next, a nonlinear programming method is adopted to obtain a suboptimal solution for the reformulated problem. Finally, a three-stage algorithm is proposed for energy consumption minimization by considering these subproblems. Simulation results demonstrate the close-to-optimal performance of the proposed algorithm for energy savings compared with three baseline algorithms and the capability to support large numbers of IoT device access by mitigating the straggler effect.
Common vulnerability scoring system prediction based on open source intelligence information sources

The number of newly published vulnerabilities is constantly increasing. Until now, the information available when a new vulnerability is published is manually assessed by experts using a Common Vulnerability Scoring System (CVSS) vector and score. This assessment is time consuming and requires expertise. Various works already try to predict CVSS vectors or scores using machine learning based on the textual descriptions of the vulnerability to enable faster assessment. However, for this purpose, previous works only use the texts available in databases such as National Vulnerability Database. With this work, the publicly available web pages referenced in the National Vulnerability Database are analyzed and made available as sources of texts through web scraping. A Deep Learning based method for predicting the CVSS vector is implemented and evaluated. The present work provides a classification of the National Vulnerability Database’s reference texts based on the suitability and crawlability of their texts. While we identified the overall influence of the additional texts is negligible, we outperformed the state-of-the-art with our Deep Learning prediction models.
Multiple Image Watermarking in YCbCr Color Space Using Schur-SVD-DCT in Wavelet Domain and its authentication using SURF

In this proposed work, YCbCr color space is used for watermarking. Three different watermark logos are used to watermark Y, Cb, and Cr color spaces. The proposed watermarking method uses fundus image, ultrasound DICOM (Digital Imaging and Communications in Medicine) image of liver and Lena image as the host images. This work uses the properties of Lifting wavelet transform (LWT), Discrete cosine transform (DCT), Schur decomposition and Singular value decomposition for watermarking. The LWT is used to split the image into different sub-bands (low-low, high-low, low-high, high-high). For watermarking HL, LH, and HH sub-bands are used. The SVD is used to extract the dominant matrix. The proposed scheme is highly robust and imperceptible when compared with other techniques. Speeded up robust features (SURF) are used for feature matching authentication. The presented scheme is resistant to numerous attacks, including Median filter, Salt and Pepper noise, Gaussian noise, Rotation, Average filter, Sharpening, Histogram equalization and Speckle noise. The Structural similarity index measurement (SSIM), Peak signal to noise ratio (PSNR), Normalized correlation coefficient (NCC), and mean square error (MSE) are the metrics that are used to evaluate the effectiveness of the proposed scheme.
Whitening Transformation of i-vectors in Closed-Set Speaker Verification of Children

Automatic speaker recognition may be beneficial for children in a wide variety of disciplines, such as child education, security and safety. However, owing to a variety of issues in children’s speech, including immature vocal tracts, native phonology, long pauses, and hesitations, many speech technology applications have trouble recognizing non-native children. In order to assess how the children’s speech impacts the system, the primary objective of this study is to build a closed-set child speaker verffication system for non-native English speakers in both textdependent and text-independent tasks. This study focuses on a Gaussian probabilistic linear discriminant analysis (GPLDA) model that was trained employing i-vectors and several whitening methods. Cosine similarity scoring (CSS) and GPLDA scoring are used to evaluate the claimed outcomes. Additionally, it has been shown that in closedset speaker verification, the zero-phase component analysis (ZCA) transformation outperformed the other whitening transformations.
A blind watermarking approach based on hybrid Imperialistic Competitive Algorithm and SURF points for color Images’ authentication

Currently, we are in the area of the digital big data world. Every day, people share billions of images over Internet from various sources. Ensuring images’ authorship rights and illegal modification and/or reproductions become a challenge. Their protection must meet current and secure development needs of multimedia applications. In this paper, we propose an efficient watermarking scheme based on combining Imperialistic Competitive Algorithm (ICA), Speed Up Robust Features (SURF) points and Discrete Cosine Transform (DCT). The high complexity and inappropriate embedding area could lead to weak performance of watermarking schemas. To address these problems, an ideal watermarking embedding area was proposed using ICA to provide an optimized cost and complexity. The watermark was embedded into the DCT coefficients of the selected embedding areas. The watermark was extracted in a blind way to ensure color image’ authentication. Experimental findings demonstrate that the proposed approach achieves promising performance in terms of robustness and imperceptibility in almost cases. The average PSNR exceeds 61db regarding the imperceptibility. For the robustness measured by the correlation coefficients between the original watermark and the extracted is close to 1 in almost cases.
Federated learning approaches for fuzzy cognitive maps to support clinical decision-making in dengue

Federated learning is a distributed machine learning approach developed to guarantee the privacy and security of data stored on local devices. In healthcare, specifically in diseases of public health interest such as dengue, it is necessary to develop strategies that guarantee such data properties. Therefore, the aim of this work was to develop three federated learning approaches for fuzzy cognitive maps for the prediction of mortality and the prescription of treatment of severe dengue. The validation of the approaches was performed on severe dengue datasets from two dengue endemic regions in Colombia. According to the results, the use of federated learning significantly improves the performance of models developed in centralized environments. Additionally, the use of federated learning allows guaranteeing the privacy and security of each client's data due to the local training of the models. Federated learning is a useful tool in healthcare because it guarantees the privacy and security of patient data. Our results demonstrated the ability of aggregated models to predict mortality and prescribe treatment for severe dengue.
Adaptive archive-based multifactorial evolutionary algorithm for constrained multitasking optimization

Evolutionary multitasking has attracted much attention in the field of evolutionary computing. Most of the existing multitasking evolutionary algorithms aim at solving unconstrained multitasking optimization problems. The study on constrained multitasking optimization problems is scarce. However, in practical applications, lots of optimization problems contain constraints. In this paper, an adaptive archive-based multifactorial evolutionary algorithm is proposed to solve constrained multitasking optimization problems. First, an archiving strategy is proposed to store infeasible solutions with better objective function values. With this strategy, useful information on infeasible solutions can be exploited to accelerate the convergence rate. Second, the random mating probability is adjusted through an adaptive strategy to facilitate positive knowledge transfer. Finally, a new mutation strategy is proposed to promote convergence by mutating some random individuals and replacing the individuals with the largest constraint violation. By comparing existing constrained multitasking evolutionary algorithms and some constrained single-task evolutionary algorithms, the results reveal the effectiveness of the proposed algorithm in solving constrained multitasking optimization problems.