Reviewerly/exHarmony huggingface dataset:
Stealthy Backdoor Attack with Adversarial Training

Research shows that deep neural networks are vulnerable to back-door attacks. The backdoor network behaves normally on clean examples, but once backdoor patterns are attached to examples, back-door examples will be classified into the target class. In the previous backdoor attack schemes, backdoor patterns are not stealthy and may be detected. Thus, to achieve the stealthiness of backdoor patterns, we explore an invisible and example-dependent backdoor attack scheme. Specifically, we employ the backdoor generation network to generate the invisible backdoor pattern for each example, and backdoor patterns are not generic to each other. However, without other measures, the backdoor attack scheme cannot bypass the neural cleanse detection. Thus, we propose adversarial training to bypass neural cleanse detection. Experiments show that the proposed backdoor attack achieves a considerable attack success rate, invisibility, and can bypass the existing defense strategies.
Interactive Multi-Level Prosody Control for Expressive Speech Synthesis

Recent neural-based text-to-speech (TTS) models are able to produce highly natural speech. To synthesize expressive speech, the prosody of the speech has to be modeled, and predicted/controlled during synthesis. However, intuitive control over prosody remains elusive. Some techniques only allow control over the global style of the speech and do not allow fine-grained adjustments. Other techniques create fine-grained prosody embeddings, but these are difficult to manipulate to obtain a desired speaking style. We thus present ConEx, a novel model for expressive speech synthesis, which can produce speech in a certain speaking style, while also allowing local adjustments to the prosody of the generated speech. The model builds upon the non-autoregressive architecture of FastSpeech and includes a reference encoder to learn global prosody embeddings, and a vector quantized variational autoencoder to create fine-grained prosody embeddings. To realize prosody manipulation, a new interactive method is proposed. Experiments on two datasets show that the model enables multi-level prosody control.
A Brief Review on 4D Weather Visualization

The accelerated changes on our planet have led to a growing interest in climate change and its consequences: natural hazards and adverse socio-economic impacts. However, the development of climate research and the proliferation of datasets require an integrated and efficient approach to the analysis, investigation, and visualization of atmospheric meteorological data. Thus, we propose a literature review of existing systems viewing meteorological phenomena in four and three dimensions. Moreover, we evaluate meteorological occurrences to better understand the dynamics associated with a meteorological phenomenon and visualize different weather data. Based on the investigation of tools and methods, we consider the existence of different ways of representing meteorological data and methodologies. However, it was imperative to obtain knowledge and create our way of visualizing weather data. This article found eleven existing solutions for 4D meteorological visualization and meteorological phenomena.
Efficient and Stable Information Directed Exploration for Continuous Reinforcement Learning

In this paper, we investigate the exploration-exploitation dilemma of reinforcement learning algorithms. We adapt the information directed sampling, an exploration framework that measures the information gain of a policy, to the continuous reinforcement learning. To stabilize the off-policy learning process and further improve the sample efficiency, we propose to use a randomized learning target and to dynamically adjust the update-to-data ratio for different parts of the neural network model. Experiments show that our approach significantly improves over existing methods and successfully completes tasks with highly sparse reward signals.
Towards end-to-end Speaker Diarization with Generalized Neural Speaker Clustering

Speaker diarization consists of many components, e.g., front-end processing, speech activity detection (SAD), overlapped speech detection (OSD) and speaker segmentation/clustering. Conventionally, most of the involved components are separately developed and optimized. The resulting speaker diarization systems are complicated and sometimes lack of satisfying generalization capabilities. In this study, we present a novel speaker diarization system, with a generalized neural speaker clustering module as the backbone. The whole system can be simplified to contain only two major parts, a speaker embedding extractor followed by a clustering module. Both parts are implemented with neural networks. In the training phase, an on-the-fly spoken dialogue generator is designed to provide the system with audio streams and the corresponding annotations in categories of non-speech, overlapped speech and active speakers. The chunk-wise inference and a speaker verification based tracing module are conducted to handle the arbitrary number of speakers. We demonstrate that the proposed speaker diarization system is able to integrate SAD, OSD and speaker segmentation/clustering, and yield competitive results in the VoxConverse20 benchmarks.
Comparative Analysis to Improve the Image Accuracy In Face Recognition System Using Hybrid LDA Compared With PCA

Aim-This study depicts the improvisation and comparison of accuracy of two different face recognition algorithms to improvise the novel face detection rate from the stored database under various lighting conditions. Materials and methods-From Kaggle, a total of 13000 face samples are being collected from LFW people face recognition dataset. Linear Discriminant Analysis (LDA) and Principal Component Analysis (PCA) algorithms are improvised and compared to recognize the faces and to increase the accuracy rate. Results-Based on the MATLAB simulation and verification, LDA shows the recognition rate of 85.2% and PCA shows 73%. From the statistical analysis, the significant accuracy ratio is <0.05. Conclusion-It is concluded that the LDA algorithm shows enhanced features in edge detection, image equalization and image normalization than the PCA algorithm in the face recognition system for the dataset considered.
Graph Convolutional Network Based Semi-Supervised Learning on Multi-Speaker Meeting Data

Unsupervised clustering on speakers is becoming increasingly important for its potential uses in semi-supervised learning. In reality, we are often presented with enormous amounts of unlabeled data from multi-party meetings and discussions. An effective unsupervised clustering approach would allow us to significantly increase the amount of training data without additional costs for annotations. Recently, methods based on graph convolutional networks (GCN) have received growing attention for unsupervised clustering, as these methods exploit the connectivity patterns between nodes to improve learning performance. In this work, we present a GCN-based approach for semi-supervised learning. Given a pre-trained embedding extractor, a graph convolutional network is trained on the labeled data and clusters unlabeled data with "pseudo-labels". We present a self-correcting training mechanism that iteratively runs the cluster-train-correct process on pseudo-labels. We show that this proposed approach effectively uses unlabeled data and improves speaker recognition accuracy.
Theticity and sentence-focus in Italian: grammatically encoded categories or categories of language use?

Abstract This article aims to examine whether Italian has theticity and sentence-focus among its grammatically encoded categories. After the categories of theticity and sentence-focus are introduced, the concepts of grammatically encoded category and category of language use are operationalized along the lines of Integral Linguistics. For a functional category to qualify as a grammatically encoded category of a language, the language should have at least one construction that structurally encodes the category as its indefeasible semantics. The article provides a qualitative investigation of both corpus-based and constructed examples of the eight Italian sentential structures that have hitherto been recognized in the literature as thetic or sentence-focus constructions. It is shown that each of the constructions has information structural uses that are incompatible with a structurally encoded theticity or sentence-focus semantics, i.e., topic-comment structure and narrow focus construal. It is argued that theticity and sentence-focus are therefore categories of language use rather than grammatically encoded categories of Italian.
Multi-Sample Subband Wavernn Via Multivariate Gaussian

This paper proposes a high-speed neural vocoder for CPU implementation. Two approaches for speeding up autoregressive neural vocoders have been proposed, 1) simultaneous multiple sample generation and 2) subband signal-based vocoder; so far they have been employed independently. Our neural vocoder is extremely fast as it generates multiple samples of subband signals simultaneously. Although there is an association between each subband signal, the conventional subband-based vocoder can degrade quality because each subband signal is generated from an independent probability distribution. To overcome this problem, we also introduce waveform generation that takes account of the association of each subband by employing multivariate Gaussian. Experiments show that 1) our proposed method is 1.81 times as fast as the conventional subband WaveRNN on a single-threaded CPU; 2) it outperformed the conventional method in a subjective evaluation in terms of naturalness, and achieved a mean opinion score (MOS) of 4.08 on text-to-speech task.
Spatio-Temporal Motion Aggregation Network for Video Action Detection

Recognizing action patterns and detecting action instances are vital for spatial temporal action detection task, which aims to recognize the actions of interest in untrimmed videos and localize them in both space and time. The mainstream action tubelet detectors, how-ever, ignore the conflicts in features between localization and classification, and use localization features for temporal modeling, which leads to ineffective action classification. In this paper, we propose the Spatio-Temporal Motion Aggregation mechanism for integrating the local motion feature from a short term snippet and the longer spatio-temporal information to predict the action category. We design the Class-Agnostic Center Localization module to perform action instance center localization in the Class-Agnostic manner. Besides, Movement and Size Regression is proposed for movement estimation and spatial extent detection by using Gaussian kernels to encode training samples. These three modules work together to generate the tubelet detection results, which could be further linked to yield video-level tubes with a matching strategy. Our detector achieves the state-of-the-art performance in both frame-mAP and video-mAP metrics, on the UCF-24 and JHMDB datasets.
Multi-Domain Unpaired Ultrasound Image Artifact Removal Using a Single Convolutional Neural Network

Ultrasound imaging (US) often suffers from distinct image artifacts from various sources. Classic approaches for solving these problems are usually model-based iterative approaches that have been developed specifically for each type of artifact, which are often computationally intensive. Recently, deep learning approaches have been proposed as computationally efficient and high performance alternatives. Unfortunately, in the current deep learning approaches, a dedicated neural network should be trained with matched training data for each artifact type. This poses a fundamental limitation in the practical use of deep learning for US, since large number of paired data is required for supervised training of multiple models to deal with various US image artifacts. Inspired by the recent success of multi-domain image transfer, herein, we propose a novel unpaired deep learning approach where a single neural network can deal with different types of US artifacts simply by changing a mask vector that switches between different target domains. The proposed method can generate high quality images by removing distinct artifacts, which are comparable to those obtained by separately trained multiple neural networks.
Image Energy Saving Recognition Technology of Monitoring System Based on Ant Colony Algorithm

In order to solve the shortcomings of traditional monitoring and alarm system, the key technology of intelligent video monitoring system-image processing and recognition technology is studied. The image energy-saving recognition technology of monitoring system based on ant colony algorithm is proposed. The ant colony algorithm is used to segment the image, extract the suspicious area, and then deal with the suspicious area separately by using the reasoning based on default rules in artificial intelligence. Using the trained neural network, the recognition experiment is carried out on 40 samples, and the recognition accuracy is more than 93%, which shows that it is very effective to use 7 invariant moments in the target image area as the characteristic parameters of target recognition. The microcomputer 2 renewal for a single target area is identified by using a moment invariant calculation force method and neural network. The execution time of the test on the microcomputer of Saijie 1.0 is microsecond. The experiment proves that it is practical and reliable, meets the real-time requirements, and can transmit the alarm signal and suspicious area image to the alarm center through the network.
Multi-View Data Representation Via Deep Autoencoder-Like Nonnegative Matrix Factorization

Since a large proportion of real-world data is made of different representations or views, learning on data represented with multiple views (e.g., numerous types of features or modalities) has garnered considerable attention recently. Nonnegative matrix factorization (NMF) has been widely adopted for multi-view learning due to its great interpretability. We focus on unsupervised multi-view data representation in this paper and propose a novel framework termed Deep Autoencoder-like NMF (DANMF-MDR), which learns an intact representation by simultaneously exploring multi-view complementary and consistent information. Furthermore, an efficient iterative optimization algorithm is developed to solve the proposed model. Experimental results on three real-world multi-view datasets demonstrate that ours performs better than the SOTA multi-view NMF-based MDR approaches.
Masked Acoustic Unit for Mispronunciation Detection and Correction

Computer-Assisted Pronunciation Training (CAPT) plays an important role in language learning. Conventional ASR-based CAPT methods require expensive annotation of the ground truth pronunciation for the supervised training. Mean-while, certain undefined non-native phonemes cannot be correctly classified into standard phonemes, making the annotation process challenging and subjective. On the other hand, ASR-based CAPT methods only give the learner text-based feedback about the mispronunciation, but cannot teach the learner how to pronounce the sentence correctly. To solve these limitations, we propose to use the acoustic unit (AU) as the intermediary feature for both mispronunciation detection and correction. The proposed method uses the masked AU sequence and the target phonemes to detect the error AU and then corrects it. This method can give the learner speech-based self-imitating feedback, making our CAPT powerful for education.
Comparative Analysis on Datasets for Sign Language Detection System

The proposed work uses Convolution Neural Network (CNN) algorithm to train the model to recognize the sign gestures made by the person, which is done by pre-processing the image and matching it with the trained dataset. As the output is text and image, one can easily interpret the meaning of a particular sign. The model's efficiency is tested using two types of input: images and CSV files. The inputs are tested for their efficiency and time to load for building the model. The image dataset used 16500 images for training and 3311 images for testing and took more than an hour to train a model. On the other hand, the CSV dataset consisted of 27455 training files and 7172 testing files and took less than half an hour to train a model. The trained models showed 93.8% and 87% accuracy for the CSV and image testing models, respectively. The paper analyses different types of inputs and suggests the most efficient input to train the model for a sign language detection system.
Panchromatic Imagery Copy-Paste Localization Through Data-Driven Sensor Attribution

Overhead images can be obtained using different acquisition and processing techniques, and they are becoming more and more popular. As with common photographs, they can be forged and manipulated by malicious users. However, not all image forensics methods tailored to normal photos can be successfully applied out of the box to overhead images. In this paper we consider the problem of localizing copy-paste forgeries on panchromatic images acquired with different satellites. We leverage a set of Convolutional Neural Networks (CNNs) that extract traces of the acquisition satellite directly from image patches. We then determine whether an image region appears to have been acquired with a different satellite than the rest of the picture. Results show that the proposed technique outperforms more sophisticated image forensics tools tailoring common photographs.
Deep Video Inpainting Localization Using Spatial and Temporal Traces

Advanced deep-learning-based video inpainting can fill a specified video region with visually plausible contents, usually leaving imperceptible traces. As inpainting can be used for malicious video manipulations, it has led to potential privacy and security issues. Therefore, it is necessary to detect and locate the video regions subjected to deep inpainting. This paper addresses this problem by exploiting the spatial and temporal traces left by inpainting. Firstly, the inpainting traces are enhanced by intra-frame and inter-frame residuals. In particular, we guide the extraction of inter-frame residual with optical-flow based frame alignment, which can better reveal the inpainting traces. Then, a dual-stream network, acting as the encoder, is designed to learn discriminative features from frame residuals. Finally, bidirectional convolutional LSTMs are embedded in the decoder network to produce pixel-wise predictions of inpainted regions for each frame. The proposed method is evaluated with tampered videos created by two state-of-the-art deep video inpainting algorithms. Extensive experimental results show that the proposed method can effectively localize the inpainted regions, outperforming existing methods.
Vision Transformer Equipped With Neural Resizer On Facial Expression Recognition Task

When it comes to wild conditions, Facial Expression Recognition is often challenged with low-quality data and imbalanced, ambiguous labels. This field has much benefited from CNN based approaches; however, CNN models have structural limitations to see the facial regions in distance. As a remedy, Transformer has been introduced to vision fields with a global receptive field but requires adjusting input spatial size to the pretrained models to enjoy its strong inductive bias at hands. We herein raise the question of whether using the deterministic interpolation method is enough to feed low-resolution data to Transformer. In this work, we propose a novel training framework, Neural Resizer, to support Transformer by compensating information and downscaling in a data-driven manner trained with loss function balancing the noisiness and imbalance. Experiments show our Neural Resizer with F-PDLS loss function improves the performance with Transformer variants in general and nearly achieves the state-of-the-art performance.
CPT: Cross-Modal Prefix-Tuning for Speech-To-Text Translation

Speech translation models benefit from adapting multilingual pretrained language models. However, such adaptation modifies the parameters in the pretrained model to favor a specific task. Prefix-tuning, as a lightweight adaptation technique, has recently emerged as an efficient adaptation method that significantly reduces the number of trainable parameters and has demonstrated great potential in low-resource settings. It inserts prefixes into the output of each layer of a pretrained model, without modifying its parameters. During training, only the parameters of prefixes are updated while the rest of the model are being frozen. In this paper, we improve the performance of speech translation in medium-/low-resource settings by a cross-modal prefix that bridges the gap between speech input and translation modules to reduce the information loss in the cascaded model. We show that the proposed cross-modal prefix-tuning is effective, robust and parameter-efficient for adapting a speech recognition and translation pipeline.
CDMA: Cross-Domain Distance Metric Adaptation for Speaker Verification

To solve the domain shift problem in speaker verification, one effective domain adaptation approach is to learn domain-invariant embeddings via aligning the source and target distributions in the embedding space. However, this approach could be problematic when the source and target domains are from the disjoint speaker label spaces as the embedding distributions of different speakers cannot be aligned. In this paper, we propose a Cross-domain Distance Metric Adaptation (CDMA) approach to alleviate the domain shift in the distance metric space, where the source and target domains share the same classes, i.e., within- and between-speaker. Specifically, the two target pairwise distance distributions are aligned with the source pairwise distance distributions and further separated to learn a domain-invariant metric, which is more suitable for speaker verification based on metric learning. Experiments indicate that CDMA significantly outperforms the approach proposed in the embedding space.
Deep discriminative learning model with calibrated attention map for the automated diagnosis of diffuse large B-cell lymphoma

Blood cancer is a potentially fatal disease that results from abnormal blood cell production and function. It encompasses a wide range of cancers, including cancers of the bone marrow, blood, and lymphatic systems. The uncontrolled growth of abnormal blood cells interrupts the process of blood cell development. As a result, the blood cell is unable to perform a variety of functions, including fighting infections and preventing severe bleeding. This necessitates early detection and treatment. For detecting hematopoietic malignancies, histopathology is the gold standard. Manual examination of Hematoxylin-Eosin (H&E) stained histopathological images has been a time-consuming and challenging task. As a result, automatic processing of these histopathological images is critical. In distinguishing more complex histopathological images, the discriminative ability of feature representations extracted from Convolutional Neural Networks (CNNs) remains limited. Keeping this in mind, the authors have proposed a unified framework for detecting diffuse large B-cell lymphoma (DLBCL) from histopathological images based on a deep discriminative learning model with calibrated attention map (DDLM-CAM). A two-channel CNN architecture is proposed, that fused the features obtained from input images and attention map feature transformer (AMFT). The influence of inter-class dispersion and within-class variance is reduced by optimizing a new objective function that considers both center and cross-entropy loss. 700 images have been used for training and 300 images for validation out of 1000 sparsely annotated images. The proposed model achieved accuracy, recall, precision, and specificity of 96%, 94.67%, 97.26% and 97.33%, respectively on the validation set. Thus, the proposed framework would be very helpful to the doctors and histopathologists in speeding up the diagnostic process.