Reviewerly/exHarmony huggingface dataset:
Insulator Defect Detection Based on Improved Faster R-CNN

In recent years, deep learning has been widely used to identify defects of insulators. This paper proposed an improved Faster R-CNN model based on deep learning to improve the accuracy of fault detection. This method is based on the original Faster R-CNN detection framework to make three improvements: First, ResNet50 is selected to replace VGGNet16 as the feature extraction network. Secondly, the feature pyramid network is used for feature fusion. Thirdly, RoIAlign is used to replace RoIPooling network to reduce the impact of quantization. The dataset in the experiment is 720 marked UAV aerial insulator images, which were divided into training set and test set according to the ratio of 8:2. The mAP of the improved network model reached 84.37%. Compared with the original framework, mAP increased by 7.52%. The results show that the improved network reduced the missed detection rate and false detection rate. On the basis of improving the recognition accuracy, it can better meet the needs of high accuracy in actual scenarios.
A Vibrant Multiple Object Detection Using Machine Learning Techniques

Dynamic Multiple object detection is significantly a difficult task in object detection. In this paper, we’ve done object detection in which there is a single object in the whole image. The CNN network has taken an image and has given the output of two things (i.e.) The class or category of the objects like a dog, person, etc. And the second is the bounding box coordinates and the detection of multiple objects in a single frame, the existing methods do not perform well. An R-CNN is a region-based convolutional network it has achieved success in region-based feature extraction, and the part filters in a DPM it is a deformable part-based model that is very suitable for detecting hidden objects.
Towards privacy-preserving and verifiable federated matrix factorization

Recent years have witnessed the rapid growth of federated learning (FL), an emerging privacy-aware machine learning paradigm that allows collaborative learning over isolated datasets distributed across multiple participants. The salient feature of FL is that the participants can keep their private datasets local and only share model updates. Very recently, some research efforts have been initiated to explore the applicability of FL for matrix factorization (MF), a prevalent method used in modern recommendation systems and services. It has been shown that sharing the gradient updates in federated MF entails privacy risks on revealing users’ personal ratings, posing a demand for protecting the shared gradients. Prior art is limited in that they incur notable accuracy loss, or rely on heavy cryptosystem, with a weak threat model assumed. In this paper, we propose VPFedMF, a new design aimed at privacy-preserving and verifiable federated MF. VPFedMF provides guarantees on the confidentiality of individual gradient updates through lightweight and secure aggregation. Moreover, VPFedMF ambitiously and newly supports correctness verification of the aggregation results produced by the coordinating server in federated MF. Experiments on a real-world movie rating dataset demonstrate the practical performance of VPFedMF in terms of computation, communication, and accuracy.
Real-time wildfire detection with semantic explanations

Wildfire detection is an indispensable component of many resilient platforms, preventing environmental disasters from damaging life.Online detection, which refers to situations in which wildfire events need to be detected in real time, is an essential tool, as the consequences of wildfires might already be irreversible by the time they are detected.Recently, wildfire detection methods based on machine learning and deep learning have been proposed.However, most of these methods take raw data as the input and then aggregate it into predictive features, which requires all of the historical data or a large part of a stream and violates the timeliness requirement of online detection.Moreover, these methods scarcely discuss the explainability (e.g., how the model comes up with the final detection).The lack of explainability reduces human trust in the system and may hinder further applications of the system, especially in high-stakes situations where decisions can have significant consequences.Unlike existing works that treat the timeliness and explainability problems separately, we propose a real-time wildfire detection system that is built upon the streaming capability of complex event processing and the expressiveness of semantic annotation.The system continuously processes raw data streams, transforms them into semantic events, and learns CEP queries that are both predictive and interpretable for humans at the same time.Experiments on four real datasets and one synthetic dataset show that our approach outperforms the baselines in terms of effciency, accuracy, explainability, and adaptivity.
Recommender system information trustworthiness: The role of perceived ability to learn, self-extension, and intelligence cues

Making consumers trust in product descriptions provided by recommender systems is important for marketers, managers, and IT developers dealing with AI-based recommenders. The previous related research has focused on the relationship between the personalization of a recommender system's offering and consumer trust in the system. This paper aims to extend this literature by investigating how the perceived recommender system ability to learn influences the perceived trustworthiness of the recommended product descriptions. Additionally, it is studied what role the consumer self-extension into system recommendations and intelligence cues play in this relationship. Two studies (Study 1 and Study 2) were conducted among young adults participating in an online research panel. Both studies used smartphones as a product category. All study participants were exposed to the same set of product descriptions ostensibly provided by recommender systems. Study 1 surveyed actual Facebook users (N = 204) and assumed Facebook as a recommender system. Perceived ability to learn, self-extension, and perceived trustworthiness were rated by the participants, and the measurements were analyzed with a mediation model. Perceived ability to learn had a positive effect on perceived description trustworthiness, and self-extension mediated this relationship. Study 2 (N = 515) was an experiment. The participants were exposed to a fictitious recommender system. Ability-to-learn cues (like asking about preferences; present vs. absent) and anthropomorphic cues (like first-person and conversation-like communication; present vs. absent) were manipulated between-subject. Perceived ability to learn, perceived anthropomorphism, and trustworthiness were rated by the participants, and the measurements were analyzed with ANOVA and moderated mediation models. Ability-to-learn cues × anthropomorphism cues interaction effect occurred on perceived ability to learn and trustworthiness. The positive effect of ability-to-learn cues on perceived ability to learn (perceived trustworthiness) was more positive (occurred only) in the presence of anthropomorphic cues. The paper extends the existing knowledge on consumer response to recommender systems by linking perceived system ability to learn and perceived recommended product description trustworthiness (with the evaluated offerings unchanged). Moreover, the paper provides novel insights into the role of self-extension and system intelligence cues in building consumer trust in recommender systems. The results may guide marketers, managers, IT developers, and policymakers dealing with AI-based systems through the possible consequences of developing AI technology in recommender systems.
A Gaussian Mixture Model for Dialogue Generation with Dynamic Parameter Sharing Strategy

Existing dialog models are trained with data in an encoder-decoder framework with the same parameters, ignoring the multinomial distribution nature in the dataset. In fact, model improvement and development commonly requires fine-grained modeling on individual data subsets. However, collecting a labeled fine-grained dialogue dataset often requires expert-level domain knowledge and therefore is difficult to scale in the real world. As we focus on better modeling multinomial data for dialog generation, we study an approach that combines the unsupervised clustering and generative model together with a GMM (Gaussian Mixture Model) based encoder-decoder framework. Specifically, our model samples from the prior and recognition distributions over the latent variables by a Gaussian mixture network and the latent layer with the capability to form multiple clusters. We also introduce knowledge distillation to guide and improve the clustering results. Finally, we use a dynamic parameter sharing strategy conditioned on different labels to train different decoders. Experimental results on a widely used dialogue dataset verify the effectiveness of the proposed method.
Joint Centrality Estimation and Graph Identification from Mixture of Low Pass Graph Signals

This paper proposes a mixture model of low pass filtered graph signals. Our aim is to jointly estimate the eigen-centrality vectors for the underlying graphs and identify the graph signal samples with their corresponding graphs, without knowing the graph topology a-priori. The problem is challenging as the observed graph signals lack any obvious identity with their associated graphs. We leverage a low-rank plus sparse structure of the unknown parameters to de-rive a customized expectation-maximization (EM) algorithm for the joint problem. Our algorithm assumes general excitation and does not require prior knowledge of the graph topology. Numerical experiments show the efficacy of our customized EM algorithm.
Custom Attribution Loss for Improving Generalization and Interpretability of Deepfake Detection

The simplicity and accessibility of tools for generating deepfakes pose a significant technical challenge for their detection and filtering. Many of the recently proposed methods for deeptake detection focus on a ‘blackbox’ approach and therefore suffer from the lack of any additional information about the nature of fake videos beyond the fake or not fake labels. In this paper, we approach deepfake detection by solving the related problem of attribution, where the goal is to distinguish each separate type of a deepfake attack. We design a training approach with customized Triplet and ArcFace losses that allow to improve the accuracy of deepfake detection on several publicly available datasets, including Google and Jigsaw, FaceForensics++, HifiFace, DeeperForensics, Celeb-DF, DeepfakeTIMIT, and DF-Mobio. Using an example of Xception net as an underlying architecture, we also demonstrate that when trained for attribution, the model can be used as a tool to analyze the deepfake space and to compare it with the space of original videos.
Distribution Learning for Age Estimation from Speech

Age estimation from speech is becoming important with increasing usage of the voice channel. Call centers can use age estimates to influence call routing or to provide security by comparison with the speaker's age on-file. Voice assistants can use it for parental control applications. The problem of age estimation from speech has been often viewed as a regression or classification problem. However these methods do not explicitly incorporate ordinal ranking or uncertainty in age estimation that humans often do. In this work, we hypothesize that the age follows a normal distribution centered around the real age with a particular confidence interval. We investigate three different distribution learning losses, namely KL divergence, GJM distance and mean-and-variance loss. Cross-dataset experiments were conducted on the NIST SRE08/10 and AgeVoxCeleb data, and their results show that the distribution learning methods are very competitive and in most cases better than traditional approaches.
Combining Multiple Style Transfer Networks and Transfer Learning For LGE-CMR Segmentation

This paper presents an algorithm for segmenting late gadolinium enhancement cardiac magnetic resonance (LGE-CMR) in the absence of labeled training data. The proposed method includes a data augmentation part and a segmentation network. Multiple style transfer networks are employed for data augmentation to increase the data diversity, and then the synthetic images are used for training an improved U-Net. Finally, the trained model is fine-tuned with a few LGE images and labels. Experiment results demonstrate the effectiveness and advantages of the proposed method.
Generalized Zero-Shot Learning Using Conditional Wasserstein Autoencoder

Generalized zero-shot learning (GZSL) is a technique to train a deep learning model to identify unseen classes. Conventionally, conditional generative models have been employed to generate training data for unseen classes from the attribute. In this paper, we propose a new conditional generative model that improves the GZSL performance greatly. In a nutshell, the proposed model, called conditional Wasserstein autoencoder (CWAE), minimizes the Wasserstein distance between the real and generated image feature distributions using an encoder-decoder architecture. From the extensive experiments on various benchmark datasets, we show that the proposed CWAE outperforms conventional generative models in terms of the GZSL classification performance.
Dual Attention Pooling Network for Recording Device Classification Using Neutral and Whispered Speech

In this work, we proposed a method for recording device classification using the recorded speech signal. With the rapid increase in different mobile and professional recording devices, determining the source device has many applications in forensics and in further improving various speech-based applications. This paper proposes dual and single attention pooling-based convolutional neural networks (CNN) for recording device classification using neutral and whispered speech. Experiments using five recording devices with simultaneous direct recordings from 88 speakers speaking both in neutral and whisper and recordings from 21 mobile devices with simultaneous playback recordings reveal that the proposed dual attention pooling based CNN method performs better than the best baseline scheme. We show that we achieve a better performance in recording device classification with whispered speech recordings than corresponding neutral speech. We also demonstrate the importance of voiced/unvoiced speech and different frequency bands in classifying the recording devices.
Contrastive Knowledge Graph Attention Network for Request-Based Recipe Recommendation

To improve daily customer experience, kitchen assistant becomes one of the enabled service in intelligent voice assistants, presenting personalized and relevant recipes to satisfy customer requests. Current solutions for recipe recommendation suffers from two limitations: First, user-recipe interactions are modeled in a uniform manner, which neglects the diversity of user preferences on recipe adoptions, inherently hurting the model performance. Second, users may interact with recipe randomly, resulting in inevitable data noise issue. In this work, we alleviate the foregoing issues by proposing contrastive knowledge graph attention network for recipe recommendation, where a knowledge graph attention-based recommender helps learn fine-grained user and recipe embeddings by modeling diversified user preferences from user behaviors. Moreover, a contrastive learning module that integrates unsupervised and supervised contrastive learning is proposed to improve model robustness. The experimental results on two real-world datasets show that the proposed approach outperforms the state-of-the-art baseline methods.
Reformulating Speaker Diarization As Community Detection With Emphasis On Topological Structure

Clustering-based speaker diarization has stood firm as one of the major approaches in reality, despite recent development in end-to-end diarization. However, clustering methods have not been explored extensively for speaker diarization. Commonly-used methods such as k-means, spectral clustering, and agglomerative hierarchical clustering only take into account properties such as proximity and relative densities. In this paper we propose to view clustering-based diarization as a community detection problem. By doing so the topological structure is considered. This work has four major contributions. First it is shown that Leiden community detection algorithm significantly outperforms the previous methods on the clustering of speaker-segments. Second, we propose to use uniform manifold approximation to reduce dimension while retaining global and local topological structure. Third, a masked filtering approach is introduced to extract "clean" speaker embeddings. Finally, the community structure is applied to an end-to-end post-processing network to obtain diarization results. The final system presents a relative DER reduction of up to 70 percent. The breakdown contribution of each component is analyzed.
Dual-Attention Network for Few-Shot Segmentation

Few-shot segmentation aims at segmenting target object areas with only a few labeled samples. Previous methods extract class-specific prototypes to guide segmentation. How-ever, using one or more prototypes to represent the whole object inevitably drops vital spatial information, ignoring many details in original images. To address the issue, we propose a Dual-Attention Network (DANet) for few-shot segmentation. Firstly, a light-dense attention module is proposed to set up pixel-wise relations between feature pairs at different levels to activate object regions, which can leverage semantic information in a coarse-to-fine manner. Secondly, in contrast to the previous prototype-based methods that offer a holistic representation for each object class, we propose a prototypical channel attention module which incorporates channel interdependencies to enhance the discriminative capacity of features. The extensive experiments on two benchmarks show that our approach outperforms the state-of-the-arts in most cases.
Zero-Shot Cross-Lingual Transfer Using Multi-Stream Encoder and Efficient Speaker Representation

We propose a novel method for zero-shot cross-lingual TTS task by using multi-stream text encoder and efficient speaker representation. Specifically, a unified multi-stream text encoder that takes both advantages of Transformer and CBHG is proposed to retain multiple hypotheses about input representations. For Transformer based stream, a multi-stream Transformer is further proposed to strengthen these hypotheses. Then the speaker representations are extracted from audio signals by a speaker encoder with a random sampling mechanism and a language adversarial loss, aiming to extract speaker embedding features that are independent of both content information and language identity. Meanwhile, we propose an efficient zero-shot cross-lingual transfer strategy with the help of other target lingual speakers’ data and a language-balanced sampling strategy. The Experimental results show the proposed method not only could achieve higher speech quality and speaker similarity (with an average absolute improvement of 0.38 and 0.27 in MOS respectively) for zero-shot cross-lingual transfer, but also helpful for few-shot cross-lingual transfer in which has multi-lingual data. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>
Robust Self-Supervised Speaker Representation Learning Via Instance Mix Regularization

Over the recent years, various self-supervised contrastive embedding learning methods for deep speaker verification were proposed. The performance of the self-supervised contrastive learning framework highly depends on the data augmentation technique, but due to the sensitive nature of speaker information within the speech signal, most speaker embedding training relies on simple augmentations such as additive noise or simulated reverberation. Thus while the conventional self-supervised speaker embedding systems can yield minimum within-utterance variability, the capability to generalize to out-of-set utterance is limited. In order to alleviate this problem, we propose a novel self-supervised learning framework for speaker verification which combines the angular prototypical loss and the instance mix (i-mix) regularization. The proposed method was evaluated on the VoxCeleb1 dataset and showed noticeable improvement over the standard self-supervised embedding method.
Context-Aware Graph-Based Self-Supervised Learning of Whole Slide Images

The gigapixel resolution of a single whole slide image (WSI), and the lack of huge annotated datasets needed for computational pathology, makes cancer diagnosis and grading with WSIs a challenging task. Moreover, downsampling of WSIs might result in loss of information critical for cancer diagnosis. Motivated by the fact that context such as topological structures in the tumor environment may contain critical information in cancer grading and diagnosis, a novel two-stage learning approach is proposed. Self-supervised learning is applied to improve training through unlabled data and graph convolutional network (GCN) is deployed to incorporate context from tumor and surrounding tissues. More specifically, we represent the whole slide as a graph, where nodes are patches from the WSIs. The patches in the graph are represented as feature vectors obtained from pre-training the patches in self-supervised learning. The graph is trained using GCN which accounts for the context of each tissue for the cancer grading and classification. In this work, WSIs for prostrate cancer are validated and the model performance is evaluated based on diagnosis and grading of prostrate cancer and compared with ResNet50 as a traditional convolutional neural network (CNN) and multi-instance learning (MIL) as a leading approach in WSI diagnosis.
From Bottom-Up To Top-Down: Characterization Of Training Process In Gaze Modeling

During training, artificial neural networks might not converge to a global minimum. Usually, using gradient descent, the training procedure cause the network to stroll in the high-dimensional weights’ space. This stroll passes adjacently to local minima and locations in the geometry of loss landscape associated with low loss. Overall, the network moves from one low loss area to a lower loss area.In this work, we explored those low loss areas and minima, and tried to understand them. A U-Net was trained based on a gaze prediction task. A network was presented with images of different scenes, and the purpose of the network was to predict the expected human gaze distribution over those images. The driving task was selected since it involves relatively strong goal-oriented behaviors. It was shown that the training had two stages: (1) At the beginning, the network selected area was associated with saliency distributions (bottom-up behavior); (2) Later, the network selected area had the characteristics of goal-oriented distributions (top-down behavior) and it shifted away from the saliency distributions.
Randomized Smoothing Under Attack: How Good is it in Practice?

Randomized smoothing is a recent and celebrated solution to certify the robustness of any classifier. While it indeed provides a theoretical robustness against adversarial attacks, the dimensionality of current classifiers necessarily imposes Monte Carlo approaches for its application in practice.This paper questions the effectiveness of randomized smoothing as a defense, against state of the art black-box attacks. This is a novel perspective, as previous research works considered the certification as an unquestionable guarantee. We first formally highlight the mismatch between a theoretical certification and the practice of attacks on classifiers. We then perform attacks on randomized smoothing as a defense. Our main observation is that there is a major mismatch in the settings of the RS for obtaining high certified robustness or when defeating black box attacks while preserving the classifier accuracy.