Reviewerly/exHarmony huggingface dataset:
Dynamically Pruning Segformer for Efficient Semantic Segmentation

As one of the successful Transformer-based models in computer vision tasks, SegFormer demonstrates superior performance in semantic segmentation. Nevertheless, the high computational cost greatly challenges the deployment of SegFormer on edge devices. In this paper, we seek to design a lightweight SegFormer for efficient semantic segmentation. Based on the observation that neurons in SegFormer layers exhibit large variances across different images, we propose a dynamic gated linear layer, which prunes the most uninformative set of neurons based on the input instance. To improve the dynamically pruned SegFormer, we also introduce two-stage knowledge distillation to transfer the knowledge within the original teacher to the pruned student network. Experimental results show that our method can significantly reduce the computation overhead of SegFormer without an apparent performance drop. For instance, we can achieve 36.9% mIoU with only 3.3G FLOPs on ADE20K, saving more than 60% computation with the drop of only 0.5% in mIoU.
Topography measurement of helical grooves on a hemisphere based on stitching interference microscopy

Hemispherical bearings fabricated with helical grooves on the working surface show improved aerodynamic performance which depends to a large extent on the machining accuracy of the grooves. Topography measurement of the entire groove from macro to micro scale is thus important to evaluate the bearing performance. White light interference microscopy is widely used in topography measurement. However, subaperture stitching is necessary to extend the field of view (FOV) for large area measurement but limited to planar surfaces. This paper presents a method of interference microscopy combined with subaperture stitching. The long groove is divided into a series of subapertures with overlapping regions. Each subaperture meets the requirement of FOV and measurable slope limit of the microscope. We show that the helical curve is approximately located on a geodesic circle of the hemisphere. Accordingly a special fixture is designed with a rotary table to rotate the groove around the axis perpendicular to the geodesic circle. A four-axis scanning platform is integrated with the interference microscope. Based on the principle of minimal deviations in overlapping regions, six-degree-of-freedom motion error introduced in subaperture scanning is simultaneously corrected through an iterative stitching algorithm. It is experimentally verified by measuring a real part with helical grooves etched on a hemisphere bearing rotor. Complete high-resolution topography of the entire groove is reconstructed and the stitching map shows no overlap sliding effect, which is instructive for assessment of the machining error and further evaluation of the bearing performance.
Adversarial Examples Generation for Deep Product Quantization Networks on Image Retrieval

Deep product quantization networks (DPQNs) have been successfully used in image retrieval tasks, due to their powerful feature extraction ability and high efficiency of encoding high-dimensional visual features. Recent studies show that deep neural networks (DNNs) are vulnerable to input with small and maliciously designed perturbations (a.k.a., adversarial examples) for classification. However, little effort has been devoted to investigating how adversarial examples affect DPQNs, which raises the potential safety hazard when deploying DPQNs in a commercial search engine. To this end, we propose an adversarial example generation framework by generating adversarial query images for DPQN-based retrieval systems. Unlike the adversarial generation for the classic image classification task that heavily relies on ground-truth labels, we alternatively perturb the probability distribution of centroids assignments for a clean query, then we can induce effective non-targeted attacks on DPQNs in white-box and black-box settings. Moreover, we further extend the non-targeted attack to a targeted attack by a novel sample space averaging scheme ([Formula: see text]AS), whose theoretical guarantee is also obtained. Extensive experiments show that our methods can create adversarial examples to successfully mislead the target DPQNs. Besides, we found that our methods both significantly degrade the retrieval performance under a wide variety of experimental settings. The source code is available at https://github.com/Kira0096/PQAG.
Continual Learning in Real-Life Applications

Existing Continual Learning benchmarks only partially address the complexity of real-life applications, limiting the realism of learning agents. In this letter, we propose and focus on benchmarks characterized by common key elements of real-life scenarios, including temporally ordered streams as input data, strong correlation of samples in short time ranges, high data distribution drift over the long time frame, and heavy class unbalancing. Moreover, we enforce online training constraints such as the need for frequent model updates without the possibility of storing a large amount of past data or passing the dataset multiple times through the model. Besides, we introduce a novel hybrid approach based on Continual Learning, whose architectural elements and replay memory management proved to be useful and effective in the considered scenarios. The experimental validation carried out, including comparisons with existing methods and an ablation study, confirms the validity and the suitability of the proposed approach.
Convolutional Neural Networks Hyperparameter Tunning for Classifying Firearms on Images

In recent years, the rates of firearm-related violent acts worldwide have risen significantly. It is a severe social problem that compromises the safety of every individual to some extent. This situation has motivated researchers to find new ways to improve the current state-of-the-art solutions, such as automatic surveillance systems, to detect and classify the presence of firearms within a specific scene. These systems reduce the drawbacks of using direct human supervision. Among the available solutions for the classification task, the performance of Deep Learning models stands out, especially those based on Convolutional Neural Networks. Since they start learning directly from raw data (e.g., images), their learning process can be improved even further by using Transfer Learning techniques. However, the classification accuracy depends significantly on choosing the optimum set of values for the different hyperparameters composing them. Thus, this paper analyses the improvement in the performance of an image-based handgun classification algorithm when tuning its hypermeters values instead of using its default values. In this work, we evaluated the performance variation using two benchmarks Convolutional Neural Networks architectures: AlexNet and Inception V3. We obtained a maximum accuracy of 94.11% when using the Inception V3 network and transfer learning. We employed Nadam as the optimizer and a learning rate equal to 0.0001, a batch size equal 256, and a total of 13 epochs. Experimental results suggest an essential relationship between the performance of the classification model and the data set, the specific combinations of values for the selected optimizer, the batch size, and the learning rate. The obtained improvement in the accuracy was up to 10.33% after the tuning process.
Non-Uniform Attention Network for Multi-modal Sentiment Analysis

Remarkable success has been achieved in the multi-modal sentiment analysis community thanks to the existence of annotated multi-modal data sets. However, coming from three different modalities, text, sound, and vision, establishes significant barriers for better feature fusion. In this paper, we introduce "NUAN", a non-uniform attention network for multi-modal feature fusion. NUAN is designed based on attention mechanism via considering three modalities simultaneously, but not uniformly: the text is seen as a determinate representation, with the hope that by leveraging the acoustic and visual representation, we are able to inject the effective information into a solid representation, named as tripartite interaction representation. A novel non-uniform attention module is inserted into adjacent time steps in LSTM (Long Shot-Term Memory) and processes information recurrently. The final outputs of LSTM and NUAM are concatenated to a vector, which is imported into a linear embedding layer to output the sentiment analysis result. The experimental analysis of two databases demonstrates the effectiveness of the proposed method.
A UAV Coverage Path Planning Algorithm Based on Double Deep Q-Network

Abstract The UAV path planning method has practical value in the military field and automated production. Based on deep reinforcement learning theory and the characteristics of coverage path planning, this paper designs and implements a set of deep reinforcement learning frameworks suitable for UAV coverage path planning and trains it in the abstract environment model built. The simulation experiment results show that the designed UAV coverage path planning frameworks can consider obstacles, no-fly zones and length constraints, plan a reasonable path to complete the coverage task, and have a certain generalization ability.
Cooking Well, With Love, Is an Art: Transformers on Youtube Hinglish Data

Youtube gives a place for viewers to express their views on a variety of issues. This study focuses on opinion mining from a dataset containing comments from Nisha Madulika’s and Kabita’s Kitchen’s YouTube culinary channels. The information includes Mix code comments in both English and a Hindi-English hybrid known as Hinglish. On both datasets, variations of the Bidirectional Encoder Representations from Transformers (BERT) have been trained. BERT outperformed the baseline trained on machine learning in terms of outcomes. The results show that the innovative BERT models (84.82% and 88.37%) are likely to outperform on Hinglish datasets with fine-tuning.
Stable Prediction with Leveraging Seed Variable

In this paper, we focus on the problem of stable prediction across unknown test data, where the test distribution might be different from the training one and is always agnostic when model training. In such a case, previous machine learning methods might exploit subtly spurious correlations induced by non-causal variables in training data for prediction. Those spurious correlations are changeable across data, leading to instability of prediction across unknown test data. To address this problem, we propose a conditional independence test based algorithm to screen out part of non-causal features and reduce those spurious correlations for a more stable prediction by leveraging a seed variable. We show, both theoretically and with empirical experiments, that our algorithm can precisely screen out the isolated non-causal variables, which have no causal relationship with other variables, and remove the spurious correlations induced by them, increasing the stability of prediction across unknown test data. Extensive experiments on both synthetic and real-world datasets demonstrate that our algorithm outperforms state-of-the-art methods for stable prediction across unknown test data.
Explainable Artificial Intelligence in Meteorology and Climate Science: Model Fine-Tuning, Calibrating Trust and Learning New Science

Abstract In recent years, artificial intelligence and specifically artificial neural networks (NNs) have shown great success in solving complex, nonlinear problems in earth sciences. Despite their success, the strategies upon which NNs make decisions are hard to decipher, which prevents scientists from interpreting and building trust in the NN predictions; a highly desired and necessary condition for the further use and exploitation of NNs’ potential. Thus, a variety of methods have been recently introduced with the aim of attributing the NN predictions to specific features in the input space and explaining their strategy. The so-called eXplainable Artificial Intelligence (XAI) is already seeing great application in a plethora of fields, offering promising results and insights about the decision strategies of NNs. Here, we provide an overview of the most recent work from our group, applying XAI to meteorology and climate science. Specifically, we present results from satellite applications that include weather phenomena identification and image to image translation, applications to climate prediction at subseasonal to decadal timescales, and detection of forced climatic changes and anthropogenic footprint. We also summarize a recently introduced synthetic benchmark dataset that can be used to improve our understanding of different XAI methods and introduce objectivity into the assessment of their fidelity. With this overview, we aim to illustrate how gaining accurate insights about the NN decision strategy can help climate scientists and meteorologists improve practices in fine-tuning model architectures, calibrating trust in climate and weather prediction and attribution, and learning new science.
Ambient-noise Free Generation of Clean Underwater Ship Engine Audios from Hydrophones using Generative Adversarial Networks

• An audio denoising GAN framework for underwater acoustic signal denoising • The AN-GAN generates clean data and distinguishes the real measurement from a fake. • Proposed AN-GAN incorporates a fully convolutional generator for generating clean data • Integrates a patch discriminator in time domain. • The proposed network removes ambient noise and improved SNR . Generative adversarial networks (GANs) have been extensively used in image domain showing promising results in generating and learning data distributions in the absence of clean data. However, the audio domain, specially underwater acoustics are not yet fully explored in reporting the efficiency and applicability of GANs. We propose an audio GAN framework called ambient noise-free GAN (AN-GAN) to address the underwater acoustic signal denoising problem by removing the background ambient noise. The proposed AN-GAN can learn a clean audio generation with improved signal-to-noise ratio (SNR) given only the noisy samples from the underwater audio dataset. The simulated and real-time data collected from online available source ShipsEar, is used for the analysis and validation purpose. The comparative analysis shows an average percentage improvement of proposed AN-GAN with GAN-based and conventional statistical underwater denoising methods as 6.27% for UWAR-GAN, 227% for Wavelet denoising, 247% for EMD and 65% for Wiener technique.
Competing Pedagogies for the Biodigital Imaginary: What Will Happen to Teachers?

In the social imaginaries that impact on education at all levels, pedagogies compete for dominance. Within the associated narratives and myths, teachers range from fusty disciplinarians to creative and inspirational mavericks, or are even replaced by robots. One-to-one dialogue with an inspirational teacher has long been regarded as a pedagogical ideal, but is incompatible with democratic approaches to education, and the social imaginary looks to innovation and technology to make up the shortfall. When infiltrated by the neoliberal imaginary, the educational imaginary again foregrounds a desire for personalized learning, but provided by commercial organizations rather than teachers. The chapter reviews the workings of social imaginaries identified in academic literature, with particular attention to pedagogy and the agency of teachers and to what may be lost through the erosion of both. There are alternative imagined futures relevant to the themes of this book, both in academic theories and the wider culture, and a biodigital imaginary may be emerging. The chapter concludes with an argument for the necessary retention of teachers in our biodigital future, to navigate the tensions, contradictions and losses exposed in the analysis and to ensure that multiple pedagogies survive.
NeuralKG: An Open Source Library for Diverse Representation Learning of Knowledge Graphs

NeuralKG is an open-source Python-based library for diverse representation learning of knowledge graphs. It implements three kinds of Knowledge Graph Embedding (KGE) methods, including conventional KGEs, GNN-based KGEs, and Rule-based KGEs. With a unified framework, NeuralKG successfully reproduces link prediction results of these methods on benchmarks, freeing users from the laborious task of reimplementing them, especially for some methods originally written in non-python programming languages. Besides, NeuralKG is highly configurable and extensible. It provides various decoupled modules that can be mixed and adapted to each other. Thus with NeuralKG, developers and researchers can quickly implement their own designed models and obtain the optimal training methods to achieve the best performance efficiently. We built a website http://neuralkg.zjukg.org to organize an open and shared KG representation learning community. The library, experimental methodologies, and model reimplement results of NeuralKG are all publicly released at https://github.com/zjukg/NeuralKG.
Enhancing Backdoor Attacks With Multi-Level MMD Regularization

While Deep Neural Networks (DNNs) excel in many tasks, the huge training resources they require become an obstacle for practitioners to develop their own models. It has become common to collect data from the Internet or hire a third party to train models. Unfortunately, recent studies have shown that these operations provide a viable pathway for maliciously injecting hidden backdoors into DNNs. Several defense methods have been developed to detect malicious samples, with the common assumption that the latent representations of benign and malicious samples extracted by the infected model exhibit different distributions. However, it is still an open question whether this assumption holds up. In this article, we investigate such differences thoroughly via answering three questions: 1) What are the characteristics of the distributional differences? 2) How can they be effectively reduced? 3) What impact does this reduction have on difference-based defense methods? First, the distributional differences of multi-level representations on the regularly trained backdoored models are verified to be significant by adopting Maximum Mean Discrepancy (MMD), Energy Distance (ED), and Sliced Wasserstein Distance (SWD) as the metrics. Then, ML-MMDR, a difference reduction method that adds multi-level MMD regularization into the loss, is proposed, and its effectiveness is testified on three typical difference-based defense methods. Across all the experimental settings, the F1 scores of these methods drop from 90%-100% on the regularly trained backdoored models to 60%-70% on the models trained with ML-MMDR. These results indicate that the proposed MMD regularization can enhance the stealthiness of existing backdoor attack methods. The prototype code of our method is now available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/xpf/Multi-Level-MMD-Regularization</uri> .
Sparse Count Data Clustering Using an Exponential Approximation to Generalized Dirichlet Multinomial Distributions

Clustering frequency vectors is a challenging task on large data sets considering its high dimensionality and sparsity nature. Generalized Dirichlet multinomial (GDM) distribution is a competitive generative model for count data in terms of accuracy, yet its parameters estimation process is slow. The exponential-family approximation of the multivariate Polya distribution has shown to be efficient to train and cluster data directly, without dimensionality reduction. In this article, we derive an exponential-family approximation to the GDM distributions, and we call it (EGDM). A mixture model is developed based on the new member of the exponential-family of distributions, and its parameters are learned through the deterministic annealing expectation-maximization (DAEM) approach as a new clustering algorithm for count data. Moreover, we propose to estimate the optimal number of EGDM mixture components based on the minimum message length (MML) criterion. We have conducted a set of empirical experiments, concerning text, image, and video clustering, to evaluate the proposed approach performance. Results show that the new model attains a superior performance, and it is considerably faster than the corresponding method for GDM distributions.
Multi-agent Systems for Distributed Data Mining Techniques: An Overview

The term “multi-agent systems” (MAS) refers to a mechanism that is used to create goal-oriented autonomous agents in a shared environment and have communication and coordination capabilities. This goal-oriented mechanism supports distributed data mining (DM) to implement various techniques for distributed clustering, classification, and prediction. Different distributed DM (DDM) techniques, MASs, the advantages of MAS-based DDM, and various MAS-based DDM approaches proposed by researchers are reviewed in this study.
Fast neural ranking on bipartite graph indices

Neural network based ranking has been widely adopted owing to its powerful capacity in modeling complex relationships (e.g., users and items, questions and answers). Online neural network ranking, i.e., the so called fast neural ranking, is considered a challenging task because neural network measures are in general non-convex and asymmetric. Traditional approximate near neighbor (ANN) search which typically focuses on metric ranking measures, is not applicable to these complex measures. To tackle this challenge, in this paper, we propose to construct BipartitE Graph INdices (BEGIN) for fast neural ranking. BEGIN contains two types of nodes: base/searching objects and sampled queries. The edges connecting these types of nodes are constructed via the neural network ranking measure. The proposed algorithm is a natural extension from traditional search on graph methods and is more suitable for fast neural ranking. Experiments demonstrate the effectiveness and efficiency of the proposed method.
Efficient Geometry Surface Coding in V-PCC

In recent video-based point cloud compression (V-PCC), 3D point clouds are projected onto 2D images and compressed by High-Efficiency Video Coding (HEVC). However, HEVC was originally designed for natural visual signals, which is a suboptimal framework for point clouds. Therefore, there are still problems in geometry information compression in V-PCC: (1) The distortion based on the sum of squared error (SSE) in the existing rate-distortion optimization (RDO) is inconsistent with the geometric quality measurement; (2) The existing prediction cannot explore the fixed relationship between the corresponding far layer and near layer depth, which means that the far layer depth can be always not less than the corresponding near layer depth. In this paper, we present an efficient geometry surface coding (EGSC) method for V-PCC to address the problems. Firstly, an error projection (EP) model is designed to establish the relationship between the SSE-based distortion and the geometry quality metric. Secondly, an EP-based RDO is employed to improve the geometry information compression by estimating the point normals with gradients. Finally, an occupancy-map driven scheme is proposed to improve the prediction accuracy of merge modes. Experimental results show that the proposed method achieves an average of over 10% bit-rate saving compared with the V-PCC reference software.