Reviewerly/exHarmony huggingface dataset:
GAN Latent Space Manipulation and Aggregation for Federated Learning in Medical Imaging

Federated learning aims at improving data privacy by training local models on distributed nodes and at integrating information on a central node, without data sharing. However, this calls for effective integration methods that are currently missing as existing strategies, e.g., averaging model gradients, are unable to deal with data multimodality due to different distributions at multiple nodes. In this work, we tackle this problem by having multiple nodes that share a synthetic version of their own data, built in a way to hide patient-specific visual cues, with a central node that is responsible for training a deep model for medical image classification. Synthetic data are generated through an aggregation strategy consisting in: 1) learning the distribution of original data via a Generative Adversarial Network (GAN); 2) projecting private data samples in the GAN latent space; 3) clustering the projected samples and generating synthetic images by interpolating the cluster centroids, thus reducing the possibility of collision with latent vectors corresponding to real samples and a consequent leak of sensitive information. The proposed approach is tested over two X-ray datasets for Tuberculosis classification to simulate a realistic scenario with two different nodes and non-i.i.d. data. Experimental results show that our approach yields performance comparable to, or even outperforming, training on the full joint dataset. We also show quantitatively and qualitatively that images synthesized with our approach are significantly different from original images, thus limiting the possibility to recover original data through attacks.
Shared-Private Memory Networks For Multimodal Sentiment Analysis

Text, visual, and acoustic are usually complementary in the Multimodal Sentiment Analysis (MSA) task. However, current methods primarily concern shared representations while neglecting the critical private aspects of data within individual modalities. In this work, we propose shared-private memory networks based on the recent advances in the attention mechanism, called SPMN, to decouple multimodal representation from shared and private perspectives. It contains three components: a) a shared memory to learn the shared representations of multimodal data; b) three private memories to learn the private representations of individual modalities, respectively; c) and adaptive fusion gates to fuse multimodal private and shared representations. To evaluate the effectiveness of SPMN, we integrate it into different pre-trained language representation models, such as BERT and XLNET, and conduct experiments on two public datasets, CMU-MOSI and CMU-MOSEI. Experimental results indicate that the performances of pre-trained language representation models are significantly improved because of SPMN and demonstrate the superiority of our model compared to the state-of-the-art methods. SPMN's source code is publicly available at: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/xiaobaicaihhh/SPMN</uri> .
Communication-Efficient Federated Learning Using Censored Heavy Ball Descent

Distributed machine learning enables scalability and computational offloading, but requires significant levels of communication. Consequently, communication efficiency in distributed learning settings is an important consideration, especially when the communications are wireless and battery-driven devices are employed. In this paper we develop a censoring-based heavy ball (CHB) method for distributed learning in a server-worker architecture. Each worker self-censors unless its local gradient is sufficiently different from the previously transmitted one. The significant practical advantages of the HB method for learning problems are well known, but the question of reducing communications has not been addressed. CHB takes advantage of the HB smoothing to eliminate reporting small changes, and provably achieves a linear convergence rate equivalent to that of the classical HB method for smooth and strongly convex objective functions. The convergence guarantee of CHB is theoretically justified for both convex and nonconvex cases. In addition we prove that, under some conditions, at least half of all communications can be eliminated without any impact on convergence rate. Extensive numerical results validate the communication efficiency of CHB on both synthetic and real datasets, for convex, nonconvex, and nondifferentiable cases. Given a target accuracy, CHB can significantly reduce the number of communications compared to existing algorithms, achieving the same accuracy without slowing down the optimization process.
Graph Attention Network for Financial Aspect-based Sentiment Classification with Contrastive Learning

Aspect-based Sentiment Classification (ASC) task is a challenge in Natural Language Processing (NLP) and is especially important for fields that require detailed analysis like finance. It aims to identify the sentiment polarity of specific aspects in sentences. In addition to tweets and posts directly related to finance, news from such as restaurants and e-commerce may also indirectly affect its stock prices. In previous approaches, attention-based neural network models were mostly adopted to implicitly connect aspects with opinion words for better aspect representations. However, due to the complexity of language and the presence of multiple aspects in a single sentence, these existing models often confuse connections. To tackle this problem, we propose a model named GAS-CL which encodes syntactical structure into aspect representations and refines it with a contrastive loss. Experiments on several datasets confirm that our approach can have better aspect representations and achieve a significant improvement.
Improving the Perceptual Quality of 2D Animation Interpolation

Traditional 2D animation is labor-intensive, often requiring animators to manually draw twelve illustrations per second of movement. While automatic frame interpolation may ease this burden, 2D animation poses additional difficulties compared to photorealistic video. In this work, we address challenges unexplored in previous animation interpolation systems, with a focus on improving perceptual quality. Firstly, we propose SoftsplatLite (SSL), a forward-warping interpolation architecture with fewer trainable parameters and better perceptual performance. Secondly, we design a Distance Transform Module (DTM) that leverages line proximity cues to correct aberrations in difficult solid-color regions. Thirdly, we define a Restricted Relative Linear Discrepancy metric (RRLD) to automate the previously manual training data collection process. Lastly, we explore evaluation of 2D animation generation through a user study, and establish that the LPIPS perceptual metric and chamfer line distance (CD) are more appropriate measures of quality than PSNR and SSIM used in prior art.
Prior-Guided Multi-scale Fusion Transformer for Face Attribute Recognition

Multi-label face attribute recognition (FAR) refers to the task of predicting a set of attribute labels for a facial image. However, existing FAR methods do not work well for recognizing attributes of different scales, since most frameworks use the features of the last layer and ignore the detailed information which is crucial for FAR. To solve this problem, we propose a prior-guided multi-scale fusion transformer, which possesses the ability to build the fusion among features of different scales with prior knowledge of attributes. First, we employ a unifying Graph Convolution Network (GCN) to model the relations between multiple attributes by the prior knowledge of facial labels and the statistical frequencies of co-occurrence between attributes. Second, we propose a multi-scale fusion module, which uses adaptive attention to fuse features from two adjacent layers, and then simultaneously fuse the features of different scales hierarchically to explore the multilevel relation. In addition, we utilize the transformer as a feature extraction module to achieve a global correlation among the acquired features. Experiments on a large-scale face attribute dataset verify the effectiveness of the proposed method both qualitatively and quantitatively.
Learning Structured Gaussians to Approximate Deep Ensembles

This paper proposes using a sparse-structured multivari-ate Gaussian to provide a closed-form approximator for the output of probabilistic ensemble models used for dense im-age prediction tasks. This is achieved through a convolutional neural network that predicts the mean and covari-ance of the distribution, where the inverse covariance is parameterised by a sparsely structured Cholesky matrix. Similarly to distillation approaches, our single network is trained to maximise the probability of samples from pre-trained probabilistic models, in this work we use a fixed en-semble of networks. Once trained, our compact represen-tation can be used to efficiently draw spatially correlated samples from the approximated output distribution. Impor-tantly, this approach captures the uncertainty and struc-tured correlations in the predictions explicitly in a formal distribution, rather than implicitly through sampling alone. This allows direct introspection of the model, enabling vi-sualisation of the learned structure. Moreover, this formu-lation provides two further benefits: estimation of a sample probability, and the introduction of arbitrary spatial conditioning at test time. We demonstrate the merits of our approach on monocular depth estimation and show that the advantages of our approach are obtained with comparable quantitative performance.
Background-Insensitive Scene Text Recognition with Text Semantic Segmentation

Scene Text Recognition (STR) has many important applications in computer vision. Complex backgrounds continue to be a big challenge for STR because they interfere with text feature extraction. Many existing methods use attentional regions, bounding boxes or polygons to reduce such interference. However, the text regions located by these methods still contain much undesirable background interference. In this paper, we propose a Background-Insensitive approach BINet by explicitly leveraging the text Semantic Segmentation (SSN) to extract texts more accurately. SSN is trained on a set of existing segmentation data, whose volume is only 0.03% of STR training data. This prevents the large-scale pixel-level annotations of the STR training data. To effectively utilize the segmentation cues, we design new segmentation refinement and embedding blocks for refining text-masks and reinforcing visual features. Additionally, we propose an efficient pipeline that utilizes Synthetic Initialization (SI) for STR models trained only on real data (1.7% of STR training data), instead of on both synthetic and real data from scratch. Experiments show that the proposed method can recognize text from complex backgrounds more effectively, achieving state-of-the-art performance on several public datasets.
Sustainable Federated Learning with Long-term Online VCG Auction Mechanism

Federated learning (FL) clients may be reluctant to participate in the energy-consuming FL unless they are incentivized. Existing incentive mechanisms seldom consider the economic properties, e.g., social welfare, individual rationality and incentive compatibility, which significantly limits the sustainability of FL to attract more clients. The Vickrey–Clarke–Groves (VCG) auction is an ideal mechanism for simultaneously guaranteeing all crucial economic properties to maximize social welfare. However, VCG auction cannot be applied directly to FL scenarios due to the following challenges: 1) It requires precise analytical derivation of the optimal strategy, which is unavailable due to the inherent model-unknown and privacy-sensitive characteristics of FL. 2) Current auction modeling decomposes the entire process into multiple independent rounds and solves them one-by-one, which breaks the successive correlation between rounds in the long-term training process of FL. To overcome these challenges, this paper presents a long-term online VCG auction mechanism for FL that employs an experience-driven deep reinforcement learning algorithm to obtain the optimal strategy. Besides, we extend long-term forms of the crucial economic properties for the successive FL process. Furthermore, knowledge transfer is applied to reduce the excessive training overhead arising from the VCG payment rules. By exploiting the environmental similarity among sub-auctions, we develop the strategy sharing to significantly cut the training time by half. Finally, we theoretically prove the extended economic properties and conduct extensive experiments on multiple real-world datasets. Compared with state-of-the-art approaches, the long-term social welfare of FL increases by 36% with a 37% reduction in payment.
Accelerating DNN Training with Structured Data Gradient Pruning

Weight pruning is a technique to make Deep Neural Network (DNN) inference more computationally efficient by reducing the number of model parameters over the course of training. However, most weight pruning techniques generally does not speed up DNN training and can even require more iterations to reach model convergence. In this work, we propose a novel Structured Data Gradient Pruning (SDGP) method that can speed up training without impacting model convergence. This approach enforces a specific sparsity structure, where only N out of every M elements in a matrix can be nonzero, making it amenable to hardware acceleration. Modern accelerators such as the Nvidia A100 GPU support this type of structured sparsity for 2 nonzeros per 4 elements in a reduction. Assuming hardware support for 2:4 sparsity, our approach can achieve a 15-25% reduction in total training time without significant impact to performance. Source code and pre-trained models are available at https://github.com/BradMcDanel/sdgp.
BEQAIN: An Effective and Efficient Identifier Normalization Approach With BERT and the Question Answering System

As one of the most important resources to express the semantics of source code, identifiers are usually composed of several common or domain-specific terms and abbreviations, thus heavily hindering developers from analyzing and comprehending source code. Hence, it is very necessary to normalize identifiers, which aims to align the vocabulary found in identifiers with natural language words found in other software artifacts. Even though researchers have proposed several identifier normalization approaches in the literature, these approaches only rely on the lexical information in identifiers and related source code entities to normalize identifiers, suffering from the lack of deep semantic understanding of identifiers. In this paper, we propose an effective and efficient identifier normalization approach BEQAIN to split identifiers into their composing words and expand the enclosed abbreviations. Specifically, BEQAIN employs a deep learning model, which is mainly composed of a Bidirectional Encoder Representation from Transformers (BERT) layer and a Conditional Random Fields (CRF) layer to embed identifiers into low-level vectors and learn the identifier splitting patterns. The BERT-CRF network is also combined with a pre-processing component and a post-processing component to resolve the problems of over-splitting and under-splitting so as to improve the identifier splitting performance. Furthermore, BEQAIN also employs a Question Answering (Q&A) system to learn the abbreviation expansion mappings and leverages the current programming context to determine the exactly correct expansion when there are multiple expansions for specific abbreviations. After BEQAIN is fully trained, it can be used to normalize identifiers. We conduct extensive experiments to validate the effectiveness and efficiency of BEQAIN over two publicly available datasets with nine projects. Experimental results show that BEQAIN achieves the overall average Accuracy of 80.20% and outperforms the existing state-of-the-art approach by 9.88% in normalizing identifiers. The pre-processing and post-processing components could improve the Accuracy of BEQAIN in identifier splitting by 11.70%. Employing the programming context information could improve the Accuracy of BEQAIN in abbreviation expansion by 11.15% on average. In addition, the average normalization time of BEQAIN is less than one second. Finally, we also discuss some observations for the road ahead for identifier normalization to inspire other researchers.
Research on the enhancement of machine fault evaluation model based on data-driven

Recently fault data diagnosis-based deep learning methods have achieved promising results. However, most of these methods' performances are difficult to improve once they have achieved accuracy. This paper mainly uses fusion theory based on data-driven to solve this problem. Firstly, the diagnostic models are divided into feature extraction and neural network. Then, four feature extraction methods are fused by pre-allocation. The neural network part consists of three single models, and the weight of the three output results is determined by regression analysis. Experiments show that the accuracy of diagnostic models is improved. Finally, we combine the two studies and propose a Fusion-Ensemble superposition (FES) model. The AUC value of the model is higher than 98% in most tasks of the DCASE2020 machine failure dataset.
iQuery: A Trustworthy and Scalable Blockchain Analytics Platform

Blockchain, a distributed and shared ledger, provides a credible and transparent solution to increase application auditability by querying the immutable records written in the ledger. Unfortunately, existing query APIs offered by the blockchain are inflexible and unscalable. Some studies propose off-chain solutions to provide more flexible and scalable query services. However, the query service providers (SPs) may deliver fake results without executing the real computation tasks and collude to cheat users. In this paper, we propose a novel intelligent blockchain analytics platform termed <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">iQuery</small> , in which we design a game theory based smart contract to ensure the trustworthiness of the query results at a reasonable monetary cost. Furthermore, the contract introduces the second opinion game that employs a randomized SP selection approach coupled with non-ordered asynchronous querying primitive to prevent collusion. We achieve a fixed price equilibrium, destroy the economic foundation of collusion, and can incentivize all rational SPs to act diligently with proper financial rewards. In particular, <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">iQuery</small> can flexibly support semantic and analytical queries for generic consortium or public blockchains, achieving query scalability to massive blockchain data. Extensive experimental evaluations show that <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">iQuery</small> is significantly faster than state-of-the-art systems. Specifically, in terms of the conditional, analytical, and multi-origin query semantics, <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">iQuery</small> is 2 ×, 7 ×, and 1.5 × faster than advanced blockchain and blockchain databases. Meanwhile, to guarantee 100% trustworthiness, only two copies of query results need to be verified in <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">iQuery</small> , while <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">iQuery</small> 's latency is <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$2 \sim 134$</tex-math></inline-formula> × smaller than the state-of-the-art systems.
Co-contrastive Learning for Multi-behavior Recommendation

Multi-behavior recommender system (MBR) typically utilizes multi-typed user interactive behaviors (e.g., view, add-to-cart and purchase) in learning user preference on target behavior (i.e., purchase). Existing MBR models suffer from sparse supervised signals, which may degrade recommendation performance to some extent. Inspired by contrastive learning's recent success in mining additional supervision signals from raw data itself, in this work, we propose a non-sampling Co-Contrastive Learning (CCL) to enhance MBR. Technically, we first exploit the multi-behavior interaction graph to augment two views (interactive view and fold view) that capture both local and high-order structures simultaneously, and then two asymmetric graph encoders are performed over the two views, which recursively leverage the different structural information to generate ground-truth samples to collaboratively supervise each other by contrastive learning and finally high-level node embeddings are learned. Moreover, divergence constraint further improves co-contrastive learning performance. Extensive experiments on two real-world benchmark datasets demonstrate that CCL significantly outperforms the state-of-the-art methods of existing MBR.
PTL-AI Furnas Dataset: A Public Dataset for Fault Detection in Power Transmission Lines Using Aerial Images

We present a new images dataset called PTL-AI Furnas Dataset as a new benchmark for fault detection in power transmission lines. This dataset has 6,295 images, with resolution $1280\times 720$, extracted from the maintenance process of the energy transmission lines at Furnas company. It contains annotations of 17,808 components classified as baliser, bird nest, insulator, spacer and stockbridge. Furnas is a company that generates or transmits electricity to 51% of households in Brazil and more than 40% of the nation’s electricity passes through their grid enabling generating the dataset in different backgrounds and climatic conditions. We performed experiments using data augmentation techniques to train Faster R-CNN, Single-Shot Detects (SSD) and YoloV5 models. The benchmark result was obtained using the metrics of Mean Average Precision (mAP) and the Mean Average Recall (mAR) with values mAP=91.9% and mAR=89.7%. The PTL-AI Furnas Dataset is publicly available at https://github.com/freds0/PTL-AI_Furnas_Dataset.
Profiling Turkish cryptocurrency owners

With ownership estimates of up to 25%, Turkey is at the forefront of cryptocurrency adoption rendering it an interesting example to study the proclaimed use cases of cryptocurrencies. Using exploratory factor analysis based on a sample of 745 Turkish cryptocurrency owners, we identify three different owner groups and their underlying motives. The first group (users) is looking at cryptocurrency as an option for payments, thereby disregarding its speculative element, while the second group (investors) can be described as experienced investors holding cryptocurrency as part of their investment strategy. The third group (traders) consists of risk-tolerant traders. Further analyses show that groups not only differentiate by demographics, income and education but also by factors such as ideology, purchase intention and the use of domestic or foreign exchanges. The results contribute to the understanding of Turkish cryptocurrency owners, their intrinsic and extrinsic motivations and can be incorporated into the pending regulatory processes in the country. Furthermore, the findings suggest that cryptocurrencies have outgrown the use case of mere speculation, provoking future research on cryptocurrency usage with regards to the theories of innovation diffusion, lead users and lead markets.