OpenRaiser/Intern-Atlas huggingface dataset:
Are Your Sensitive Attributes Private? Novel Model Inversion Attribute Inference Attacks on Classification Models
Venue: arXiv cs.CR 2022
Year: 2022

Increasing use of machine learning (ML) technologies in privacy-sensitive domains such as medical diagnoses, lifestyle predictions, and business decisions highlights the need to better understand if these ML technologies are introducing leakage of sensitive and proprietary training data. In this paper, we focus on model inversion attacks where the adversary knows non-sensitive attributes about records in the training data and aims to infer the value of a sensitive attribute unknown to the adversary, using only black-box access to the target classification model. We first devise a novel confidence score-based model inversion attribute inference attack that significantly outperforms the state-of-the-art. We then introduce a label-only model inversion attack that relies only on the model's predicted labels but still matches our confidence score-based attack in terms of attack effectiveness. We also extend our attacks to the scenario where some of the other (non-sensitive) attributes of a target record are unknown to the adversary. We evaluate our attacks on two types of machine learning models, decision tree and deep neural network, trained on three real datasets. Moreover, we empirically demonstrate the disparate vulnerability of model inversion attacks, i.e., specific groups in the training dataset (grouped by gender, race, etc.) could be more vulnerable to model inversion attacks.
Iterative Joint Parameters Estimation and Decoding in a Distributed Receiver for Satellite Applications and Relevant Cramer-Rao Bounds
Venue: arXiv cs.IT 2022
Year: 2022

This paper presents an algorithm for iterative joint channel parameter (carrier phase, Doppler shift and Doppler rate) estimation and decoding of transmission over channels affected by Doppler shift and Doppler rate using a distributed receiver. This algorithm is derived by applying the sum-product algorithm (SPA) to a factor graph representing the joint a posteriori distribution of the information symbols and channel parameters given the channel output. In this paper, we present two methods for dealing with intractable messages of the sum-product algorithm. In the first approach, we use particle filtering with sequential importance sampling (SIS) for the estimation of the unknown parameters. We also propose a method for fine-tuning of particles for improved convergence. In the second approach, we approximate our model with a random walk phase model, followed by a phase tracking algorithm and polynomial regression algorithm to estimate the unknown parameters. We derive the Weighted Bayesian Cramer-Rao Bounds (WBCRBs) for joint carrier phase, Doppler shift and Doppler rate estimation, which take into account the prior distribution of the estimation parameters and are accurate lower bounds for all considered Signal to Noise Ratio (SNR) values. Numerical results (of bit error rate (BER) and the mean-square error (MSE) of parameter estimation) suggest that phase tracking with the random walk model slightly outperforms particle filtering. However, particle filtering has a lower computational cost than the random walk model based method.
AttentionHTR: Handwritten Text Recognition Based on Attention Encoder-Decoder Networks
Venue: arXiv cs.CV 2022
Year: 2022

This work proposes an attention-based sequence-to-sequence model for handwritten word recognition and explores transfer learning for data-efficient training of HTR systems. To overcome training data scarcity, this work leverages models pre-trained on scene text images as a starting point towards tailoring the handwriting recognition models. ResNet feature extraction and bidirectional LSTM-based sequence modeling stages together form an encoder. The prediction stage consists of a decoder and a content-based attention mechanism. The effectiveness of the proposed end-to-end HTR system has been empirically evaluated on a novel multi-writer dataset Imgur5K and the IAM dataset. The experimental results evaluate the performance of the HTR framework, further supported by an in-depth analysis of the error cases. Source code and pre-trained models are available at https://github.com/dmitrijsk/AttentionHTR.
Cobol2Vec: Learning Representations of Cobol code
Venue: arXiv cs.PL 2022
Year: 2022

There has been a steadily growing interest in development of novel methods to learn a representation of a given input data and subsequently using them for several downstream tasks. The field of natural language processing has seen a significant improvement in different tasks by incorporating pre-trained embeddings into their pipelines. Recently, these methods have been applied to programming languages with a view to improve developer productivity. In this paper, we present an unsupervised learning approach to encode old mainframe languages into a fixed dimensional vector space. We use COBOL as our motivating example and create a corpus and demonstrate the efficacy of our approach in a code-retrieval task on our corpus.
A Two-phase Recommendation Framework for Consistent Java Method Names
Venue: arXiv cs.SE 2022
Year: 2022

In software engineering (SE) tasks, the naming approach is so important that it attracts many scholars from all over the world to study how to improve the quality of method names. To accurately recommend method names, we employ a novel framework to handle this problem. In our expeirments, nearly 8 million Java methods are collected from open source organizations as our evaluation dataset. In the first-phase recommendation, we introduce a fast and simple classifier based on the fast text neural network for reccomending potential method category. In the second-phase recomendation, we employ both two Long Short Term Memory Networks to reccomend consitent method names from each classification. Evaluation results prove that the proposed approach significantly outperforms state-of-the-art approach.
Synthetic Books
Venue: arXiv cs.CL 2022
Year: 2022

The article explores new ways of written language aided by AI technologies, like GPT-2 and GPT-3. The question that is stated in the paper is not about whether these novel technologies will eventually replace authored books, but how to relate to and contextualize such publications and what kind of new tools, processes, and ideas are behind them. For that purpose, a new concept of synthetic books is introduced in the article. It stands for the publications created by deploying AI technology, more precisely autoregressive language models that are able to generate human-like text. Supported by the case studies, the value and reasoning of the synthetic books are discussed. The paper emphasizes that artistic quality is an issue when it comes to AI-generated content. The article introduces projects that demonstrate an interactive input by an artist and/or audience combined with the deep-learning-based language models. In the end, the paper focuses on understanding the neural aesthetics of written language in the art context.
Communication-Efficient Stochastic Zeroth-Order Optimization for Federated Learning
Venue: arXiv cs.LG 2022
Year: 2022

Federated learning (FL), as an emerging edge artificial intelligence paradigm, enables many edge devices to collaboratively train a global model without sharing their private data. To enhance the training efficiency of FL, various algorithms have been proposed, ranging from first-order to second-order methods. However, these algorithms cannot be applied in scenarios where the gradient information is not available, e.g., federated black-box attack and federated hyperparameter tuning. To address this issue, in this paper we propose a derivative-free federated zeroth-order optimization (FedZO) algorithm featured by performing multiple local updates based on stochastic gradient estimators in each communication round and enabling partial device participation. Under non-convex settings, we derive the convergence performance of the FedZO algorithm on non-independent and identically distributed data and characterize the impact of the numbers of local iterates and participating edge devices on the convergence. To enable communication-efficient FedZO over wireless networks, we further propose an over-the-air computation (AirComp) assisted FedZO algorithm. With an appropriate transceiver design, we show that the convergence of AirComp-assisted FedZO can still be preserved under certain signal-to-noise ratio conditions. Simulation results demonstrate the effectiveness of the FedZO algorithm and validate the theoretical observations.
Dynamic Bandwidth Allocation and Edge Caching Optimization for Nonlinear Content Delivery through Flexible Multibeam Satellites
Venue: arXiv cs.IT 2022
Year: 2022

The next generation multibeam satellites open up a new way to design satellite communication channels with the full flexibility in bandwidth, transmit power and beam coverage management. In this paper, we exploit the flexible multibeam satellite capabilities and the geographical distribution of users to improve the performance of satellite-assisted edge caching systems. Our aim is to jointly optimize the bandwidth allocation in multibeam and caching decisions at the edge nodes to address two important problems: i) cache feeding time minimization and ii) cache hits maximization. To tackle the non-convexity of the joint optimization problem, we transform the original problem into a difference-of-convex (DC) form, which is then solved by the proposed iterative algorithm whose convergence to at least a local optimum is theoretically guaranteed. Furthermore, the effectiveness of the proposed design is evaluated under the realistic beams coverage of the satellite SES-14 and Movielens data set. Numerical results show that our proposed joint design can reduce the caching feeding time by 50\% and increase the cache hit ratio (CHR) by 10\% to 20\% compared to existing solutions. Furthermore, we examine the impact of multispot beam and multicarrier wide-beam on the joint design and discuss potential research directions.
Learning Geometric Constraints in Task and Motion Planning
Venue: arXiv cs.RO 2022
Year: 2022

Searching for bindings of geometric parameters in task and motion planning (TAMP) is a finite-horizon stochastic planning problem with high-dimensional decision spaces. A robot manipulator can only move in a subspace of its whole range that is subjected to many geometric constraints. A TAMP solver usually takes many explorations before finding a feasible binding set for each task. It is favorable to learn those constraints once and then transfer them over different tasks within the same workspace. We address this problem by representing constraint knowledge with transferable primitives and using Bayesian optimization (BO) based on these primitives to guide binding search in further tasks. Via semantic and geometric backtracking in TAMP, we construct constraint primitives to encode the geometric constraints respectively in a reusable form. Then we devise a BO approach to efficiently utilize the accumulated constraints for guiding node expansion of an MCTS-based binding planner. We further compose a transfer mechanism to enable free knowledge flow between TAMP tasks. Results indicate that our approach reduces the expensive exploration calls in binding search by 43.60to 71.69 when compared to the baseline unguided planner.
SEN12MS-CR-TS: A Remote Sensing Data Set for Multi-modal Multi-temporal Cloud Removal
Venue: arXiv cs.CV 2022
Year: 2022

About half of all optical observations collected via spaceborne satellites are affected by haze or clouds. Consequently, cloud coverage affects the remote sensing practitioner's capabilities of a continuous and seamless monitoring of our planet. This work addresses the challenge of optical satellite image reconstruction and cloud removal by proposing a novel multi-modal and multi-temporal data set called SEN12MS-CR-TS. We propose two models highlighting the benefits and use cases of SEN12MS-CR-TS: First, a multi-modal multi-temporal 3D-Convolution Neural Network that predicts a cloud-free image from a sequence of cloudy optical and radar images. Second, a sequence-to-sequence translation model that predicts a cloud-free time series from a cloud-covered time series. Both approaches are evaluated experimentally, with their respective models trained and tested on SEN12MS-CR-TS. The conducted experiments highlight the contribution of our data set to the remote sensing community as well as the benefits of multi-modal and multi-temporal information to reconstruct noisy information. Our data set is available at https://patrickTUM.github.io/cloud_removal
Leveraging Data and Analytics for Digital Business Transformation through DataOps: An Information Processing Perspective
Venue: arXiv cs.CY 2022
Year: 2022

Digital business transformation has become increasingly important for organizations. Since transforming business digitally is an ongoing process, it requires an integrated and disciplined approach. Data Operations (DataOps), emerging in practice, can provide organizations with such an approach to leverage data and analytics for digital business transformation. This paper proposes a framework that integrates digital business transformation, data analytics, and DataOps through the lens of information processing theory (IPT). The details of this framework explain how organizations can employ DataOps as an integrated and disciplined approach to understand their analytical information needs and develop the analytical information processing capability required for digital business transformation. DataOps-enabled digital business transformation, in turn, improves organizational performance by improving operational efficiency and creating new business models. This research extends current knowledge on digital transformation by bringing in DataOps and analytics through IPT and thereby provide organizations with a novel approach for their digital business transformations.
Paired Image to Image Translation for Strikethrough Removal From Handwritten Words
Venue: arXiv cs.CV 2022
Year: 2022

Transcribing struck-through, handwritten words, for example for the purpose of genetic criticism, can pose a challenge to both humans and machines, due to the obstructive properties of the superimposed strokes. This paper investigates the use of paired image to image translation approaches to remove strikethrough strokes from handwritten words. Four different neural network architectures are examined, ranging from a few simple convolutional layers to deeper ones, employing Dense blocks. Experimental results, obtained from one synthetic and one genuine paired strikethrough dataset, confirm that the proposed paired models outperform the CycleGAN-based state of the art, while using less than a sixth of the trainable parameters.
Towards Collaborative Question Answering: A Preliminary Study
Venue: arXiv cs.AI 2022
Year: 2022

Knowledge and expertise in the real-world can be disjointedly owned. To solve a complex question, collaboration among experts is often called for. In this paper, we propose CollabQA, a novel QA task in which several expert agents coordinated by a moderator work together to answer questions that cannot be answered with any single agent alone. We make a synthetic dataset of a large knowledge graph that can be distributed to experts. We define the process to form a complex question from ground truth reasoning path, neural network agent models that can learn to solve the task, and evaluation metrics to check the performance. We show that the problem can be challenging without introducing prior of the collaboration structure, unless experts are perfect and uniform. Based on this experience, we elaborate extensions needed to approach collaboration tasks in real-world settings.
Machine Learning Algorithms for Prediction of Penetration Depth and Geometrical Analysis of Weld in Friction Stir Spot Welding Process
Venue: arXiv cs.LG 2022
Year: 2022

Nowadays, manufacturing sectors harness the power of machine learning and data science algorithms to make predictions for the optimization of mechanical and microstructure properties of fabricated mechanical components. The application of these algorithms reduces the experimental cost beside leads to reduce the time of experiments. The present research work is based on the prediction of penetration depth using Supervised Machine Learning algorithms such as Support Vector Machines (SVM), Random Forest Algorithm, and Robust Regression algorithm. A Friction Stir Spot Welding (FSSW) was used to join two elements of AA1230 aluminum alloys. The dataset consists of three input parameters: Rotational Speed (rpm), Dwelling Time (seconds), and Axial Load (KN), on which the machine learning models were trained and tested. It observed that the Robust Regression machine learning algorithm outperformed the rest of the algorithms by resulting in the coefficient of determination of 0.96. The research work also highlights the application of image processing techniques to find the geometrical features of the weld formation.
In Defence of Visual Analytics Systems: Replies to Critics
Venue: arXiv cs.HC 2022
Year: 2022

The last decade has witnessed many visual analytics (VA) systems that make successful applications to wide-ranging domains like urban analytics and explainable AI. However, their research rigor and contributions have been extensively challenged within the visualization community. We come in defence of VA systems by contributing two interview studies for gathering critics and responses to those criticisms. First, we interview 24 researchers to collect criticisms the review comments on their VA work. Through an iterative coding and refinement process, the interview feedback is summarized into a list of 36 common criticisms. Second, we interview 17 researchers to validate our list and collect their responses, thereby discussing implications for defending and improving the scientific values and rigor of VA systems. We highlight that the presented knowledge is deep, extensive, but also imperfect, provocative, and controversial, and thus recommend reading with an inclusive and critical eye. We hope our work can provide thoughts and foundations for conducting VA research and spark discussions to promote the research field forward more rigorously and vibrantly.
Neural Architecture Searching for Facial Attributes-based Depression Recognition
Venue: arXiv cs.CV 2022
Year: 2022

Recent studies show that depression can be partially reflected from human facial attributes. Since facial attributes have various data structure and carry different information, existing approaches fail to specifically consider the optimal way to extract depression-related features from each of them, as well as investigates the best fusion strategy. In this paper, we propose to extend Neural Architecture Search (NAS) technique for designing an optimal model for multiple facial attributes-based depression recognition, which can be efficiently and robustly implemented in a small dataset. Our approach first conducts a warmer up step to the feature extractor of each facial attribute, aiming to largely reduce the search space and providing customized architecture, where each feature extractor can be either a Convolution Neural Networks (CNN) or Graph Neural Networks (GNN). Then, we conduct an end-to-end architecture search for all feature extractors and the fusion network, allowing the complementary depression cues to be optimally combined with less redundancy. The experimental results on AVEC 2016 dataset show that the model explored by our approach achieves breakthrough performance with 27\% and 30\% RMSE and MAE improvements over the existing state-of-the-art. In light of these findings, this paper provides solid evidences and a strong baseline for applying NAS to time-series data-based mental health analysis.
Analytic Mutual Information in Bayesian Neural Networks
Venue: arXiv cs.IT 2022
Year: 2022

Bayesian neural networks have successfully designed and optimized a robust neural network model in many application problems, including uncertainty quantification. However, with its recent success, information-theoretic understanding about the Bayesian neural network is still at an early stage. Mutual information is an example of an uncertainty measure in a Bayesian neural network to quantify epistemic uncertainty. Still, no analytic formula is known to describe it, one of the fundamental information measures to understand the Bayesian deep learning framework. In this paper, we derive the analytical formula of the mutual information between model parameters and the predictive output by leveraging the notion of the point process entropy. Then, as an application, we discuss the parameter estimation of the Dirichlet distribution and show its practical application in the active learning uncertainty measures by demonstrating that our analytical formula can improve the performance of active learning further in practice.
Iterative Activation-based Structured Pruning
Venue: arXiv cs.LG 2022
Year: 2022

Deploying complex deep learning models on edge devices is challenging because they have substantial compute and memory resource requirements, whereas edge devices' resource budget is limited. To solve this problem, extensive pruning techniques have been proposed for compressing networks. Recent advances based on the Lottery Ticket Hypothesis (LTH) show that iterative model pruning tends to produce smaller and more accurate models. However, LTH research focuses on unstructured pruning, which is hardware-inefficient and difficult to accelerate on hardware platforms. In this paper, we investigate iterative pruning in the context of structured pruning because structurally pruned models map well on commodity hardware. We find that directly applying a structured weight-based pruning technique iteratively, called iterative L1-norm based pruning (ILP), does not produce accurate pruned models. To solve this problem, we propose two activation-based pruning methods, Iterative Activation-based Pruning (IAP) and Adaptive Iterative Activation-based Pruning (AIAP). We observe that, with only 1% accuracy loss, IAP and AIAP achieve 7.75X and 15.88$X compression on LeNet-5, and 1.25X and 1.71X compression on ResNet-50, whereas ILP achieves 4.77X and 1.13X, respectively.
Ordinal-Quadruplet: Retrieval of Missing Classes in Ordinal Time Series
Venue: arXiv cs.LG 2022
Year: 2022

In this paper, we propose an ordered time series classification framework that is robust against missing classes in the training data, i.e., during testing we can prescribe classes that are missing during training. This framework relies on two main components: (1) our newly proposed ordinal-quadruplet loss, which forces the model to learn latent representation while preserving the ordinal relation among labels, (2) testing procedure, which utilizes the property of latent representation (order preservation). We conduct experiments based on real world multivariate time series data and show the significant improvement in the prediction of missing labels even with 40% of the classes are missing from training. Compared with the well-known triplet loss optimization augmented with interpolation for missing information, in some cases, we nearly double the accuracy.