OpenRaiser/Intern-Atlas huggingface dataset:
Toward Force Estimation in Robot-Assisted Surgery using Deep Learning with Vision and Robot State
Venue: arXiv cs.RO 2022
Year: 2022

Knowledge of interaction forces during teleoperated robot-assisted surgery could be used to enable force feedback to human operators and evaluate tissue handling skill. However, direct force sensing at the end-effector is challenging because it requires biocompatible, sterilizable, and cost-effective sensors. Vision-based deep learning using convolutional neural networks is a promising approach for providing useful force estimates, though questions remain about generalization to new scenarios and real-time inference. We present a force estimation neural network that uses RGB images and robot state as inputs. Using a self-collected dataset, we compared the network to variants that included only a single input type, and evaluated how they generalized to new viewpoints, workspace positions, materials, and tools. We found that vision-based networks were sensitive to shifts in viewpoints, while state-only networks were robust to changes in workspace. The network with both state and vision inputs had the highest accuracy for an unseen tool, and was moderately robust to changes in viewpoints. Through feature removal studies, we found that using only position features produced better accuracy than using only force features as input. The network with both state and vision inputs outperformed a physics-based baseline model in accuracy. It showed comparable accuracy but faster computation times than a baseline recurrent neural network, making it better suited for real-time applications.
Joint Optimization of Privacy and Cost of in-App Mobile User Profiling and Targeted Ads
Venue: arXiv cs.CR 2022
Year: 2022

Online mobile advertising ecosystems provide advertising and analytics services that collect, aggregate, process, and trade a rich amount of consumers' personal data and carry out interest-based ad targeting, which raised serious privacy risks and growing trends of users feeling uncomfortable while using the internet services. In this paper, we address users' privacy concerns by developing an optimal dynamic optimisation cost-effective framework for preserving user privacy for profiling, ads-based inferencing, temporal apps usage behavioral patterns, and interest-based ad targeting. A major challenge in solving this dynamic model is the lack of knowledge of time-varying updates during the profiling process. We formulate a mixed-integer optimisation problem and develop an equivalent problem to show that the proposed algorithm does not require knowledge of time-varying updates in user behavior. Following, we develop an online control algorithm to solve the equivalent problem and overcome the difficulty of solving nonlinear programming by decomposing it into various cases and to achieve a trade-off between user privacy, cost, and targeted ads. We carry out extensive experimentations and demonstrate the proposed framework's applicability by implementing its critical components using POC (Proof Of Concept) `System App'. We compare the proposed framework with other privacy-protecting approaches and investigate whether it achieves better privacy and functionality for various performance parameters.
FedSL: Federated Split Learning on Distributed Sequential Data in Recurrent Neural Networks
Venue: arXiv cs.LG 2022
Year: 2022

Federated Learning (FL) and Split Learning (SL) are privacy-preserving Machine-Learning (ML) techniques that enable training ML models over data distributed among clients without requiring direct access to their raw data. Existing FL and SL approaches work on horizontally or vertically partitioned data and cannot handle sequentially partitioned data where segments of multiple-segment sequential data are distributed across clients. In this paper, we propose a novel federated split learning framework, FedSL, to train models on distributed sequential data. The most common ML models to train on sequential data are Recurrent Neural Networks (RNNs). Since the proposed framework is privacy-preserving, segments of multiple-segment sequential data cannot be shared between clients or between clients and server. To circumvent this limitation, we propose a novel SL approach tailored for RNNs. A RNN is split into sub-networks, and each sub-network is trained on one client containing single segments of multiple-segment training sequences. During local training, the sub-networks on different clients communicate with each other to capture latent dependencies between consecutive segments of multiple-segment sequential data on different clients, but without sharing raw data or complete model parameters. After training local sub-networks with local sequential data segments, all clients send their sub-networks to a federated server where sub-networks are aggregated to generate a global model. The experimental results on simulated and real-world datasets demonstrate that the proposed method successfully trains models on distributed sequential data, while preserving privacy, and outperforms previous FL and centralized learning approaches in terms of achieving higher accuracy in fewer communication rounds.
Single-Node Attacks for Fooling Graph Neural Networks
Venue: arXiv cs.LG 2022
Year: 2022

Graph neural networks (GNNs) have shown broad applicability in a variety of domains. These domains, e.g., social networks and product recommendations, are fertile ground for malicious users and behavior. In this paper, we show that GNNs are vulnerable to the extremely limited (and thus quite realistic) scenarios of a single-node adversarial attack, where the perturbed node cannot be chosen by the attacker. That is, an attacker can force the GNN to classify any target node to a chosen label, by only slightly perturbing the features or the neighbor list of another single arbitrary node in the graph, even when not being able to select that specific attacker node. When the adversary is allowed to select the attacker node, these attacks are even more effective. We demonstrate empirically that our attack is effective across various common GNN types (e.g., GCN, GraphSAGE, GAT, GIN) and robustly optimized GNNs (e.g., Robust GCN, SM GCN, GAL, LAT-GCN), outperforming previous attacks across different real-world datasets both in a targeted and non-targeted attacks. Our code is available at https://github.com/benfinkelshtein/SINGLE .
On Counting (Quantum-)Graph Homomorphisms in Finite Fields of Prime Order
Venue: arXiv cs.CC 2022
Year: 2022

We study the problem of counting the number of homomorphisms from an input graph $G$ to a fixed (quantum) graph $\bar{H}$ in any finite field of prime order $\mathbb{Z}_p$. The subproblem with graph $H$ was introduced by Faben and Jerrum [ToC'15] and its complexity is subject to a growing series of research articles, e.g. the work of Focke, Goldberg, Roth, and Zivný [SIDMA'21] and the work of Bulatov and Kazeminia [STOC'22], subsequent to this article's conference version. Our contribution is threefold. First, we introduce the study of quantum graphs to the study of modular counting homomorphisms. We show that the complexity for a quantum graph $\bar{H}$ collapses to the complexity criteria found at dimension 1: graphs. Second, in order to prove cases of intractability we establish a further reduction to the study of bipartite graphs. Lastly, we establish a dichotomy for all bipartite ($K_{3,3}\backslash\{e\}$, ${domino}$)-free graphs by a thorough structural study incorporating both local and global arguments. This result subsumes all results on bipartite graphs known for all prime moduli and extends them significantly. Even for the subproblem with $p$ equal to $2$, this establishes new results.
Two-stage Training of Graph Neural Networks for Graph Classification
Venue: arXiv cs.LG 2022
Year: 2022

Graph neural networks (GNNs) have received massive attention in the field of machine learning on graphs. Inspired by the success of neural networks, a line of research has been conducted to train GNNs to deal with various tasks, such as node classification, graph classification, and link prediction. In this work, our task of interest is graph classification. Several GNN models have been proposed and shown great accuracy in this task. However, the question is whether usual training methods fully realize the capacity of the GNN models. In this work, we propose a two-stage training framework based on triplet loss. In the first stage, GNN is trained to map each graph to a Euclidean-space vector so that graphs of the same class are close while those of different classes are mapped far apart. Once graphs are well-separated based on labels, a classifier is trained to distinguish between different classes. This method is generic in the sense that it is compatible with any GNN model. By adapting five GNN models to our method, we demonstrate the consistent improvement in accuracy and utilization of each GNN's allocated capacity over the original training method of each model up to 5.4\% points in 12 datasets.
ATCN: Resource-Efficient Processing of Time Series on Edge
Venue: arXiv cs.LG 2022
Year: 2022

This paper presents a scalable deep learning model called Agile Temporal Convolutional Network (ATCN) for high-accurate fast classification and time series prediction in resource-constrained embedded systems. ATCN is a family of compact networks with formalized hyperparameters that enable application-specific adjustments to be made to the model architecture. It is primarily designed for embedded edge devices with very limited performance and memory, such as wearable biomedical devices and real-time reliability monitoring systems. ATCN makes fundamental improvements over the mainstream temporal convolutional neural networks, including residual connections to increase the network depth and accuracy, and the incorporation of separable depth-wise convolution to reduce the computational complexity of the model. As part of the present work, two ATCN families, namely T0, and T1 are also presented and evaluated on different ranges of embedded processors - Cortex-M7 and Cortex-A57 processor. An evaluation of the ATCN models against the best-in-class InceptionTime and MiniRocket shows that ATCN almost maintains accuracy while improving the execution time on a broad range of embedded and cyber-physical applications with demand for real-time processing on the embedded edge. At the same time, in contrast to existing solutions, ATCN is the first time-series classifier based on deep learning that can be run bare-metal on embedded microcontrollers (Cortex-M7) with limited computational performance and memory capacity while delivering state-of-the-art accuracy.
Learning Models for Actionable Recourse
Venue: arXiv cs.LG 2022
Year: 2022

As machine learning models are increasingly deployed in high-stakes domains such as legal and financial decision-making, there has been growing interest in post-hoc methods for generating counterfactual explanations. Such explanations provide individuals adversely impacted by predicted outcomes (e.g., an applicant denied a loan) with recourse -- i.e., a description of how they can change their features to obtain a positive outcome. We propose a novel algorithm that leverages adversarial training and PAC confidence sets to learn models that theoretically guarantee recourse to affected individuals with high probability without sacrificing accuracy. We demonstrate the efficacy of our approach via extensive experiments on real data.
Value Function Approximations via Kernel Embeddings for No-Regret Reinforcement Learning
Venue: arXiv cs.LG 2022
Year: 2022

We consider the regret minimization problem in reinforcement learning (RL) in the episodic setting. In many real-world RL environments, the state and action spaces are continuous or very large. Existing approaches establish regret guarantees by either a low-dimensional representation of the stochastic transition model or an approximation of the $Q$-functions. However, the understanding of function approximation schemes for state-value functions largely remains missing. In this paper, we propose an online model-based RL algorithm, namely the CME-RL, that learns representations of transition distributions as embeddings in a reproducing kernel Hilbert space while carefully balancing the exploitation-exploration tradeoff. We demonstrate the efficiency of our algorithm by proving a frequentist (worst-case) regret bound that is of order $\tilde{O}\big(Hγ_N\sqrt{N}\big)$\footnote{ $\tilde{O}(\cdot)$ hides only absolute constant and poly-logarithmic factors.}, where $H$ is the episode length, $N$ is the total number of time steps and $γ_N$ is an information theoretic quantity relating the effective dimension of the state-action feature space. Our method bypasses the need for estimating transition probabilities and applies to any domain on which kernels can be defined. It also brings new insights into the general theory of kernel methods for approximate inference and RL regret minimization.
The Problem with XSD Binary Floating Point Datatypes in RDF
Venue: arXiv cs.DB 2022
Year: 2022

The XSD binary floating point datatypes are regularly used for precise numeric values in RDF. However, the use of these datatypes for knowledge representation can systematically impair the quality of data and, compared to the XSD decimal datatype, increases the probability of data processing producing false results. We argue why in most cases the XSD decimal datatype is better suited to represent numeric values in RDF. A survey of the actual usage of datatypes on the relevant subset of the December 2020 Web Data Commons dataset, containing 19453060341 literals from real web data, substantiates the practical relevancy of the described problem: 29 %-68 % of binary floating point values are distorted due to the datatype.
Dynamic Occupancy Grid Mapping with Recurrent Neural Networks
Venue: arXiv cs.RO 2022
Year: 2022

Modeling and understanding the environment is an essential task for autonomous driving. In addition to the detection of objects, in complex traffic scenarios the motion of other road participants is of special interest. Therefore, we propose to use a recurrent neural network to predict a dynamic occupancy grid map, which divides the vehicle surrounding in cells, each containing the occupancy probability and a velocity estimate. During training, our network is fed with sequences of measurement grid maps, which encode the lidar measurements of a single time step. Due to the combination of convolutional and recurrent layers, our approach is capable to use spatial and temporal information for the robust detection of static and dynamic environment. In order to apply our approach with measurements from a moving ego-vehicle, we propose a method for ego-motion compensation that is applicable in neural network architectures with recurrent layers working on different resolutions. In our evaluations, we compare our approach with a state-of-the-art particle-based algorithm on a large publicly available dataset to demonstrate the improved accuracy of velocity estimates and the more robust separation of the environment in static and dynamic area. Additionally, we show that our proposed method for ego-motion compensation leads to comparable results in scenarios with stationary and with moving ego-vehicle.
Stochastic Client Selection for Federated Learning with Volatile Clients
Venue: arXiv cs.LG 2022
Year: 2022

Federated Learning (FL), arising as a privacy-preserving machine learning paradigm, has received notable attention from the public. In each round of synchronous FL training, only a fraction of available clients are chosen to participate, and the selection decision might have a significant effect on the training efficiency, as well as the final model performance. In this paper, we investigate the client selection problem under a volatile context, in which the local training of heterogeneous clients is likely to fail due to various kinds of reasons and in different levels of frequency. {\color{black}Intuitively, too much training failure might potentially reduce the training efficiency, while too much selection on clients with greater stability might introduce bias, thereby resulting in degradation of the training effectiveness. To tackle this tradeoff, we in this paper formulate the client selection problem under joint consideration of effective participation and fairness.} Further, we propose E3CS, a stochastic client selection scheme to solve the problem, and we corroborate its effectiveness by conducting real data-based experiments. According to our experimental results, the proposed selection scheme is able to achieve up to 2x faster convergence to a fixed model accuracy while maintaining the same level of final model accuracy, compared with the state-of-the-art selection schemes.
Experimenting Touchless Gestural Interaction for a University Public Web-based Display
Venue: arXiv cs.HC 2022
Year: 2022

Interest in and development of touchless gestural interfaces has recently exploded, fueled by the diffusion of both commercial midair gesture platforms and public interactive displays. This paper focuses on an application based on Microsoft Kinect that allows students to browse a university website, hosted on a public display, through simple gestures. We present two empirical evaluations where we evaluated how users react to this new way of interaction. In addition to confirming the current lack of standards, our results provide some inspiration for the design of touchless interaction.
Fast Conformal Parameterization of Disks and Sphere Sectors
Venue: arXiv cs.CG 2022
Year: 2022

We prove a novel method for the embedding of a 3-fold rotationally symmetric sphere-type mesh onto a subset of the plane with 3-fold rotational symmetry. The embedding is free-boundary with the only additional constraint on the image set is that its translations tile the plane, in turn this forces the angles at the embedding of the branch points in the construction. These parameterizations are optimal with respect to the Dirichlet energy functional defined on simplicial complexes. Since the parameterization is over a fixed area domain, it is conformal (i.e. a minimizer of the LSCM energy). The embedding is done by a novel construction of a torus from 63 copies of the original sphere. As a foundation for this result we first prove the optimality of the embedding of disk-type meshes onto special types of triangles in the plane, and rectangles. The embedding of the 3-fold symmetric torus is full rank and so cannot be reduced by simpler constructions. 3-fold symmetric surfaces appear in nature, for example the surface of the 3-fold symmetric proteins PIEZO1 and PIEZO2 which are an important target of current studies.
Challenges in Deploying Machine Learning: a Survey of Case Studies
Venue: arXiv cs.LG 2022
Year: 2022

In recent years, machine learning has transitioned from a field of academic research interest to a field capable of solving real-world business problems. However, the deployment of machine learning models in production systems can present a number of issues and concerns. This survey reviews published reports of deploying machine learning solutions in a variety of use cases, industries and applications and extracts practical considerations corresponding to stages of the machine learning deployment workflow. By mapping found challenges to the steps of the machine learning deployment workflow we show that practitioners face issues at each stage of the deployment process. The goal of this paper is to lay out a research agenda to explore approaches addressing these challenges.
Continuous Blackjack: Equilibrium, Deviation and Adaptive Strategy
Venue: arXiv cs.GT 2022
Year: 2022

We introduce a variant of the classic poker game blackjack -- the continuous blackjack. We study the Nash Equilibrium as well as the case where players deviate from it. We then pivot to the study of a large class of adaptive strategies and obtain a model-free strategy. Finally, we apply reinforcement learning techniques to the game and address several associated engineering challenges.
Rethinking Weight Decay For Efficient Neural Network Pruning
Venue: arXiv cs.NE 2022
Year: 2022

Introduced in the late 1980s for generalization purposes, pruning has now become a staple for compressing deep neural networks. Despite many innovations in recent decades, pruning approaches still face core issues that hinder their performance or scalability. Drawing inspiration from early work in the field, and especially the use of weight decay to achieve sparsity, we introduce Selective Weight Decay (SWD), which carries out efficient, continuous pruning throughout training. Our approach, theoretically grounded on Lagrangian smoothing, is versatile and can be applied to multiple tasks, networks, and pruning structures. We show that SWD compares favorably to state-of-the-art approaches, in terms of performance-to-parameters ratio, on the CIFAR-10, Cora, and ImageNet ILSVRC2012 datasets.
Ranking Neural Checkpoints
Venue: arXiv cs.LG 2022
Year: 2022

This paper is concerned with ranking many pre-trained deep neural networks (DNNs), called checkpoints, for the transfer learning to a downstream task. Thanks to the broad use of DNNs, we may easily collect hundreds of checkpoints from various sources. Which of them transfers the best to our downstream task of interest? Striving to answer this question thoroughly, we establish a neural checkpoint ranking benchmark (NeuCRaB) and study some intuitive ranking measures. These measures are generic, applying to the checkpoints of different output types without knowing how the checkpoints are pre-trained on which dataset. They also incur low computation cost, making them practically meaningful. Our results suggest that the linear separability of the features extracted by the checkpoints is a strong indicator of transferability. We also arrive at a new ranking measure, NLEEP, which gives rise to the best performance in the experiments.
Stochastic sparse adversarial attacks
Venue: arXiv cs.LG 2022
Year: 2022

This paper introduces stochastic sparse adversarial attacks (SSAA), standing as simple, fast and purely noise-based targeted and untargeted attacks of neural network classifiers (NNC). SSAA offer new examples of sparse (or $L_0$) attacks for which only few methods have been proposed previously. These attacks are devised by exploiting a small-time expansion idea widely used for Markov processes. Experiments on small and large datasets (CIFAR-10 and ImageNet) illustrate several advantages of SSAA in comparison with the-state-of-the-art methods. For instance, in the untargeted case, our method called Voting Folded Gaussian Attack (VFGA) scales efficiently to ImageNet and achieves a significantly lower $L_0$ score than SparseFool (up to $\frac{2}{5}$) while being faster. Moreover, VFGA achieves better $L_0$ scores on ImageNet than Sparse-RS when both attacks are fully successful on a large number of samples.