OpenRaiser/Intern-Atlas huggingface dataset:
Hurricane Forecasting: A Novel Multimodal Machine Learning Framework
Venue: arXiv cs.LG 2022
Year: 2022

This paper describes a novel machine learning (ML) framework for tropical cyclone intensity and track forecasting, combining multiple ML techniques and utilizing diverse data sources. Our multimodal framework, called Hurricast, efficiently combines spatial-temporal data with statistical data by extracting features with deep-learning encoder-decoder architectures and predicting with gradient-boosted trees. We evaluate our models in the North Atlantic and Eastern Pacific basins on 2016-2019 for 24-hour lead time track and intensity forecasts and show they achieve comparable mean absolute error and skill to current operational forecast models while computing in seconds. Furthermore, the inclusion of Hurricast into an operational forecast consensus model could improve over the National Hurricane Center's official forecast, thus highlighting the complementary properties with existing approaches. In summary, our work demonstrates that utilizing machine learning techniques to combine different data sources can lead to new opportunities in tropical cyclone forecasting.
Learning to Segment Dynamic Objects using SLAM Outliers
Venue: arXiv cs.CV 2022
Year: 2022

We present a method to automatically learn to segment dynamic objects using SLAM outliers. It requires only one monocular sequence per dynamic object for training and consists in localizing dynamic objects using SLAM outliers, creating their masks, and using these masks to train a semantic segmentation network. We integrate the trained network in ORB-SLAM 2 and LDSO. At runtime we remove features on dynamic objects, making the SLAM unaffected by them. We also propose a new stereo dataset and new metrics to evaluate SLAM robustness. Our dataset includes consensus inversions, i.e., situations where the SLAM uses more features on dynamic objects that on the static background. Consensus inversions are challenging for SLAM as they may cause major SLAM failures. Our approach performs better than the State-of-the-Art on the TUM RGB-D dataset in monocular mode and on our dataset in both monocular and stereo modes.
Quantum statistical mechanics of encryption: reaching the speed limit of classical block ciphers
Venue: arXiv cs.CR 2022
Year: 2022

We cast encryption via classical block ciphers in terms of operator spreading in a dual space of Pauli strings, a formulation which allows us to characterize classical ciphers by using tools well known in the analysis of quantum many-body systems. We connect plaintext and ciphertext attacks to out-of-time order correlators (OTOCs) and quantify the quality of ciphers using measures of delocalization in string space such as participation ratios and corresponding entropies obtained from the wave function amplitudes in string space. The saturation of the string-space information entropy is accompanied by the vanishing of OTOCs. Together these signal irreversibility and chaos, which we take to be the defining properties of good classical ciphers. More precisely, we define a good cipher by requiring that the OTOCs vanish to exponential precision and that the string entropies saturate to the values associated with a random permutation, which are computed explicitly in the paper. We argue that these conditions can be satisfied by $n$-bit block ciphers implemented via random reversible circuits with ${\cal O}(n \log n)$ gates arranged on a tree structure, with layers of $n/3$ 3-bit gates, for which a "key" specifies uniquely the sequence of gates that comprise the circuit. We show that in order to reach this "speed limit" one must employ a three-stage circuit consisting of a stage implemented by layers of nonlinear gates that proliferate the number of strings, flanked by two other stages, each deploying layers of a special set of linear "inflationary" gates that accelerate the growth of small individual strings. A shallow, ${\cal O}(\log n)$-depth cipher of the type described here can be used in constructing a polynomial-overhead scheme for computation on encrypted data proposed in another publication as an alternative to Homomorphic Encryption.
RL-QN: A Reinforcement Learning Framework for Optimal Control of Queueing Systems
Venue: arXiv cs.PF 2022
Year: 2022

With the rapid advance of information technology, network systems have become increasingly complex and hence the underlying system dynamics are often unknown or difficult to characterize. Finding a good network control policy is of significant importance to achieve desirable network performance (e.g., high throughput or low delay). In this work, we consider using model-based reinforcement learning (RL) to learn the optimal control policy for queueing networks so that the average job delay (or equivalently the average queue backlog) is minimized. Traditional approaches in RL, however, cannot handle the unbounded state spaces of the network control problem. To overcome this difficulty, we propose a new algorithm, called Reinforcement Learning for Queueing Networks (RL-QN), which applies model-based RL methods over a finite subset of the state space, while applying a known stabilizing policy for the rest of the states. We establish that the average queue backlog under RL-QN with an appropriately constructed subset can be arbitrarily close to the optimal result. We evaluate RL-QN in dynamic server allocation, routing and switching problems. Simulation results show that RL-QN minimizes the average queue backlog effectively.
Functorial Manifold Learning
Venue: arXiv cs.LG 2022
Year: 2022

We adapt previous research on category theory and topological unsupervised learning to develop a functorial perspective on manifold learning, also known as nonlinear dimensionality reduction. We first characterize manifold learning algorithms as functors that map pseudometric spaces to optimization objectives and that factor through hierarchical clustering functors. We then use this characterization to prove refinement bounds on manifold learning loss functions and construct a hierarchy of manifold learning algorithms based on their equivariants. We express several popular manifold learning algorithms as functors at different levels of this hierarchy, including Metric Multidimensional Scaling, IsoMap, and UMAP. Next, we use interleaving distance to study the stability of a broad class of manifold learning algorithms. We present bounds on how closely the embeddings these algorithms produce from noisy data approximate the embeddings they would learn from noiseless data. Finally, we use our framework to derive a set of novel manifold learning algorithms, which we experimentally demonstrate are competitive with the state of the art.
Tight Bounds for Adversarially Robust Streams and Sliding Windows via Difference Estimators
Venue: arXiv cs.DS 2022
Year: 2022

In the adversarially robust streaming model, a stream of elements is presented to an algorithm and is allowed to depend on the output of the algorithm at earlier times during the stream. In the classic insertion-only model of data streams, Ben-Eliezer et. al. (PODS 2020, best paper award) show how to convert a non-robust algorithm into a robust one with a roughly $1/\varepsilon$ factor overhead. This was subsequently improved to a $1/\sqrt{\varepsilon}$ factor overhead by Hassidim et. al. (NeurIPS 2020, oral presentation), suppressing logarithmic factors. For general functions the latter is known to be best-possible, by a result of Kaplan et. al. (CRYPTO 2021). We show how to bypass this impossibility result by developing data stream algorithms for a large class of streaming problems, with no overhead in the approximation factor. Our class of streaming problems includes the most well-studied problems such as the $L_2$-heavy hitters problem, $F_p$-moment estimation, as well as empirical entropy estimation. We substantially improve upon all prior work on these problems, giving the first optimal dependence on the approximation factor. As in previous work, we obtain a general transformation that applies to any non-robust streaming algorithm and depends on the so-called twist number. However, the key technical innovation is that we apply the transformation to what we call a difference estimator for the streaming problem, rather than an estimator for the streaming problem itself. We then develop the first difference estimators for a wide range of problems. Our difference estimator methodology is not only applicable to the adversarially robust model, but to other streaming models where temporal properties of the data play a central role. (Abstract shortened to meet arXiv limit.)
Cluster-Specific Predictions with Multi-Task Gaussian Processes
Venue: arXiv cs.LG 2022
Year: 2022

A model involving Gaussian processes (GPs) is introduced to simultaneously handle multi-task learning, clustering, and prediction for multiple functional data. This procedure acts as a model-based clustering method for functional data as well as a learning step for subsequent predictions for new tasks. The model is instantiated as a mixture of multi-task GPs with common mean processes. A variational EM algorithm is derived for dealing with the optimisation of the hyper-parameters along with the hyper-posteriors' estimation of latent variables and processes. We establish explicit formulas for integrating the mean processes and the latent clustering variables within a predictive distribution, accounting for uncertainty on both aspects. This distribution is defined as a mixture of cluster-specific GP predictions, which enhances the performances when dealing with group-structured data. The model handles irregular grid of observations and offers different hypotheses on the covariance structure for sharing additional information across tasks. The performances on both clustering and prediction tasks are assessed through various simulated scenarios and real datasets. The overall algorithm, called MagmaClust, is publicly available as an R package.
Multi-Step Recurrent Q-Learning for Robotic Velcro Peeling
Venue: arXiv cs.RO 2022
Year: 2022

Learning object manipulation is a critical skill for robots to interact with their environment. Even though there has been significant progress in robotic manipulation of rigid objects, interacting with non-rigid objects remains challenging for robots. In this work, we introduce velcro peeling as a representative application for robotic manipulation of non-rigid objects in complex environments. We present a method of learning force-based manipulation from noisy and incomplete sensor inputs in partially observable environments by modeling long term dependencies between measurements with a multi-step deep recurrent network. We present experiments on a real robot to show the necessity of modeling these long term dependencies and validate our approach in simulation and robot experiments. Our results show that using tactile input enables the robot to overcome geometric uncertainties present in the environment with high fidelity in ~90% of all cases, outperforming the baselines by a large margin.
Learning Efficient GANs for Image Translation via Differentiable Masks and co-Attention Distillation
Venue: arXiv cs.CV 2022
Year: 2022

Generative Adversarial Networks (GANs) have been widely-used in image translation, but their high computation and storage costs impede the deployment on mobile devices. Prevalent methods for CNN compression cannot be directly applied to GANs due to the peculiarties of GAN tasks and the unstable adversarial training. To solve these, in this paper, we introduce a novel GAN compression method, termed DMAD, by proposing a Differentiable Mask and a co-Attention Distillation. The former searches for a light-weight generator architecture in a training-adaptive manner. To overcome channel inconsistency when pruning the residual connections, an adaptive cross-block group sparsity is further incorporated. The latter simultaneously distills informative attention maps from both the generator and discriminator of a pre-trained model to the searched generator, effectively stabilizing the adversarial training of our light-weight model. Experiments show that DMAD can reduce the Multiply Accumulate Operations (MACs) of CycleGAN by 13x and that of Pix2Pix by 4x while retaining a comparable performance against the full model. Our code can be available at https://github.com/SJLeo/DMAD.
Multi Receptive Field Network for Semantic Segmentation
Venue: arXiv cs.CV 2022
Year: 2022

Semantic segmentation is one of the key tasks in computer vision, which is to assign a category label to each pixel in an image. Despite significant progress achieved recently, most existing methods still suffer from two challenging issues: 1) the size of objects and stuff in an image can be very diverse, demanding for incorporating multi-scale features into the fully convolutional networks (FCNs); 2) the pixels close to or at the boundaries of object/stuff are hard to classify due to the intrinsic weakness of convolutional networks. To address the first issue, we propose a new Multi-Receptive Field Module (MRFM), explicitly taking multi-scale features into account. For the second issue, we design an edge-aware loss which is effective in distinguishing the boundaries of object/stuff. With these two designs, our Multi Receptive Field Network achieves new state-of-the-art results on two widely-used semantic segmentation benchmark datasets. Specifically, we achieve a mean IoU of 83.0 on the Cityscapes dataset and 88.4 mean IoU on the Pascal VOC2012 dataset.
Double-Prong ConvLSTM for Spatiotemporal Occupancy Prediction in Dynamic Environments
Venue: arXiv cs.RO 2022
Year: 2022

Predicting the future occupancy state of an environment is important to enable informed decisions for autonomous vehicles. Common challenges in occupancy prediction include vanishing dynamic objects and blurred predictions, especially for long prediction horizons. In this work, we propose a double-prong neural network architecture to predict the spatiotemporal evolution of the occupancy state. One prong is dedicated to predicting how the static environment will be observed by the moving ego vehicle. The other prong predicts how the dynamic objects in the environment will move. Experiments conducted on the real-world Waymo Open Dataset indicate that the fused output of the two prongs is capable of retaining dynamic objects and reducing blurriness in the predictions for longer time horizons than baseline models.
Practical Privacy Attacks on Vertical Federated Learning
Venue: arXiv cs.CR 2022
Year: 2022

Federated learning (FL) is a privacy-preserving learning paradigm that allows multiple parities to jointly train a powerful machine learning model without sharing their private data. According to the form of collaboration, FL can be further divided into horizontal federated learning (HFL) and vertical federated learning (VFL). In HFL, participants share the same feature space and collaborate on data samples, while in VFL, participants share the same sample IDs and collaborate on features. VFL has a broader scope of applications and is arguably more suitable for joint model training between large enterprises. In this paper, we focus on VFL and investigate potential privacy leakage in real-world VFL frameworks. We design and implement two practical privacy attacks: reverse multiplication attack for the logistic regression VFL protocol; and reverse sum attack for the XGBoost VFL protocol. We empirically show that the two attacks are (1) effective - the adversary can successfully steal the private training data, even when the intermediate outputs are encrypted to protect data privacy; (2) evasive - the attacks do not deviate from the protocol specification nor deteriorate the accuracy of the target model; and (3) easy - the adversary needs little prior knowledge about the data distribution of the target participant. We also show the leaked information is as effective as the raw training data in training an alternative classifier. We further discuss potential countermeasures and their challenges, which we hope can lead to several promising research directions.
A Definition and a Test for Human-Level Artificial Intelligence
Venue: arXiv cs.AI 2022
Year: 2022

Despite recent advances of AI research in many application-specific domains, we do not know how to build a human-level artificial intelligence (HLAI). We conjecture that learning from others' experience with the language is the essential characteristic that distinguishes human intelligence from the rest. Humans can update the action-value function with the verbal description as if they experience states, actions, and corresponding rewards sequences firsthand. In this paper, we present a classification of intelligence according to how individual agents learn and propose a definition and a test for HLAI. The main idea is that language acquisition without explicit rewards can be a sufficient test for HLAI.
FedEval: A Holistic Evaluation Framework for Federated Learning
Venue: arXiv cs.LG 2022
Year: 2022

Federated Learning (FL) has been widely accepted as the solution for privacy-preserving machine learning without collecting raw data. While new technologies proposed in the past few years do evolve the FL area, unfortunately, the evaluation results presented in these works fall short in integrity and are hardly comparable because of the inconsistent evaluation metrics and experimental settings. In this paper, we propose a holistic evaluation framework for FL called FedEval, and present a benchmarking study on seven state-of-the-art FL algorithms. Specifically, we first introduce the core evaluation taxonomy model, called FedEval-Core, which covers four essential evaluation aspects for FL: Privacy, Robustness, Effectiveness, and Efficiency, with various well-defined metrics and experimental settings. Based on the FedEval-Core, we further develop an FL evaluation platform with standardized evaluation settings and easy-to-use interfaces. We then provide an in-depth benchmarking study between the seven well-known FL algorithms, including FedSGD, FedAvg, FedProx, FedOpt, FedSTC, SecAgg, and HEAgg. We comprehensively analyze the advantages and disadvantages of these algorithms and further identify the suitable practical scenarios for different algorithms, which is rarely done by prior work. Lastly, we excavate a set of take-away insights and future research directions, which are very helpful for researchers in the FL area.
Data Driven Modeling of Interfacial Traction Separation Relations using a Thermodynamically Consistent Neural Network
Venue: arXiv cs.CE 2022
Year: 2022

For multilayer structures, interfacial failure is one of the most important elements related to device reliability. For cohesive zone modelling, traction-separation relations represent the adhesive interactions across interfaces. However, existing theoretical models do not currently capture traction-separation relations that have been extracted using direct methods, particularly under mixed-mode conditions. Given the complexity of the problem, models derived from the neural network approach are attractive. Although they can be trained to fit data along the loading paths taken in a particular set of mixed-mode fracture experiments, they may fail to obey physical laws for paths not covered by the training data sets. In this paper, a thermodynamically consistent neural network (TCNN) approach is established to model the constitutive behavior of interfaces when faced with sparse training data sets. Accordingly, three conditions are examined and implemented here: (i) thermodynamic consistency, (ii) maximum energy dissipation path control and (iii) J-integral conservation. These conditions are treated as constraints and are implemented as such in the loss function. The feasibility of this approach is demonstrated by comparing the modeling results with a range of physical constraints. Moreover, a Bayesian optimization algorithm is then adopted to optimize the weight factors associated with each of the constraints in order to overcome convergence issues that can arise when multiple constraints are present. The resultant numerical implementation of the ideas presented here produced well-behaved, mixed-mode traction separation surfaces that maintained the fidelity of the experimental data that was provided as input. The proposed approach heralds a new autonomous, point-to-point constitutive modeling concept for interface mechanics.
Improved rates for prediction and identification of partially observed linear dynamical systems
Venue: arXiv cs.LG 2022
Year: 2022

Identification of a linear time-invariant dynamical system from partial observations is a fundamental problem in control theory. Particularly challenging are systems exhibiting long-term memory. A natural question is how learn such systems with non-asymptotic statistical rates depending on the inherent dimensionality (order) $d$ of the system, rather than on the possibly much larger memory length. We propose an algorithm that given a single trajectory of length $T$ with gaussian observation noise, learns the system with a near-optimal rate of $\widetilde O\left(\sqrt\frac{d}{T}\right)$ in $\mathcal{H}_2$ error, with only logarithmic, rather than polynomial dependence on memory length. We also give bounds under process noise and improved bounds for learning a realization of the system. Our algorithm is based on multi-scale low-rank approximation: SVD applied to Hankel matrices of geometrically increasing sizes. Our analysis relies on careful application of concentration bounds on the Fourier domain -- we give sharper concentration bounds for sample covariance of correlated inputs and for $\mathcal H_\infty$ norm estimation, which may be of independent interest.
SIMF: Single-Instruction Multiple-Flush Mechanism for Processor Temporal Isolation
Venue: arXiv cs.CR 2022
Year: 2022

Microarchitectural timing attacks are a type of information leakage attack, which exploit the time-shared microarchitectural components, such as caches, translation look-aside buffers (TLBs), branch prediction unit (BPU), and speculative execution, in modern processors to leak critical information from a victim process or thread. To mitigate such attacks, the mechanism for flushing the on-core state is extensively used by operating-system-level solutions, since on-core state is too expensive to partition. In these systems, the flushing operations are implemented in software (using cache maintenance instructions), which severely limit the efficiency of timing attack protection. To bridge this gap, we propose specialized hardware support, a single-instruction multiple-flush (SIMF) mechanism to flush the core-level state, which consists of L1 caches, BPU, TLBs, and register file. We demonstrate SIMF by implementing it as an ISA extension, i.e., flushx instruction, in scalar in-order RISC-V processor. The resultant processor is prototyped on Xilinx ZCU102 FPGA and validated with state-of-art seL4 microkernel, Linux kernel in multi-core scenarios, and a cache timing attack. Our evaluation shows that SIMF significantly alleviates the overhead of flushing by more than a factor of two in execution time and reduces dynamic instruction count by orders-of-magnitude.