OpenRaiser/Intern-Atlas huggingface dataset:
Stability of the Gaussian Stationary Point in the Han-Kobayashi Region for Z-Interference Channels
Venue: arXiv cs.IT 2022
Year: 2022

The Gaussian stationary point in an inequality motivated by the Z-interference channel was recently conjectured by Costa, Nair, Ng, and Wang to be the global optimizer, which, if true, would imply the optimality of the Han-Kobayashi region for the Gaussian Z-interference channel. This conjecture was known to be true for some parameter regimes, but the validity for all parameters, although suggested by Gaussian tensorization, was previously open. In this paper we construct several counterexamples showing that this conjecture may fail in certain regimes: A simple construction without Hermite polynomial perturbation is proposed, where distributions far from Gaussian are analytically shown to be better than the Gaussian stationary point. As alternatives, we consider perturbation along geodesics under either the $L^2$ or Wasserstein-2 metric, showing that the Gaussian stationary point is unstable in a certain regime. Similarity to stability of the Levy-Cramer theorem is discussed. The stability phase transition point admits a simple characterization in terms of the maximum eigenvalue of the Gaussian maximizer. Similar to the Holley-Stroock principle, we can show that in the stable regime the Gaussian stationary point is optimal in a neighborhood under the $L^{\infty}$-norm with respect to the Gaussian measure. For protocols with constant power control, our counterexamples imply Gaussian suboptimality for the Han-Kobayashi region. Allowing variable power control, we show that the Gaussian optimizers for the Han-Kobayashi region always lie in the stable regime. We propose an amended conjecture, whose validity would imply Gaussian optimality of the Han-Kobayashi bound in a certain regime.
Continuous-time Particle Filtering for Latent Stochastic Differential Equations
Venue: arXiv cs.LG 2022
Year: 2022

Particle filtering is a standard Monte-Carlo approach for a wide range of sequential inference tasks. The key component of a particle filter is a set of particles with importance weights that serve as a proxy of the true posterior distribution of some stochastic process. In this work, we propose continuous latent particle filters, an approach that extends particle filtering to the continuous-time domain. We demonstrate how continuous latent particle filters can be used as a generic plug-in replacement for inference techniques relying on a learned variational posterior. Our experiments with different model families based on latent neural stochastic differential equations demonstrate superior performance of continuous-time particle filtering in inference tasks like likelihood estimation and sequential prediction for a variety of stochastic processes.
Hermes: Accelerating Long-Latency Load Requests via Perceptron-Based Off-Chip Load Prediction
Venue: arXiv cs.AR 2022
Year: 2022

Long-latency load requests continue to limit the performance of high-performance processors. To increase the latency tolerance of a processor, architects have primarily relied on two key techniques: sophisticated data prefetchers and large on-chip caches. In this work, we show that: 1) even a sophisticated state-of-the-art prefetcher can only predict half of the off-chip load requests on average across a wide range of workloads, and 2) due to the increasing size and complexity of on-chip caches, a large fraction of the latency of an off-chip load request is spent accessing the on-chip cache hierarchy. The goal of this work is to accelerate off-chip load requests by removing the on-chip cache access latency from their critical path. To this end, we propose a new technique called Hermes, whose key idea is to: 1) accurately predict which load requests might go off-chip, and 2) speculatively fetch the data required by the predicted off-chip loads directly from the main memory, while also concurrently accessing the cache hierarchy for such loads. To enable Hermes, we develop a new lightweight, perceptron-based off-chip load prediction technique that learns to identify off-chip load requests using multiple program features (e.g., sequence of program counters). For every load request, the predictor observes a set of program features to predict whether or not the load would go off-chip. If the load is predicted to go off-chip, Hermes issues a speculative request directly to the memory controller once the load's physical address is generated. If the prediction is correct, the load eventually misses the cache hierarchy and waits for the ongoing speculative request to finish, thus hiding the on-chip cache hierarchy access latency from the critical path of the off-chip load. Our evaluation shows that Hermes significantly improves performance of a state-of-the-art baseline. We open-source Hermes.
Spherical Graph Drawing by Multi-dimensional Scaling
Venue: arXiv cs.CG 2022
Year: 2022

We describe an efficient and scalable spherical graph embedding method. The method uses a generalization of the Euclidean stress function for Multi-Dimensional Scaling adapted to spherical space, where geodesic pairwise distances are employed instead of Euclidean distances. The resulting spherical stress function is optimized by means of stochastic gradient descent. Quantitative and qualitative evaluations demonstrate the scalability and effectiveness of the proposed method. We also show that some graph families can be embedded with lower distortion on the sphere, than in Euclidean and hyperbolic spaces.
Combating Noisy Labels in Long-Tailed Image Classification
Venue: arXiv cs.CV 2022
Year: 2022

Most existing methods that cope with noisy labels usually assume that the class distributions are well balanced, which has insufficient capacity to deal with the practical scenarios where training samples have imbalanced distributions. To this end, this paper makes an early effort to tackle the image classification task with both long-tailed distribution and label noise. Existing noise-robust learning methods cannot work in this scenario as it is challenging to differentiate noisy samples from clean samples of tail classes. To deal with this problem, we propose a new learning paradigm based on matching between inferences on weak and strong data augmentations to screen out noisy samples and introduce a leave-noise-out regularization to eliminate the effect of the recognized noisy samples. Furthermore, we incorporate a novel prediction penalty based on online prior distribution to avoid bias towards head classes. This mechanism has superiority in capturing the class fitting degree in realtime compared to the existing long-tail classification methods. Exhaustive experiments demonstrate that the proposed method outperforms state-of-the-art algorithms that address the distribution imbalance problem in long-tailed classification under noisy labels.
Self-Supervised Pretraining for 2D Medical Image Segmentation
Venue: arXiv cs.CV 2022
Year: 2022

Supervised machine learning provides state-of-the-art solutions to a wide range of computer vision problems. However, the need for copious labelled training data limits the capabilities of these algorithms in scenarios where such input is scarce or expensive. Self-supervised learning offers a way to lower the need for manually annotated data by pretraining models for a specific domain on unlabelled data. In this approach, labelled data are solely required to fine-tune models for downstream tasks. Medical image segmentation is a field where labelling data requires expert knowledge and collecting large labelled datasets is challenging; therefore, self-supervised learning algorithms promise substantial improvements in this field. Despite this, self-supervised learning algorithms are used rarely to pretrain medical image segmentation networks. In this paper, we elaborate and analyse the effectiveness of supervised and self-supervised pretraining approaches on downstream medical image segmentation, focusing on convergence and data efficiency. We find that self-supervised pretraining on natural images and target-domain-specific images leads to the fastest and most stable downstream convergence. In our experiments on the ACDC cardiac segmentation dataset, this pretraining approach achieves 4-5 times faster fine-tuning convergence compared to an ImageNet pretrained model. We also show that this approach requires less than five epochs of pretraining on domain-specific data to achieve such improvement in the downstream convergence time. Finally, we find that, in low-data scenarios, supervised ImageNet pretraining achieves the best accuracy, requiring less than 100 annotated samples to realise close to minimal error.
Identifying Out-of-Distribution Samples in Real-Time for Safety-Critical 2D Object Detection with Margin Entropy Loss
Venue: arXiv cs.CV 2022
Year: 2022

Convolutional Neural Networks (CNNs) are nowadays often employed in vision-based perception stacks for safetycritical applications such as autonomous driving or Unmanned Aerial Vehicles (UAVs). Due to the safety requirements in those use cases, it is important to know the limitations of the CNN and, thus, to detect Out-of-Distribution (OOD) samples. In this work, we present an approach to enable OOD detection for 2D object detection by employing the margin entropy (ME) loss. The proposed method is easy to implement and can be applied to most existing object detection architectures. In addition, we introduce Separability as a metric for detecting OOD samples in object detection. We show that a CNN trained with the ME loss significantly outperforms OOD detection using standard confidence scores. At the same time, the runtime of the underlying object detection framework remains constant rendering the ME loss a powerful tool to enable OOD detection.
Has My Release Disobeyed Semantic Versioning? Static Detection Based on Semantic Differencing
Venue: arXiv cs.SE 2022
Year: 2022

To enhance the compatibility in the version control of Java Third-party Libraries (TPLs), Maven adopts Semantic Versioning (SemVer) to standardize the underlying meaning of versions, but users could still confront abnormal execution and crash after upgrades even if compilation and linkage succeed. It is caused by semantic breaking (SemB) issues, such that APIs directly used by users have identical signatures but inconsistent semantics across upgrades. To strengthen compliance with SemVer rules, developers and users should be alerted of such issues. Unfortunately, it is challenging to detect them statically, because semantic changes in the internal methods of APIs are difficult to capture. Dynamic testing can confirmingly uncover some, but it is limited by inadequate coverage. To detect SemB issues over compatible upgrades (Patch and Minor) by SemVer rules, we conduct an empirical study on 180 SemB issues to understand the root causes, inspired by which, we propose Sembid (Semantic Breaking Issue Detector) to statically detect such issues of TPLs for developers and users. Since APIs are directly used by users, Sembid detects and reports SemB issues based on APIs. For a pair of APIs, Sembid walks through the call chains originating from the API to locate breaking changes by measuring semantic diff. Then, Sembid checks if the breaking changes can affect API's output along call chains. The evaluation showed Sembid achieved 90.26% recall and 81.29% precision and outperformed other API checkers on SemB API detection. We also revealed Sembid detected over 3 times more SemB APIs with better coverage than unit tests, the commonly used solution. Furthermore, we carried out an empirical study on 1,629,589 APIs from 546 version pairs of top Java libraries and found there were 2-4 times more SemB APIs than those with signature-based issues.
Review of the AMLAS Methodology for Application in Healthcare
Venue: arXiv cs.LG 2022
Year: 2022

In recent years, the number of machine learning (ML) technologies gaining regulatory approval for healthcare has increased significantly allowing them to be placed on the market. However, the regulatory frameworks applied to them were originally devised for traditional software, which has largely rule-based behaviour, compared to the data-driven and learnt behaviour of ML. As the frameworks are in the process of reformation, there is a need to proactively assure the safety of ML to prevent patient safety being compromised. The Assurance of Machine Learning for use in Autonomous Systems (AMLAS) methodology was developed by the Assuring Autonomy International Programme based on well-established concepts in system safety. This review has appraised the methodology by consulting ML manufacturers to understand if it converges or diverges from their current safety assurance practices, whether there are gaps and limitations in its structure and if it is fit for purpose when applied to the healthcare domain. Through this work we offer the view that there is clear utility for AMLAS as a safety assurance methodology when applied to healthcare machine learning technologies, although development of healthcare specific supplementary guidance would benefit those implementing the methodology.
Identifying Films with Noir Characteristics Using Audience's Tags on MovieLens
Venue: arXiv cs.IR 2022
Year: 2022

We consider the noir classification problem by exploring noir attributes and what films are likely to be regarded as noirish from the perspective of a wide Internet audience. We use a dataset consisting of more than 30,000 films with relevant tags added by users of MovieLens, a web-based recommendation system. Based on this data, we develop a statistical model to identify films with noir characteristics using these free-form tags. After retrieving information for describing films from tags, we implement a one-class nearest neighbors algorithm to recognize noirish films by learning from IMDb-labeled noirs. Our analysis evidences film noirs' close relationship with German Expressionism, French Poetic Realism, British thrillers, and American pre-code crime pictures, revealing the similarities and differences between neo noirs after 1960 and noirs in the classic period.
A Multi-Channel Next POI Recommendation Framework with Multi-Granularity Check-in Signals
Venue: arXiv cs.IR 2022
Year: 2022

Current study on next POI recommendation mainly explores user sequential transitions with the fine-grained individual-user POI check-in trajectories only, which suffers from the severe check-in data sparsity issue. In fact, coarse-grained signals (i.e., region- and global-level check-ins) in such sparse check-ins would also benefit to augment user preference learning. Specifically, our data analysis unveils that user movement exhibits noticeable patterns w.r.t. the regions of visited POIs. Meanwhile, the global all-user check-ins can help reflect sequential regularities shared by the crowd. We are, therefore, inspired to propose the MCMG: a Multi-Channel next POI recommendation framework with Multi-Granularity signals categorized from two orthogonal perspectives, i.e., fine-coarse grained check-ins at either POI/region level or local/global level. Being equipped with three modules (i.e., global user behavior encoder, local multi-channel encoder, and region-aware weighting strategy), MCMG is capable of capturing both fine- and coarse-grained sequential regularities as well as exploring the dynamic impact of multi-channel by differentiating the region check-in patterns. Extensive experiments on four real-world datasets show that our MCMG significantly outperforms state-of-the-art next POI recommendation approaches.
Incremental Online Learning Algorithms Comparison for Gesture and Visual Smart Sensors
Venue: arXiv cs.LG 2022
Year: 2022

Tiny machine learning (TinyML) in IoT systems exploits MCUs as edge devices for data processing. However, traditional TinyML methods can only perform inference, limited to static environments or classes. Real case scenarios usually work in dynamic environments, thus drifting the context where the original neural model is no more suitable. For this reason, pre-trained models reduce accuracy and reliability during their lifetime because the data recorded slowly becomes obsolete or new patterns appear. Continual learning strategies maintain the model up to date, with runtime fine-tuning of the parameters. This paper compares four state-of-the-art algorithms in two real applications: i) gesture recognition based on accelerometer data and ii) image classification. Our results confirm these systems' reliability and the feasibility of deploying them in tiny-memory MCUs, with a drop in the accuracy of a few percentage points with respect to the original models for unconstrained computing platforms.
Python Implementation of the Dynamic Distributed Dimensional Data Model
Venue: arXiv cs.DC 2022
Year: 2022

Python has become a standard scientific computing language with fast-growing support of machine learning and data analysis modules, as well as an increasing usage of big data. The Dynamic Distributed Dimensional Data Model (D4M) offers a highly composable, unified data model with strong performance built to handle big data fast and efficiently. In this work we present an implementation of D4M in Python. $D4M.py$ implements all foundational functionality of D4M and includes Accumulo and SQL database support via Graphulo. We describe the mathematical background and motivation, an explanation of the approaches made for its fundamental functions and building blocks, and performance results which compare $D4M.py$'s performance to D4M-MATLAB and D4M.jl.
Online Meta-Learning for Model Update Aggregation in Federated Learning for Click-Through Rate Prediction
Venue: arXiv cs.IR 2022
Year: 2022

In Federated Learning (FL) of click-through rate (CTR) prediction, users' data is not shared for privacy protection. The learning is performed by training locally on client devices and communicating only model changes to the server. There are two main challenges: (i) the client heterogeneity, making FL algorithms that use the weighted averaging to aggregate model updates from the clients have slow progress and unsatisfactory learning results; and (ii) the difficulty of tuning the server learning rate with trial-and-error methodology due to the big computation time and resources needed for each experiment. To address these challenges, we propose a simple online meta-learning method to learn a strategy of aggregating the model updates, which adaptively weighs the importance of the clients based on their attributes and adjust the step sizes of the update. We perform extensive evaluations on public datasets. Our method significantly outperforms the state-of-the-art in both the speed of convergence and the quality of the final learning results.
A (Co)Algebraic Framework for Ordered Processes
Venue: arXiv cs.LO 2022
Year: 2022

A recently published paper (Schmid, Rozowski, Silva, and Rot, 2022) offers a (co)algebraic framework for studying processes with algebraic branching structures and recursion operators. The framework captures Milner's algebra of regular behaviours (Milner, 1984) but fails to give an honest account of a closely related calculus of probabilistic processes (Stark and Smolka, 1999). We capture Stark and Smolka's calculus by giving an alternative framework, aimed at studying a family of ordered process calculi with inequationally specified branching structures and recursion operators. We observe that a recent probabilistic extension of guarded Kleene algebra with tests (Rozowski, Kozen, Kappe, Schmid, Silva, 2022) is a fragment of one of our calculi, along with other examples. We also compare the intrinsic order in our process calculi with the notion of similarity in coalgebra.
Zero-Shot Multi-Modal Artist-Controlled Retrieval and Exploration of 3D Object Sets
Venue: arXiv cs.CV 2022
Year: 2022

When creating 3D content, highly specialized skills are generally needed to design and generate models of objects and other assets by hand. We address this problem through high-quality 3D asset retrieval from multi-modal inputs, including 2D sketches, images and text. We use CLIP as it provides a bridge to higher-level latent features. We use these features to perform a multi-modality fusion to address the lack of artistic control that affects common data-driven approaches. Our approach allows for multi-modal conditional feature-driven retrieval through a 3D asset database, by utilizing a combination of input latent embeddings. We explore the effects of different combinations of feature embeddings across different input types and weighting methods.
Online Demand Scheduling with Failovers
Venue: arXiv cs.DS 2022
Year: 2022

Motivated by cloud computing applications, we study the problem of how to optimally deploy new hardware subject to both power and robustness constraints. To model the situation observed in large-scale data centers, we introduce the Online Demand Scheduling with Failover problem. There are $m$ identical devices with capacity constraints. Demands come one-by-one and, to be robust against a device failure, need to be assigned to a pair of devices. When a device fails (in a failover scenario), each demand assigned to it is rerouted to its paired device (which may now run at increased capacity). The goal is to assign demands to the devices to maximize the total utilization subject to both the normal capacity constraints as well as these novel failover constraints. These latter constraints introduce new decision tradeoffs not present in classic assignment problems such as the Multiple Knapsack problem and AdWords. In the worst-case model, we design a deterministic $\approx \frac{1}{2}$-competitive algorithm, and show this is essentially tight. To circumvent this constant-factor loss, which in the context of big cloud providers represents substantial capital losses, we consider the stochastic arrival model, where all demands come i.i.d. from an unknown distribution. In this model we design an algorithm that achieves a sub-linear additive regret (i.e. as OPT or $m$ increases, the multiplicative competitive ratio goes to $1$). This requires a combination of different techniques, including a configuration LP with a non-trivial post-processing step and an online monotone matching procedure introduced by Rhee and Talagrand.
Universal Fourier Attack for Time Series
Venue: arXiv cs.CR 2022
Year: 2022

A wide variety of adversarial attacks have been proposed and explored using image and audio data. These attacks are notoriously easy to generate digitally when the attacker can directly manipulate the input to a model, but are much more difficult to implement in the real-world. In this paper we present a universal, time invariant attack for general time series data such that the attack has a frequency spectrum primarily composed of the frequencies present in the original data. The universality of the attack makes it fast and easy to implement as no computation is required to add it to an input, while time invariance is useful for real-world deployment. Additionally, the frequency constraint ensures the attack can withstand filtering. We demonstrate the effectiveness of the attack in two different domains, speech recognition and unintended radiated emission, and show that the attack is robust against common transform-and-compare defense pipelines.