OpenRaiser/Intern-Atlas huggingface dataset:
Scanpath Prediction on Information Visualisations
Venue: arXiv cs.CV 2023
Year: 2023

We propose Unified Model of Saliency and Scanpaths (UMSS) -- a model that learns to predict visual saliency and scanpaths (i.e. sequences of eye fixations) on information visualisations. Although scanpaths provide rich information about the importance of different visualisation elements during the visual exploration process, prior work has been limited to predicting aggregated attention statistics, such as visual saliency. We present in-depth analyses of gaze behaviour for different information visualisation elements (e.g. Title, Label, Data) on the popular MASSVIS dataset. We show that while, overall, gaze patterns are surprisingly consistent across visualisations and viewers, there are also structural differences in gaze dynamics for different elements. Informed by our analyses, UMSS first predicts multi-duration element-level saliency maps, then probabilistically samples scanpaths from them. Extensive experiments on MASSVIS show that our method consistently outperforms state-of-the-art methods with respect to several, widely used scanpath and saliency evaluation metrics. Our method achieves a relative improvement in sequence score of 11.5% for scanpath prediction, and a relative improvement in Pearson correlation coefficient of up to 23.6% for saliency prediction. These results are auspicious and point towards richer user models and simulations of visual attention on visualisations without the need for any eye tracking equipment.
Understanding Dynamic Spatio-Temporal Contexts in Long Short-Term Memory for Road Traffic Speed Prediction
Venue: arXiv cs.LG 2023
Year: 2023

Reliable traffic flow prediction is crucial to creating intelligent transportation systems. Many big-data-based prediction approaches have been developed but they do not reflect complicated dynamic interactions between roads considering time and location. In this study, we propose a dynamically localised long short-term memory (LSTM) model that involves both spatial and temporal dependence between roads. To do so, we use a localised dynamic spatial weight matrix along with its dynamic variation. Moreover, the LSTM model can deal with sequential data with long dependency as well as complex non-linear features. Empirical results indicated superior prediction performances of the proposed model compared to two different baseline methods.
Voxelized 3D Feature Aggregation for Multiview Detection
Venue: arXiv cs.CV 2023
Year: 2023

Multi-view detection incorporates multiple camera views to alleviate occlusion in crowded scenes, where the state-of-the-art approaches adopt homography transformations to project multi-view features to the ground plane. However, we find that these 2D transformations do not take into account the object's height, and with this neglection features along the vertical direction of same object are likely not projected onto the same ground plane point, leading to impure ground-plane features. To solve this problem, we propose VFA, voxelized 3D feature aggregation, for feature transformation and aggregation in multi-view detection. Specifically, we voxelize the 3D space, project the voxels onto each camera view, and associate 2D features with these projected voxels. This allows us to identify and then aggregate 2D features along the same vertical line, alleviating projection distortions to a large extent. Additionally, because different kinds of objects (human vs. cattle) have different shapes on the ground plane, we introduce the oriented Gaussian encoding to match such shapes, leading to increased accuracy and efficiency. We perform experiments on multiview 2D detection and multiview 3D detection problems. Results on four datasets (including a newly introduced MultiviewC dataset) show that our system is very competitive compared with the state-of-the-art approaches. %Our code and data will be open-sourced.Code and MultiviewC are released at https://github.com/Robert-Mar/VFA.
Learning a Robust Multiagent Driving Policy for Traffic Congestion Reduction
Venue: arXiv cs.AI 2023
Year: 2023

In most modern cities, traffic congestion is one of the most salient societal challenges. Past research has shown that inserting a limited number of autonomous vehicles (AVs) within the traffic flow, with driving policies learned specifically for the purpose of reducing congestion, can significantly improve traffic conditions. However, to date these AV policies have generally been evaluated under the same limited conditions under which they were trained. On the other hand, to be considered for practical deployment, they must be robust to a wide variety of traffic conditions. This article establishes for the first time that a multiagent driving policy can be trained in such a way that it generalizes to different traffic flows, AV penetration, and road geometries, including on multi-lane roads. Inspired by our successful results in a high-fidelity microsimulation, this article further contributes a novel extension of the well-known Cell Transmission Model (CTM) that, unlike past CTMs, is suitable for modeling congestion in traffic networks, and is thus suitable for studying congestion-reduction policies such as those considered in this article.
Semantic OcTree Mapping and Shannon Mutual Information Computation for Robot Exploration
Venue: arXiv cs.RO 2023
Year: 2023

Autonomous robot operation in unstructured and unknown environments requires efficient techniques for mapping and exploration using streaming range and visual observations. Information-based exploration techniques, such as Cauchy-Schwarz quadratic mutual information (CSQMI) and fast Shannon mutual information (FSMI), have successfully achieved active binary occupancy mapping with range measurements. However, as we envision robots performing complex tasks specified with semantically meaningful concepts, it is necessary to capture semantics in the measurements, map representation, and exploration objective. This work presents Semantic octree mapping and Shannon Mutual Information (SSMI) computation for robot exploration. We develop a Bayesian multi-class mapping algorithm based on an octree data structure, where each voxel maintains a categorical distribution over semantic classes. We derive a closed-form efficiently-computable lower bound of the Shannon mutual information between a multi-class octomap and a set of range-category measurements using semantic run-length encoding of the sensor rays. The bound allows rapid evaluation of many potential robot trajectories for autonomous exploration and mapping. We compare our method against state-of-the-art exploration techniques and apply it in a variety of simulated and real-world experiments.
CoMP-Enhanced Flexible Functional Split for Mixed Services in Beyond 5G Wireless Networks
Venue: arXiv cs.NI 2023
Year: 2023

With explosively escalating service demands, beyond fifth generation (B5G) aims to realize various requirements for multi-service networks, i.e., higher performance of mixed enhanced mobile broadband (eMBB) and ultra-reliable low-latency communication (URLLC) services than 5G. To flexibly serve diverse traffic, various functional split options (FSOs) are specified by 5G protocols enabling different network functions. In order to improve signal qualities for edge users, we consider FSO-based coordinated multi-point (CoMP) transmission as a prominent technique capable of supporting high traffic demands. However, due to conventional confined hardware processing capability, a processor sharing (PS) model is introduced to deal with high latency for multi-service FSO-based networks. Therefore, it becomes essential to assign CoMP-enhanced functional split modes under PS model. A more tractable FSO-based network in terms of ergodic rate and reliability is derived by stochastic geometry approach. Moreover, we have proposed CoMP-enhanced functional split mode allocation (CFSMA) scheme to adaptively assign FSOs to provide enhanced mixed throughput and latency-aware services. The simulation results have validated analytical derivation and demonstrated that the proposed CFSMA scheme optimizes system spectrum efficiency while guaranteeing stringent latency requirement. The proposed CFSMA scheme with the designed PS FFS-CoMP system outperforms the benchmarks of conventional FCFS scheduling, non-FSO network, fixed FSOs, and limited available FSO selections in open literature.
A Review for Deep Reinforcement Learning in Atari:Benchmarks, Challenges, and Solutions
Venue: arXiv cs.AI 2023
Year: 2023

The Arcade Learning Environment (ALE) is proposed as an evaluation platform for empirically assessing the generality of agents across dozens of Atari 2600 games. ALE offers various challenging problems and has drawn significant attention from the deep reinforcement learning (RL) community. From Deep Q-Networks (DQN) to Agent57, RL agents seem to achieve superhuman performance in ALE. However, is this the case? In this paper, to explore this problem, we first review the current evaluation metrics in the Atari benchmarks and then reveal that the current evaluation criteria of achieving superhuman performance are inappropriate, which underestimated the human performance relative to what is possible. To handle those problems and promote the development of RL research, we propose a novel Atari benchmark based on human world records (HWR), which puts forward higher requirements for RL agents on both final performance and learning efficiency. Furthermore, we summarize the state-of-the-art (SOTA) methods in Atari benchmarks and provide benchmark results over new evaluation metrics based on human world records. We concluded that at least four open challenges hinder RL agents from achieving superhuman performance from those new benchmark results. Finally, we also discuss some promising ways to handle those problems.
Ambiguous Dynamic Treatment Regimes: A Reinforcement Learning Approach
Venue: arXiv cs.LG 2023
Year: 2023

A main research goal in various studies is to use an observational data set and provide a new set of counterfactual guidelines that can yield causal improvements. Dynamic Treatment Regimes (DTRs) are widely studied to formalize this process. However, available methods in finding optimal DTRs often rely on assumptions that are violated in real-world applications (e.g., medical decision-making or public policy), especially when (a) the existence of unobserved confounders cannot be ignored, and (b) the unobserved confounders are time-varying (e.g., affected by previous actions). When such assumptions are violated, one often faces ambiguity regarding the underlying causal model. This ambiguity is inevitable, since the dynamics of unobserved confounders and their causal impact on the observed part of the data cannot be understood from the observed data. Motivated by a case study of finding superior treatment regimes for patients who underwent transplantation in our partner hospital and faced a medical condition known as New Onset Diabetes After Transplantation (NODAT), we extend DTRs to a new class termed Ambiguous Dynamic Treatment Regimes (ADTRs), in which the causal impact of treatment regimes is evaluated based on a "cloud" of causal models. We then connect ADTRs to Ambiguous Partially Observable Mark Decision Processes (APOMDPs) and develop Reinforcement Learning methods, which enable using the observed data to efficiently learn an optimal treatment regime. We establish theoretical results for these learning methods, including (weak) consistency and asymptotic normality. We further evaluate the performance of these learning methods both in our case study and in simulation experiments.
BungeeNeRF: Progressive Neural Radiance Field for Extreme Multi-scale Scene Rendering
Venue: arXiv cs.CV 2023
Year: 2023

Neural radiance fields (NeRF) has achieved outstanding performance in modeling 3D objects and controlled scenes, usually under a single scale. In this work, we focus on multi-scale cases where large changes in imagery are observed at drastically different scales. This scenario vastly exists in real-world 3D environments, such as city scenes, with views ranging from satellite level that captures the overview of a city, to ground level imagery showing complex details of an architecture; and can also be commonly identified in landscape and delicate minecraft 3D models. The wide span of viewing positions within these scenes yields multi-scale renderings with very different levels of detail, which poses great challenges to neural radiance field and biases it towards compromised results. To address these issues, we introduce BungeeNeRF, a progressive neural radiance field that achieves level-of-detail rendering across drastically varied scales. Starting from fitting distant views with a shallow base block, as training progresses, new blocks are appended to accommodate the emerging details in the increasingly closer views. The strategy progressively activates high-frequency channels in NeRF's positional encoding inputs and successively unfolds more complex details as the training proceeds. We demonstrate the superiority of BungeeNeRF in modeling diverse multi-scale scenes with drastically varying views on multiple data sources (city models, synthetic, and drone captured data) and its support for high-quality rendering in different levels of detail.
Neural Network Quantization for Efficient Inference: A Survey
Venue: arXiv cs.LG 2023
Year: 2023

As neural networks have become more powerful, there has been a rising desire to deploy them in the real world; however, the power and accuracy of neural networks is largely due to their depth and complexity, making them difficult to deploy, especially in resource-constrained devices. Neural network quantization has recently arisen to meet this demand of reducing the size and complexity of neural networks by reducing the precision of a network. With smaller and simpler networks, it becomes possible to run neural networks within the constraints of their target hardware. This paper surveys the many neural network quantization techniques that have been developed in the last decade. Based on this survey and comparison of neural network quantization techniques, we propose future directions of research in the area.
Towards More Efficient Insertion Transformer with Fractional Positional Encoding
Venue: arXiv cs.CL 2023
Year: 2023

Auto-regressive neural sequence models have been shown to be effective across text generation tasks. However, their left-to-right decoding order prevents generation from being parallelized. Insertion Transformer (Stern et al., 2019) is an attractive alternative that allows outputting multiple tokens in a single generation step. Nevertheless, due to the incompatibility between absolute positional encoding and insertion-based generation schemes, it needs to refresh the encoding of every token in the generated partial hypothesis at each step, which could be costly. We design a novel reusable positional encoding scheme for Insertion Transformers called Fractional Positional Encoding (FPE), which allows reusing representations calculated in previous steps. Empirical studies on various text generation tasks demonstrate the effectiveness of FPE, which leads to floating-point operation reduction and latency improvements on batched decoding.
Pool-Party: Exploiting Browser Resource Pools as Side-Channels for Web Tracking
Venue: arXiv cs.CR 2023
Year: 2023

We identify class of covert channels in browsers that are not mitigated by current defenses, which we call "pool-party" attacks. Pool-party attacks allow sites to create covert channels by manipulating limited-but-unpartitioned resource pools. These class of attacks have been known, but in this work we show that they are both more prevalent, more practical for exploitation, and allow exploitation in more ways, than previously identified. These covert channels have sufficient bandwidth to pass cookies and identifiers across site boundaries under practical and real-world conditions. We identify pool-party attacks in all popular browsers, and show they are practical cross-site tracking techniques (i.e., attacks take 0.6s in Chrome and Edge, and 7s in Firefox and Tor Browser). In this paper we make the following contributions: first, we describe pool-party covert channel attacks that exploit limits in application-layer resource pools in browsers. Second, we demonstrate that pool-party attacks are practical, and can be used to track users in all popular browsers; we also share open source implementations of the attack and evaluate them through a representative web crawl. Third, we show that in Gecko based-browsers (including the Tor Browser) pool-party attacks can also be used for cross-profile tracking (e.g., linking user behavior across normal and private browsing sessions). Finally, we discuss possible mitigation strategies and defenses
Synapse Compression for Event-Based Convolutional-Neural-Network Accelerators
Venue: arXiv cs.AR 2023
Year: 2023

Manufacturing-viable neuromorphic chips require novel computer architectures to achieve the massively parallel and efficient information processing the brain supports so effortlessly. Emerging event-based architectures are making this dream a reality. However, the large memory requirements for synaptic connectivity are a showstopper for the execution of modern convolutional neural networks (CNNs) on massively parallel, event-based (spiking) architectures. This work overcomes this roadblock by contributing a lightweight hardware scheme to compress the synaptic memory requirements by several thousand times, enabling the execution of complex CNNs on a single chip of small form factor. A silicon implementation in a 12-nm technology shows that the technique increases the system's implementation cost by only 2%, despite achieving a total memory-footprint reduction of up to 374x compared to the best previously published technique.
New Binary Quantum Codes Constructed from Quasi-Cyclic Codes
Venue: arXiv cs.IT 2023
Year: 2023

It is well known that quantum codes can be constructed by means of classical symplectic dual-containing codes. This paper considers a family of two-generator quasi-cyclic codes and derives sufficient conditions for these codes to be symplectic dual-containing. Then, a new method for constructing binary quantum codes using symplectic dual-containing codes is proposed. As an application, we construct 8 binary quantum codes that exceed the best-known results. Further, another 36 new binary quantum codes are obtained by propagation rules, all of which improve the lower bound on the minimum distances.
Verifying an Effect-Handler-Based Define-By-Run Reverse-Mode AD Library
Venue: arXiv cs.LO 2023
Year: 2023

We apply program verification technology to the problem of specifying and verifying automatic differentiation (AD) algorithms. We focus on define-by-run, a style of AD where the program that must be differentiated is executed and monitored by the automatic differentiation algorithm. We begin by asking, "what is an implementation of AD?" and "what does it mean for an implementation of AD to be correct?" We answer these questions both at an informal level, in precise English prose, and at a formal level, using types and logical assertions. After answering these broad questions, we focus on a specific implementation of AD, which involves a number of subtle programming-language features, including dynamically allocated mutable state, first-class functions, and effect handlers. We present a machine-checked proof, expressed in a modern variant of Separation Logic, of its correctness. We view this result as an advanced exercise in program verification, with potential future applications to the verification of more realistic automatic differentiation systems and of other software components that exploit delimited-control effects.
Exploring the Limits of Natural Language Inference Based Setup for Few-Shot Intent Detection
Venue: arXiv cs.CL 2023
Year: 2023

Intent Detection is one of the core tasks of dialog systems. Few-shot Intent Detection is challenging due to limited number of annotated utterances for novel classes. Generalized Few-shot intent detection is more realistic but challenging setup which aims to discriminate the joint label space of both novel intents which have few examples each and existing intents consisting of enough labeled data. Large label spaces and fewer number of shots increase the complexity of the task. In this work, we employ a simple and effective method based on Natural Language Inference that leverages the semantics in the class-label names to learn and predict the novel classes. Our method achieves state-of-the-art results on 1-shot and 5-shot intent detection task with gains ranging from 2-8\% points in F1 score on four benchmark datasets. Our method also outperforms existing approaches on a more practical setting of generalized few-shot intent detection with gains up to 20% F1 score. We show that the suggested approach performs well across single and multi domain datasets with the number of class labels from as few as 7 to as high as 150.
D-LNBot: A Scalable, Cost-Free and Covert Hybrid Botnet on Bitcoin's Lightning Network
Venue: arXiv cs.CR 2023
Year: 2023

While various covert botnets were proposed in the past, they still lack complete anonymization for their servers/botmasters or suffer from slow communication between the botmaster and the bots. In this paper, we first propose a new generation hybrid botnet that covertly and efficiently communicates over Bitcoin Lightning Network (LN), called LNBot. Exploiting various anonymity features of LN, we show the feasibility of a scalable two-layer botnet which completely anonymizes the identity of the botmaster. In the first layer, the botmaster anonymously sends the commands to the command and control (C&C) servers through regular LN payments. Specifically, LNBot allows botmaster's commands to be sent in the form of surreptitious multi-hop LN payments, where the commands are either encoded with the payments or attached to the payments to provide covert communications. In the second layer, C&C servers further relay those commands to the bots in their mini-botnets to launch any type of attacks to victim machines. We further improve on this design by introducing D-LNBot; a distributed version of LNBot that generates its C&C servers by infecting users on the Internet and forms the C&C connections by opening channels to the existing nodes on LN. In contrary to the LNBot, the whole botnet formation phase is distributed and the botmaster is never involved in the process. By utilizing Bitcoin's Testnet and the new message attachment feature of LN, we show that D-LNBot can be run for free and commands are propagated faster to all the C&C servers compared to LNBot. We presented proof-of-concept implementations for both LNBot and D-LNBot on the actual LN and extensively analyzed their delay and cost performance. Finally, we also provide and discuss a list of potential countermeasures to detect LNBot and D-LNBot activities and minimize their impacts.