OpenRaiser/Intern-Atlas huggingface dataset:
Variable Division and Optimization for Constrained Multiobjective Portfolio Problems
Venue: arXiv cs.NE 2021
Year: 2021

Variable division and optimization (D\&O) is a frequently utilized algorithm design paradigm in Evolutionary Algorithms (EAs). A D\&O EA divides a variable into partial variables and then optimize them respectively. A complicated problem is thus divided into simple subtasks. For example, a variable of portfolio problem can be divided into two partial variables, i.e. the selection of assets and the allocation of capital. Thereby, we optimize these two partial variables respectively. There is no formal discussion about how are the partial variables iteratively optimized and why can it work for both single- and multi-objective problems in D\&O. In this paper, this gap is filled. According to the discussion, an elitist selection method for partial variables in multiobjective problems is developed. Then this method is incorporated into the Decomposition-Based Multiobjective Evolutionary Algorithm (D\&O-MOEA/D). With the help of a mathematical programming optimizer, it is achieved on the constrained multiobjective portfolio problems. In the empirical study, D\&O-MOEA/D is implemented for 20 instances and recent Chinese stock markets. The results show the superiority and versatility of D\&O-MOEA/D on large-scale instances while the performance of it on small-scale problems is also not bad. The former targets convergence towards the Pareto front and the latter helps promote diversity among the non-dominated solutions during the search process.
MPASNET: Motion Prior-Aware Siamese Network for Unsupervised Deep Crowd Segmentation in Video Scenes
Venue: arXiv cs.CV 2021
Year: 2021

Crowd segmentation is a fundamental task serving as the basis of crowded scene analysis, and it is highly desirable to obtain refined pixel-level segmentation maps. However, it remains a challenging problem, as existing approaches either require dense pixel-level annotations to train deep learning models or merely produce rough segmentation maps from optical or particle flows with physical models. In this paper, we propose the Motion Prior-Aware Siamese Network (MPASNET) for unsupervised crowd semantic segmentation. This model not only eliminates the need for annotation but also yields high-quality segmentation maps. Specially, we first analyze the coherent motion patterns across the frames and then apply a circular region merging strategy on the collective particles to generate pseudo-labels. Moreover, we equip MPASNET with siamese branches for augmentation-invariant regularization and siamese feature aggregation. Experiments over benchmark datasets indicate that our model outperforms the state-of-the-arts by more than 12% in terms of mIoU.
Mindless Attractor: A False-Positive Resistant Intervention for Drawing Attention Using Auditory Perturbation
Venue: arXiv cs.HC 2021
Year: 2021

Explicitly alerting users is not always an optimal intervention, especially when they are not motivated to obey. For example, in video-based learning, learners who are distracted from the video would not follow an alert asking them to pay attention. Inspired by the concept of Mindless Computing, we propose a novel intervention approach, Mindless Attractor, that leverages the nature of human speech communication to help learners refocus their attention without relying on their motivation. Specifically, it perturbs the voice in the video to direct their attention without consuming their conscious awareness. Our experiments not only confirmed the validity of the proposed approach but also emphasized its advantages in combination with a machine learning-based sensing module. Namely, it would not frustrate users even though the intervention is activated by false-positive detection of their attentive state. Our intervention approach can be a reliable way to induce behavioral change in human-AI symbiosis.
Dive into Decision Trees and Forests: A Theoretical Demonstration
Venue: arXiv cs.LG 2021
Year: 2021

Based on decision trees, many fields have arguably made tremendous progress in recent years. In simple words, decision trees use the strategy of "divide-and-conquer" to divide the complex problem on the dependency between input features and labels into smaller ones. While decision trees have a long history, recent advances have greatly improved their performance in computational advertising, recommender system, information retrieval, etc. We introduce common tree-based models (e.g., Bayesian CART, Bayesian regression splines) and training techniques (e.g., mixed integer programming, alternating optimization, gradient descent). Along the way, we highlight probabilistic characteristics of tree-based models and explain their practical and theoretical benefits. Except machine learning and data mining, we try to show theoretical advances on tree-based models from other fields such as statistics and operation research. We list the reproducible resource at the end of each method.
Probabilistic Placement Optimization for Non-coherent and Coherent Joint Transmission in Cache-Enabled Cellular Networks
Venue: arXiv cs.IT 2021
Year: 2021

How to design proper content placement strategies is one of the major areas of interest in cache-enabled cellular networks. In this paper, we study the probabilistic content placement optimization of base station (BS) caching with cooperative transmission in the downlink of cellular networks. With placement probability vector being the design parameter, non-coherent joint transmission (NC-JT) and coherent joint transmission (C-JT) schemes are investigated according to whether channel state information (CSI) is available. Using stochastic geometry, we derive an integral expression for the successful transmission probability (STP) in NC-JT scheme, and present an upper bound and a tight approximation for the STP of the C-JT scheme. Next, we maximize the STP in NC-JT and the approximation of STP in C-JT by optimizing the placement probability vector, respectively. An algorithm is proposed and applied to both optimization problems. By utilizing some properties of the STP, we obtain globally optimal solutions in certain cases. Moreover, locally optimal solutions in general cases are obtained by using the interior point method. Finally, numerical results show the optimized placement strategy achieves significant gains in STP over several comparative baselines both in NC-JT and C-JT. The optimal STP in C-JT outperforms the one in NC-JT, indicating the benefits of knowing CSI in cooperative transmission.
Adversarial Machine Learning in Text Analysis and Generation
Venue: arXiv cs.LG 2021
Year: 2021

The research field of adversarial machine learning witnessed a significant interest in the last few years. A machine learner or model is secure if it can deliver main objectives with acceptable accuracy, efficiency, etc. while at the same time, it can resist different types and/or attempts of adversarial attacks. This paper focuses on studying aspects and research trends in adversarial machine learning specifically in text analysis and generation. The paper summarizes main research trends in the field such as GAN algorithms, models, types of attacks, and defense against those attacks.
Leafy Automata for Higher-Order Concurrency
Venue: arXiv cs.FL 2021
Year: 2021

Finitary Idealized Concurrent Algol (FICA) is a prototypical programming language combining functional, imperative, and concurrent computation. There exists a fully abstract game model of FICA, which in principle can be used to prove equivalence and safety of FICA programs. Unfortunately, the problems are undecidable for the whole language, and only very rudimentary decidable sub-languages are known. We propose leafy automata as a dedicated automata-theoretic formalism for representing the game semantics of FICA. The automata use an infinite alphabet with a tree structure. We show that the game semantics of any FICA term can be represented by traces of a leafy automaton. Conversely, the traces of any leafy automaton can be represented by a FICA term. Because of the close match with FICA, we view leafy automata as a promising starting point for finding decidable subclasses of the language and, more generally, to provide a new perspective on models of higher-order concurrent computation. Moreover, we identify a fragment of FICA that is amenable to verification by translation into a particular class of leafy automata. Using a locality property of the latter class, where communication between levels is restricted and every other level is bounded, we show that their emptiness problem is decidable by reduction to Petri net reachability.
Mechanism Design for Cumulative Prospect Theoretic Agents: A General Framework and the Revelation Principle
Venue: arXiv cs.GT 2021
Year: 2021

This paper initiates a discussion of mechanism design when the participating agents exhibit preferences that deviate from expected utility theory (EUT). In particular, we consider mechanism design for systems where the agents are modeled as having cumulative prospect theory (CPT) preferences, which is a generalization of EUT preferences. We point out some of the key modifications needed in the theory of mechanism design that arise from agents having CPT preferences and some of the shortcomings of the classical mechanism design framework. In particular, we show that the revelation principle, which has traditionally played a fundamental role in mechanism design, does not continue to hold under CPT. We develop an appropriate framework that we call mediated mechanism design which allows us to recover the revelation principle for CPT agents. We conclude with some interesting directions for future work.
Deductive Verification of Floating-Point Java Programs in KeY
Venue: arXiv cs.PL 2021
Year: 2021

Deductive verification has been successful in verifying interesting properties of real-world programs. One notable gap is the limited support for floating-point reasoning. This is unfortunate, as floating-point arithmetic is particularly unintuitive to reason about due to rounding as well as the presence of the special values infinity and `Not a Number' (NaN). In this paper, we present the first floating-point support in a deductive verification tool for the Java programming language. Our support in the KeY verifier handles arithmetic via floating-point decision procedures inside SMT solvers and transcendental functions via axiomatization. We evaluate this integration on new benchmarks, and show that this approach is powerful enough to prove the absence of floating-point special values -- often a prerequisite for further reasoning about numerical computations -- as well as certain functional properties for realistic benchmarks.
Clairvoyant Prefetching for Distributed Machine Learning I/O
Venue: arXiv cs.DC 2021
Year: 2021

I/O is emerging as a major bottleneck for machine learning training, especially in distributed environments. Indeed, at large scale, I/O takes as much as 85% of training time. Addressing this I/O bottleneck necessitates careful optimization, as optimal data ingestion pipelines differ between systems, and require a delicate balance between access to local storage, external filesystems, and remote nodes. We introduce NoPFS, a machine learning I/O middleware, which provides a scalable, flexible, and easy-to-use solution to the I/O bottleneck. NoPFS uses clairvoyance: Given the seed generating the random access pattern for training with SGD, it can exactly predict when and where a sample will be accessed. We combine this with an analysis of access patterns and a performance model to provide distributed caching policies that adapt to different datasets and storage hierarchies. NoPFS reduces I/O times and improves end-to-end training by up to 5.4x on the ImageNet-1k, ImageNet-22k, and CosmoFlow datasets.
A New Knowledge Gradient-based Method for Constrained Bayesian Optimization
Venue: arXiv cs.LG 2021
Year: 2021

Black-box problems are common in real life like structural design, drug experiments, and machine learning. When optimizing black-box systems, decision-makers always consider multiple performances and give the final decision by comprehensive evaluations. Motivated by such practical needs, we focus on constrained black-box problems where the objective and constraints lack known special structure, and evaluations are expensive and even with noise. We develop a novel constrained Bayesian optimization approach based on the knowledge gradient method ($c-\rm{KG}$). A new acquisition function is proposed to determine the next batch of samples considering optimality and feasibility. An unbiased estimator of the gradient of the new acquisition function is derived to implement the $c-\rm{KG}$ approach.
A Critical Comparison on Six Static Analysis Tools: Detection, Agreement, and Precision
Venue: arXiv cs.SE 2021
Year: 2021

Background. Developers use Automated Static Analysis Tools (ASATs) to control for potential quality issues in source code, including defects and technical debt. Tool vendors have devised quite a number of tools, which makes it harder for practitioners to select the most suitable one for their needs. To better support developers, researchers have been conducting several studies on ASATs to favor the understanding of their actual capabilities. Aims. Despite the work done so far, there is still a lack of knowledge regarding (1) which source quality problems can actually be detected by static analysis tool warnings, (2) what is their agreement, and (3) what is the precision of their recommendations. We aim at bridging this gap by proposing a large-scale comparison of six popular static analysis tools for Java projects: Better Code Hub, CheckStyle, Coverity Scan, Findbugs, PMD, and SonarQube. Method. We analyze 47 Java projects and derive a taxonomy of warnings raised by 6 state-of-the-practice ASATs. To assess their agreement, we compared them by manually analyzing - at line-level - whether they identify the same issues. Finally, we manually evaluate the precision of the tools. Results. The key results report a comprehensive taxonomy of ASATs warnings, show little to no agreement among the tools and a low degree of precision. Conclusions. We provide a taxonomy that can be useful to researchers, practitioners, and tool vendors to map the current capabilities of the tools. Furthermore, our study provides the first overview on the agreement among different tools as well as an extensive analysis of their precision.
Time-Correlated Sparsification for Communication-Efficient Federated Learning
Venue: arXiv cs.LG 2021
Year: 2021

Federated learning (FL) enables multiple clients to collaboratively train a shared model without disclosing their local datasets. This is achieved by exchanging local model updates with the help of a parameter server (PS). However, due to the increasing size of the trained models, the communication load due to the iterative exchanges between the clients and the PS often becomes a bottleneck in the performance. Sparse communication is often employed to reduce the communication load, where only a small subset of the model updates are communicated from the clients to the PS. In this paper, we introduce a novel time-correlated sparsification (TCS) scheme, which builds upon the notion that sparse communication framework can be considered as identifying the most significant elements of the underlying model. Hence, TCS seeks a certain correlation between the sparse representations used at consecutive iterations in FL, so that the overhead due to encoding and transmission of the sparse representation can be significantly reduced without compromising the test accuracy. Through extensive simulations on the CIFAR-10 dataset, we show that TCS can achieve centralized training accuracy with 100 times sparsification, and up to 2000 times reduction in the communication load when employed together with quantization.
A Spike Learning System for Event-driven Object Recognition
Venue: arXiv cs.CV 2021
Year: 2021

Event-driven sensors such as LiDAR and dynamic vision sensor (DVS) have found increased attention in high-resolution and high-speed applications. A lot of work has been conducted to enhance recognition accuracy. However, the essential topic of recognition delay or time efficiency is largely under-explored. In this paper, we present a spiking learning system that uses the spiking neural network (SNN) with a novel temporal coding for accurate and fast object recognition. The proposed temporal coding scheme maps each event's arrival time and data into SNN spike time so that asynchronously-arrived events are processed immediately without delay. The scheme is integrated nicely with the SNN's asynchronous processing capability to enhance time efficiency. A key advantage over existing systems is that the event accumulation time for each recognition task is determined automatically by the system rather than pre-set by the user. The system can finish recognition early without waiting for all the input events. Extensive experiments were conducted over a list of 7 LiDAR and DVS datasets. The results demonstrated that the proposed system had state-of-the-art recognition accuracy while achieving remarkable time efficiency. Recognition delay was shown to reduce by 56.3% to 91.7% in various experiment settings over the popular KITTI dataset.
Virtual Memory Partitioning for Enhancing Application Performance in Mobile Platforms
Venue: arXiv cs.AR 2021
Year: 2021

Recently, the amount of running software on smart mobile devices is gradually increasing due to the introduction of application stores. The application store is a type of digital distribution platform for application software, which is provided as a component of an operating system on a smartphone or tablet. Mobile devices have limited memory capacity and, unlike server and desktop systems, due to their mobility they do not have a memory slot that can expand the memory capacity. Low memory killer (LMK) and out-of-memory killer (OOMK) are widely used memory management solutions in mobile systems. They forcibly terminate applications when the available physical memory becomes insufficient. In addition, before the forced termination, the memory shortage incurs thrashing and fragmentation, thus slowing down application performance. Although the existing page reclamation mechanism is designed to secure available memory, it could seriously degrade user responsiveness due to the thrashing. Memory management is therefore still important especially in mobile devices with small memory capacity. This paper presents a new memory partitioning technique that resolves the deterioration of the existing application life cycle induced by LMK and OOMK. It provides a completely isolated virtual memory node at the operating system level. Evaluation results demonstrate that the proposed method improves application execution time under memory shortage, compared with methods in previous studies.
Cloud-Based Content Cooperation System to Assist Collaborative Learning Environment
Venue: arXiv cs.DC 2021
Year: 2021

Online educational systems running on smart devices have the advantage of allowing users to learn online regardless of the location of the users. In particular, data synchronization enables users to cooperate on contents in real time anywhere by sharing their files via cloud storage. However, users cannot collaborate by simultaneously modifying files that are shared with each other. In this paper, we propose a content collaboration method and a history management technique that are based on distributed system structure and can synchronize data shared in the cloud for multiple users and multiple devices.
AS-Net: Fast Photoacoustic Reconstruction with Multi-feature Fusion from Sparse Data
Venue: arXiv cs.CV 2021
Year: 2021

Photoacoustic (PA) imaging is a biomedical imaging modality capable of acquiring high-contrast images of optical absorption at depths much greater than traditional optical imaging techniques. However, practical instrumentation and geometry limit the number of available acoustic sensors surrounding the imaging target, which results in the sparsity of sensor data. Conventional PA image reconstruction methods give severe artifacts when they are applied directly to the sparse PA data. In this paper, we firstly propose to employ a novel signal processing method to make sparse PA raw data more suitable for the neural network, concurrently speeding up image reconstruction. Then we propose Attention Steered Network (AS-Net) for PA reconstruction with multi-feature fusion. AS-Net is validated on different datasets, including simulated photoacoustic data from fundus vasculature phantoms and experimental data from in vivo fish and mice. Notably, the method is also able to eliminate some artifacts present in the ground truth for in vivo data. Results demonstrated that our method provides superior reconstructions at a faster speed.
Knowledge Graph Completion with Text-aided Regularization
Venue: arXiv cs.CL 2021
Year: 2021

Knowledge Graph Completion is a task of expanding the knowledge graph/base through estimating possible entities, or proper nouns, that can be connected using a set of predefined relations, or verb/predicates describing interconnections of two things. Generally, we describe this problem as adding new edges to a current network of vertices and edges. Traditional approaches mainly focus on using the existing graphical information that is intrinsic of the graph and train the corresponding embeddings to describe the information; however, we think that the corpus that are related to the entities should also contain information that can positively influence the embeddings to better make predictions. In our project, we try numerous ways of using extracted or raw textual information to help existing KG embedding frameworks reach better prediction results, in the means of adding a similarity function to the regularization part in the loss function. Results have shown that we have made decent improvements over baseline KG embedding methods.
Predictive Wireless Based Status Update for Communication-Agnostic Sampling
Venue: arXiv cs.IT 2021
Year: 2021

In a wireless network that conveys status updates from sources (i.e., sensors) to destinations, one of the key issues studied by existing literature is how to design an optimal source sampling strategy on account of the communication constraints which are often modeled as queues. In this paper, an alternative perspective is presented -- a novel status-aware communication scheme, namely \emph{parallel communications}, is proposed which allows sensors to be communication-agnostic. Specifically, the proposed scheme can determine, based on an online prediction functionality, whether a status packet is worth transmitting considering both the network condition and status prediction, such that sensors can generate status packets without communication constraints. We evaluate the proposed scheme on a Software-Defined-Radio (SDR) test platform, which is integrated with a collaborative autonomous driving simulator, i.e., Simulation-of-Urban-Mobility (SUMO), to produce realistic vehicle control models and road conditions. The results show that with online status predictions, the channel occupancy is significantly reduced, while guaranteeing low status recovery error. Then the framework is applied to two scenarios: a multi-density platooning scenario, and a flight formation control scenario. Simulation results show that the scheme achieves better performance on the network level, in terms of keeping the minimum safe distance in both vehicle platooning and flight control.