OpenRaiser/Intern-Atlas huggingface dataset:
VoIPLoc: Passive VoIP call provenance via acoustic side-channels
Venue: arXiv cs.CR 2021
Year: 2021

We propose VoIPLoc, a novel location fingerprinting technique and apply it to the VoIP call provenance problem. It exploits echo-location information embedded within VoIP audio to support fine-grained location inference. We found consistent statistical features induced by the echo-reflection characteristics of the location into recorded speech. These features are discernible within traces received at the VoIP destination, enabling location inference. We evaluated VoIPLoc by developing a dataset of audio traces received through VoIP channels over the Tor network. We show that recording locations can be fingerprinted and detected remotely with a low false-positive rate, even when a majority of the audio samples are unlabelled. Finally, we note that the technique is fully passive and thus undetectable, unlike prior art. VoIPLoc is robust to the impact of environmental noise and background sounds, as well as the impact of compressive codecs and network jitter. The technique is also highly scalable and offers several degrees of freedom terms of the fingerprintable space.
ApproxNet: Content and Contention-Aware Video Analytics System for Embedded Clients
Venue: arXiv cs.CV 2021
Year: 2021

Videos take a lot of time to transport over the network, hence running analytics on the live video on embedded or mobile devices has become an important system driver. Considering that such devices, e.g., surveillance cameras or AR/VR gadgets, are resource constrained, creating lightweight deep neural networks (DNNs) for embedded devices is crucial. None of the current approximation techniques for object classification DNNs can adapt to changing runtime conditions, e.g., changes in resource availability on the device, the content characteristics, or requirements from the user. In this paper, we introduce ApproxNet, a video object classification system for embedded or mobile clients. It enables novel dynamic approximation techniques to achieve desired inference latency and accuracy trade-off under changing runtime conditions. It achieves this by enabling two approximation knobs within a single DNN model, rather than creating and maintaining an ensemble of models (e.g., MCDNN [MobiSys-16]. We show that ApproxNet can adapt seamlessly at runtime to these changes, provides low and stable latency for the image and video frame classification problems, and show the improvement in accuracy and latency over ResNet [CVPR-16], MCDNN [MobiSys-16], MobileNets [Google-17], NestDNN [MobiCom-18], and MSDNet [ICLR-18].
Regularized Anderson Acceleration for Off-Policy Deep Reinforcement Learning
Venue: arXiv cs.LG 2021
Year: 2021

Model-free deep reinforcement learning (RL) algorithms have been widely used for a range of complex control tasks. However, slow convergence and sample inefficiency remain challenging problems in RL, especially when handling continuous and high-dimensional state spaces. To tackle this problem, we propose a general acceleration method for model-free, off-policy deep RL algorithms by drawing the idea underlying regularized Anderson acceleration (RAA), which is an effective approach to accelerating the solving of fixed point problems with perturbations. Specifically, we first explain how policy iteration can be applied directly with Anderson acceleration. Then we extend RAA to the case of deep RL by introducing a regularization term to control the impact of perturbation induced by function approximation errors. We further propose two strategies, i.e., progressive update and adaptive restart, to enhance the performance. The effectiveness of our method is evaluated on a variety of benchmark tasks, including Atari 2600 and MuJoCo. Experimental results show that our approach substantially improves both the learning speed and final performance of state-of-the-art deep RL algorithms.
Democracy, Complexity, and Science: Exploring Structural Sources of National Scientific Performance
Venue: arXiv cs.DL 2021
Year: 2021

Scholars have long hypothesized that democratic forms of government are more compatible with scientific advancement. However, empirical analysis testing the democracy-science compatibility hypothesis remains underdeveloped. This article explores the effect of democratic governance on scientific performance using panel data on 124 countries between 2007 and 2017. We find evidence supporting the democracy-science hypothesis. Further, using both internal and external measures of complexity, we estimate the effects of complexity as a moderating factor between the democracy-science connection. The results show differential main effects of economic complexity, globalization, and international collaboration on scientific performance, as well as significant interaction effects that moderate the effect of democracy on scientific performance. The findings show the significance of democratic governance and complex systems in national scientific performance.
Is culture related to strong science? An empirical investigation
Venue: arXiv cs.DL 2021
Year: 2021

National culture is among those societal factors which could influence research and innovation activities. In this study, we investigated the associations of two national culture models with citation impact of nations (measured by the proportion of papers belonging to the 10% and 1% most cited papers in the corresponding fields, PPtop 10% and PPtop 1%). Bivariate statistical analyses showed that of six Hofstede's national culture dimensions (HNCD), uncertainty avoidance and power distance had a statistically significant negative associa-tion, while individualism and indulgence had a statistically significant positive associationwith both citation impact indicators. The study also revealed that of two Inglehart-Welzel cultural values (IWCV), the value survival versus self-expression is statistically significantly related to citation impact indicators. We additionally calculated multiple regression analyses controlling for the possible effects of confounding factors including national self-citations, international co-authorships, invest-ments in research and development, international migrant stock, number of researchers ofeach nation, language, and productivity. The results revealed that the statistically significant associations of HNCD with citation impact indicators disappeared. But the statistically significant relationship between survivals versus self-expression values and citation impact indicators remained stable even after controlling for the confounding variables. Thus, the freedom of expression and trust in society might contribute to better scholarly communication systems, higher level of international collaborations, and further quality research.
Repeat-Free Codes
Venue: arXiv cs.IT 2021
Year: 2021

In this paper we consider the problem of encoding data into \textit{repeat-free} sequences in which sequences are imposed to contain any $k$-tuple at most once (for predefined $k$). First, the capacity of the repeat-free constraint are calculated. Then, an efficient algorithm, which uses two bits of redundancy, is presented to encode length-$n$ sequences for $k=2+2\log (n)$. This algorithm is then improved to support any value of $k$ of the form $k=a\log (n)$, for $1<a$, while its redundancy is $o(n)$. We also calculate the capacity of repeat-free sequences when combined with local constraints which are given by a constrained system, and the capacity of multi-dimensional repeat-free codes.
Coarse-scale PDEs from fine-scale observations via machine learning
Venue: arXiv cs.LG 2021
Year: 2021

Complex spatiotemporal dynamics of physicochemical processes are often modeled at a microscopic level (through e.g. atomistic, agent-based or lattice models) based on first principles. Some of these processes can also be successfully modeled at the macroscopic level using e.g. partial differential equations (PDEs) describing the evolution of the right few macroscopic observables (e.g. concentration and momentum fields). Deriving good macroscopic descriptions (the so-called "closure problem") is often a time-consuming process requiring deep understanding/intuition about the system of interest. Recent developments in data science provide alternative ways to effectively extract/learn accurate macroscopic descriptions approximating the underlying microscopic observations. In this paper, we introduce a data-driven framework for the identification of unavailable coarse-scale PDEs from microscopic observations via machine learning algorithms. Specifically, using Gaussian Processes, Artificial Neural Networks, and/or Diffusion Maps, the proposed framework uncovers the relation between the relevant macroscopic space fields and their time evolution (the right-hand-side of the explicitly unavailable macroscopic PDE). Interestingly, several choices equally representative of the data can be discovered. The framework will be illustrated through the data-driven discovery of macroscopic, concentration-level PDEs resulting from a fine-scale, Lattice Boltzmann level model of a reaction/transport process. Once the coarse evolution law is identified, it can be simulated to produce long-term macroscopic predictions. Different features (pros as well as cons) of alternative machine learning algorithms for performing this task (Gaussian Processes and Artificial Neural Networks), are presented and discussed.
The Computational Complexity of Finding Temporal Paths under Waiting Time Constraints
Venue: arXiv cs.DM 2021
Year: 2021

Computing a (short) path between two vertices is one of the most fundamental primitives in graph algorithmics. In recent years, the study of paths in temporal graphs, that is, graphs where the vertex set is fixed but the edge set changes over time, gained more and more attention. A path is time-respecting, or temporal, if it uses edges with non-decreasing time stamps. We investigate a basic constraint for temporal paths, where the time spent at each vertex must not exceed a given duration $Δ$, referred to as $Δ$-restless temporal paths. This constraint arises naturally in the modeling of real-world processes like packet routing in communication networks and infection transmission routes of diseases where recovery confers lasting resistance. While finding temporal paths without waiting time restrictions is known to be doable in polynomial time, we show that the "restless variant" of this problem becomes computationally hard even in very restrictive settings. For example, it is W[1]-hard when parameterized by the distance to disjoint path of the underlying graph, which implies W[1]-hardness for many other parameters like feedback vertex number and pathwidth. A natural question is thus whether the problem becomes tractable in some natural settings. We explore several natural parameterizations, presenting FPT algorithms for three kinds of parameters: (1) output-related parameters (here, the maximum length of the path), (2) classical parameters applied to the underlying graph (e.g., feedback edge number), and (3) a new parameter called timed feedback vertex number, which captures finer-grained temporal features of the input temporal graph, and which may be of interest beyond this work.
Deep Neural Networks for Choice Analysis: Architectural Design with Alternative-Specific Utility Functions
Venue: arXiv cs.LG 2021
Year: 2021

Whereas deep neural network (DNN) is increasingly applied to choice analysis, it is challenging to reconcile domain-specific behavioral knowledge with generic-purpose DNN, to improve DNN's interpretability and predictive power, and to identify effective regularization methods for specific tasks. This study designs a particular DNN architecture with alternative-specific utility functions (ASU-DNN) by using prior behavioral knowledge. Unlike a fully connected DNN (F-DNN), which computes the utility value of an alternative k by using the attributes of all the alternatives, ASU-DNN computes it by using only k's own attributes. Theoretically, ASU-DNN can dramatically reduce the estimation error of F-DNN because of its lighter architecture and sparser connectivity. Empirically, ASU-DNN has 2-3% higher prediction accuracy than F-DNN over the whole hyperparameter space in a private dataset that we collected in Singapore and a public dataset in R mlogit package. The alternative-specific connectivity constraint, as a domain-knowledge-based regularization method, is more effective than the most popular generic-purpose explicit and implicit regularization methods and architectural hyperparameters. ASU-DNN is also more interpretable because it provides a more regular substitution pattern of travel mode choices than F-DNN does. The comparison between ASU-DNN and F-DNN can also aid in testing the behavioral knowledge. Our results reveal that individuals are more likely to compute utility by using an alternative's own attributes, supporting the long-standing practice in choice modeling. Overall, this study demonstrates that prior behavioral knowledge could be used to guide the architecture design of DNN, to function as an effective domain-knowledge-based regularization method, and to improve both the interpretability and predictive power of DNN in choice analysis.
MER-SDN: Machine Learning Framework for Traffic Aware Energy Efficient Routing in SDN
Venue: arXiv cs.NI 2021
Year: 2021

Software Defined Networking (SDN) achieves programmability of a network through separation of the control and data planes. It enables flexibility in network management and control. Energy efficiency is one of the challenging global problems which has both economic and environmental impact. A massive amount of information is generated in the controller of an SDN based network. Machine learning gives the ability to computers to progressively learn from data without having to write specific instructions. In this work, we propose MER-SDN: a machine learning framework for traffic-aware energy efficient routing in SDN. Feature extraction, training, and testing are the three main stages of the learning machine. Experiments are conducted on Mininet and POX controller using real-world network topology and dynamic traffic traces from SNDlib. Results show that our approach achieves more than 65\% feature size reduction, more than 70% accuracy in parameter prediction of an energy efficient heuristics algorithm, also our prediction refine heuristics converges the predicted value to the optimal parameters values with up to 25X speedup as compared to the brute force method.
From Server-Based to Client-Based Machine Learning: A Comprehensive Survey
Venue: arXiv cs.LG 2021
Year: 2021

In recent years, mobile devices have gained increasing development with stronger computation capability and larger storage space. Some of the computation-intensive machine learning tasks can now be run on mobile devices. To exploit the resources available on mobile devices and preserve personal privacy, the concept of client-based machine learning has been proposed. It leverages the users' local hardware and local data to solve machine learning sub-problems on mobile devices and only uploads computation results rather than the original data for the optimization of the global model. Such an architecture can not only relieve computation and storage burdens on servers but also protect the users' sensitive information. Another benefit is the bandwidth reduction because various kinds of local data can be involved in the training process without being uploaded. In this article, we provide a literature review on the progressive development of machine learning from server based to client based. We revisit a number of widely used server-based and client-based machine learning methods and applications. We also extensively discuss the challenges and future directions in this area. We believe that this survey will give a clear overview of client-based machine learning and provide guidelines on applying client-based machine learning to practice.
Utilizing Dependence among Variables in Evolutionary Algorithms for Mixed-Integer Programming: A Case Study on Multi-Objective Constrained Portfolio Optimization
Venue: arXiv cs.CE 2021
Year: 2021

Several real-world applications could be modeled as Mixed-Integer Non-Linear Programming (MINLP) problems, and some prominent examples include portfolio optimization, remote sensing technology, and so on. Most of the models for these applications are non-convex and always involve some conflicting objectives. The mathematical and heuristic methods have their advantages in solving this category of problems. In this work, we turn to Multi-Objective Evolutionary Algorithms (MOEAs) for finding elegant solutions for such problems. In this framework, we investigate a multi-objective constrained portfolio optimization problem, which can be cast as a classical financial problem and can also be naturally modeled as an MINLP problem. Consequently, we point out one challenge, faced by a direct coding scheme for MOEAs, to this problem. It is that the dependence among variables, like the selection and weights for one same asset, will likely make the search difficult. We thus, propose a Compressed Coding Scheme (CCS), compressing the two dependent variables into one variable to utilize the dependence and thereby meeting this challenge. Subsequently, we carry out a detailed empirical study on two sets of instances. The first part consists of 5 instances from OR-Library, which is solvable for the general mathematical optimizer, like CPLEX, while the remaining 15 instances from NGINX are addressed only by MOEAs. The two benchmarks, involving the number of assets from 31 to 2235, consistently indicate that CCS is not only efficient but also robust for dealing with the constrained multi-objective portfolio optimization.
Learning and Segmenting Dense Voxel Embeddings for 3D Neuron Reconstruction
Venue: arXiv cs.CV 2021
Year: 2021

We show dense voxel embeddings learned via deep metric learning can be employed to produce a highly accurate segmentation of neurons from 3D electron microscopy images. A "metric graph" on a set of edges between voxels is constructed from the dense voxel embeddings generated by a convolutional network. Partitioning the metric graph with long-range edges as repulsive constraints yields an initial segmentation with high precision, with substantial accuracy gain for very thin objects. The convolutional embedding net is reused without any modification to agglomerate the systematic splits caused by complex "self-contact" motifs. Our proposed method achieves state-of-the-art accuracy on the challenging problem of 3D neuron reconstruction from the brain images acquired by serial section electron microscopy. Our alternative, object-centered representation could be more generally useful for other computational tasks in automated neural circuit reconstruction.
Deep Reinforcement Learning with Modulated Hebbian plus Q Network Architecture
Venue: arXiv cs.LG 2021
Year: 2021

This paper presents a new neural architecture that combines a modulated Hebbian network (MOHN) with DQN, which we call modulated Hebbian plus Q network architecture (MOHQA). The hypothesis is that such a combination allows MOHQA to solve difficult partially observable Markov decision process (POMDP) problems which impair temporal difference (TD)-based RL algorithms such as DQN, as the TD error cannot be easily derived from observations. The key idea is to use a Hebbian network with bio-inspired neural traces in order to bridge temporal delays between actions and rewards when confounding observations and sparse rewards result in inaccurate TD errors. In MOHQA, DQN learns low level features and control, while the MOHN contributes to the high-level decisions by associating rewards with past states and actions. Thus the proposed architecture combines two modules with significantly different learning algorithms, a Hebbian associative network and a classical DQN pipeline, exploiting the advantages of both. Simulations on a set of POMDPs and on the MALMO environment show that the proposed algorithm improved DQN's results and even outperformed control tests with A2C, QRDQN+LSTM and REINFORCE algorithms on some POMDPs with confounding stimuli and sparse rewards.
Analysis of a Model for Generating Weakly Scale-free Networks
Venue: arXiv cs.DS 2021
Year: 2021

It is commonly believed that real networks are scale-free and fraction of nodes $P(k)$ with degree $k$ satisfies the power law $P(k) \propto k^{-γ} \text{ for } k > k_{min} > 0$. Preferential attachment is the mechanism that has been considered responsible for such organization of these networks. In many real networks, degree distribution before the $k_{min}$ varies very slowly to the extent of being uniform as compared with the degree distribution for $k > k_{min}$ . In this paper, we proposed a model that describe this particular degree distribution for the whole range of $k>0$. We adopt a two step approach. In the first step, at every time stamp we add a new node to the network and attach it with an existing node using preferential attachment method. In the second step, we add edges between existing pairs of nodes with the node selection based on the uniform probability distribution. Our approach generates weakly scale-free networks that closely follow the degree distribution of real-world networks. We perform comprehensive mathematical analysis of the model in the discrete domain and compare the degree distribution generated by these models with that of real-world networks.
Resilient Coordinated Movement of Connected Autonomous Vehicles
Venue: arXiv cs.MA 2021
Year: 2021

In this paper, we consider coordinated movement of a network of vehicles consisting of a bounded number of malicious agents, that is, vehicles must reach consensus in longitudinal position and a common predefined velocity. The motions of vehicles are modeled by double-integrator dynamics and communications over the network are asynchronous with delays. Each normal vehicle updates its states by utilizing the information it receives from vehicles in its vicinity. On the other hand, misbehaving vehicles make updates arbitrarily and might threaten the consensus within the network by intentionally changing their moving direction or broadcasting faulty information in their neighborhood. We propose an asynchronous updating strategy for normal vehicles, based on filtering extreme values received from neighboring vehicles, to save them from being misguided by malicious vehicles. We show that there exist topological constraints on the network in terms of graph robustness under which the vehicles resiliently achieve coordinated movement. Numerical simulations are provided to evaluate the results.
Extremely Small BERT Models from Mixed-Vocabulary Training
Venue: arXiv cs.CL 2021
Year: 2021

Pretrained language models like BERT have achieved good results on NLP tasks, but are impractical on resource-limited devices due to memory footprint. A large fraction of this footprint comes from the input embeddings with large input vocabulary and embedding dimensions. Existing knowledge distillation methods used for model compression cannot be directly applied to train student models with reduced vocabulary sizes. To this end, we propose a distillation method to align the teacher and student embeddings via mixed-vocabulary training. Our method compresses BERT-LARGE to a task-agnostic model with smaller vocabulary and hidden dimensions, which is an order of magnitude smaller than other distilled BERT models and offers a better size-accuracy trade-off on language understanding benchmarks as well as a practical dialogue task.