OpenRaiser/Intern-Atlas huggingface dataset:
Private Inner Product Retrieval for Distributed Machine Learning
Venue: arXiv cs.IT 2019
Year: 2019

In this paper, we argue that in many basic algorithms for machine learning, including support vector machine (SVM) for classification, principal component analysis (PCA) for dimensionality reduction, and regression for dependency estimation, we need the inner products of the data samples, rather than the data samples themselves. Motivated by the above observation, we introduce the problem of private inner product retrieval for distributed machine learning, where we have a system including a database of some files, duplicated across some non-colluding servers. A user intends to retrieve a subset of specific size of the inner products of the data files with minimum communication load, without revealing any information about the identity of the requested subset. For achievability, we use the algorithms for multi-message private information retrieval. For converse, we establish that as the length of the files becomes large, the set of all inner products converges to independent random variables with uniform distribution, and derive the rate of convergence. To prove that, we construct special dependencies among sequences of the sets of all inner products with different length, which forms a time-homogeneous irreducible Markov chain, without affecting the marginal distribution. We show that this Markov chain has a uniform distribution as its unique stationary distribution, with rate of convergence dominated by the second largest eigenvalue of the transition probability matrix. This allows us to develop a converse, which converges to a tight bound in some cases, as the size of the files becomes large. While this converse is based on the one in multi-message private information retrieval, due to the nature of retrieving inner products instead of data itself some changes are made to reach the desired result.
Quantized Frank-Wolfe: Faster Optimization, Lower Communication, and Projection Free
Venue: arXiv cs.LG 2019
Year: 2019

How can we efficiently mitigate the overhead of gradient communications in distributed optimization? This problem is at the heart of training scalable machine learning models and has been mainly studied in the unconstrained setting. In this paper, we propose Quantized-Frank-Wolfe (QFW), the first projection-free and communication-efficient algorithm for solving constrained optimization problems at scale. We consider both convex and non-convex objective functions, expressed as a finite-sum or more generally a stochastic optimization problem, and provide strong theoretical guarantees on the convergence rate of QFW. This is accomplished by proposing novel quantization schemes that efficiently compress gradients while controlling the noise variance introduced during this process. Finally, we empirically validate the efficiency of QFW in terms of communication and the quality of returned solution against natural baselines.
Detecting and Diagnosing Incipient Building Faults Using Uncertainty Information from Deep Neural Networks
Venue: arXiv cs.LG 2019
Year: 2019

Early detection of incipient faults is of vital importance to reducing maintenance costs, saving energy, and enhancing occupant comfort in buildings. Popular supervised learning models such as deep neural networks are considered promising due to their ability to directly learn from labeled fault data; however, it is known that the performance of supervised learning approaches highly relies on the availability and quality of labeled training data. In Fault Detection and Diagnosis (FDD) applications, the lack of labeled incipient fault data has posed a major challenge to applying these supervised learning techniques to commercial buildings. To overcome this challenge, this paper proposes using Monte Carlo dropout (MC-dropout) to enhance the supervised learning pipeline, so that the resulting neural network is able to detect and diagnose unseen incipient fault examples. We also examine the proposed MC-dropout method on the RP-1043 dataset to demonstrate its effectiveness in indicating the most likely incipient fault types.
Topics of Concern: Identifying User Issues in Reviews of IoT Apps and Devices
Venue: arXiv cs.HC 2019
Year: 2019

Internet of Things (IoT) systems are bundles of networked sensors and actuators that are deployed in an environment and act upon the sensory data that they receive. These systems, especially consumer electronics, have two main cooperating components: a device and a mobile app. The unique combination of hardware and software in IoT systems presents challenges that are lesser known to mainstream software developers. They might require innovative solutions to support the development and integration of such systems. In this paper, we analyze more than 90,000 reviews of ten IoT devices and their corresponding apps and extract the issues that users encountered while using these systems. Our results indicate that issues with connectivity, timing, and updates are particularly prevalent in the reviews. Our results call for a new software-hardware development framework to assist the development of reliable IoT systems.
Asymptotic Limits of Privacy in Bayesian Time Series Matching
Venue: arXiv cs.IT 2019
Year: 2019

Various modern and highly popular applications make use of user data traces in order to offer specific services, often for the purpose of improving the user's experience while using such applications. However, even when user data is privatized by employing privacy-preserving mechanisms (PPM), users' privacy may still be compromised by an external party who leverages statistical matching methods to match users' traces with their previous activities. In this paper, we obtain the theoretical bounds on user privacy for situations in which user traces are matchable to sequences of prior behavior, despite anonymization of data time series. We provide both achievability and converse results for the case where the data trace of each user consists of independent and identically distributed (i.i.d.) random samples drawn from a multinomial distribution, as well as the case that the users' data points are dependent over time and the data trace of each user is governed by a Markov chain model.
Reactive, Proactive, and Inductive Agents: An evolutionary path for biological and artificial spiking networks
Venue: arXiv cs.NE 2019
Year: 2019

Complex environments provide structured yet variable sensory inputs. To best exploit information from these environments, organisms must evolve the ability to anticipate consequences of unknown stimuli, and act on these predictions. We propose an evolutionary path for neural networks, leading an organism from reactive behavior to simple proactive behavior and from simple proactive behavior to induction-based behavior. Through in-vitro and in-silico experiments, we define the conditions necessary in a network with spike-timing dependent plasticity for the organism to go from reactive to proactive behavior. Our results support the existence of specific evolutionary steps and four conditions necessary for embodied neural networks to evolve predictive and inductive abilities from an initial reactive strategy. We extend these conditions to more general structures.
CBOW Is Not All You Need: Combining CBOW with the Compositional Matrix Space Model
Venue: arXiv cs.CL 2019
Year: 2019

Continuous Bag of Words (CBOW) is a powerful text embedding method. Due to its strong capabilities to encode word content, CBOW embeddings perform well on a wide range of downstream tasks while being efficient to compute. However, CBOW is not capable of capturing the word order. The reason is that the computation of CBOW's word embeddings is commutative, i.e., embeddings of XYZ and ZYX are the same. In order to address this shortcoming, we propose a learning algorithm for the Continuous Matrix Space Model, which we call Continual Multiplication of Words (CMOW). Our algorithm is an adaptation of word2vec, so that it can be trained on large quantities of unlabeled text. We empirically show that CMOW better captures linguistic properties, but it is inferior to CBOW in memorizing word content. Motivated by these findings, we propose a hybrid model that combines the strengths of CBOW and CMOW. Our results show that the hybrid CBOW-CMOW-model retains CBOW's strong ability to memorize word content while at the same time substantially improving its ability to encode other linguistic information by 8%. As a result, the hybrid also performs better on 8 out of 11 supervised downstream tasks with an average improvement of 1.2%.
Experimental Demonstration of Real-time PDCP-RLC V-RAN Split Transmission over Fixed XGS-PON access
Venue: arXiv cs.NI 2019
Year: 2019

In this work, we experimentally assess the transmission of a PDCP-RLC virtualised RAN split interface through a commercial XGS-PON system. We investigate the impacts of DBA on the uplink and packet jitter on the downlink.
Information-theoretic lower bounds for quantum sorting
Venue: arXiv cs.CC 2019
Year: 2019

We analyze the quantum query complexity of sorting under partial information. In this problem, we are given a partially ordered set $P$ and are asked to identify a linear extension of $P$ using pairwise comparisons. For the standard sorting problem, in which $P$ is empty, it is known that the quantum query complexity is not asymptotically smaller than the classical information-theoretic lower bound. We prove that this holds for a wide class of partially ordered sets, thereby improving on a result from Yao (STOC'04).
Designing recurrent neural networks by unfolding an l1-l1 minimization algorithm
Venue: arXiv cs.LG 2019
Year: 2019

We propose a new deep recurrent neural network (RNN) architecture for sequential signal reconstruction. Our network is designed by unfolding the iterations of the proximal gradient method that solves the l1-l1 minimization problem. As such, our network leverages by design that signals have a sparse representation and that the difference between consecutive signal representations is also sparse. We evaluate the proposed model in the task of reconstructing video frames from compressive measurements and show that it outperforms several state-of-the-art RNN models.
Network connectivity dynamics affect the evolution of culturally transmitted variants
Venue: arXiv cs.SI 2019
Year: 2019

The distribution of cultural variants in a population is shaped by both neutral evolutionary dynamics and by selection pressures, which include several individual cognitive biases, demographic factors and social network structures. The temporal dynamics of social network connectivity, i.e. the order in which individuals in a population interact with each other, has been largely unexplored. In this paper we investigate how, in a fully connected social network, connectivity dynamics, alone and in interaction with different cognitive biases, affect the evolution of cultural variants. Using agent-based computer simulations, we manipulate population connectivity dynamics (early, middle and late full-population connectivity); content bias, or a preference for high-quality variants; coordination bias, or whether agents tend to use self-produced variants (egocentric bias), or to switch to variants observed in others (allocentric bias); and memory size, or the number of items that agents can store in their memory. We show that connectivity dynamics affect the time-course of variant spread, with lower connectivity slowing down convergence of the population onto a single cultural variant. We also show that, compared to a neutral evolutionary model, content bias accelerates convergence and amplifies the effects of connectivity dynamics, whilst larger memory size and coordination bias, especially egocentric bias, slow down convergence. Furthermore, connectivity dynamics affect the frequency of high quality variants (adaptiveness), with late connectivity populations showing bursts of rapid change in adaptiveness followed by periods of relatively slower change, and early connectivity populations following a single-peak evolutionary dynamic. In this way, we provide for the first time a direct connection between the order of agents' interactions and punctuational evolution.
An integrated model predictive control approach for optimal HVAC and energy storage operation in large-scale buildings
Venue: arXiv cs.SY 2019
Year: 2019

This paper deals with the problem of cost-optimal operation of smart buildings that integrate a centralized HVAC system, photovoltaic generation and both thermal and electrical storage devices. Building participation in a Demand-Response program is also considered. The proposed solution is based on a specialized Model Predictive Control strategy to optimally manage the HVAC system and the storage devices under thermal comfort and technological constraints. The related optimization problems turn out to be computationally appealing, even for large-scale problem instances. Performance evaluation, also in the presence of uncertainties and disturbances, is carried out using a realistic simulation framework.
ENBB Processor: Towards the ExaScale Numerical Brain Box [Position Paper]
Venue: arXiv cs.AR 2019
Year: 2019

ExaScale systems will be a key driver for simulations that are essential for advance of science and economic growth. We aim to present a new concept of microprocessor for floating-point computations useful for being a basic building block of ExaScale systems and beyond. The proposed microprocessor architecture has a frontend for programming interface based on the concept of event-driven simulation. The user program is executed as an event-driven simulation using a hardware/software co-designed simulator. This is the flexible part of the system. The back-end exploits the concept of uniform topology as in a brain: a massive packet switched interconnection network with flit credit-based flow control with virtual channels that incorporates seamlessly communication, arithmetic and storage. Floating-point computations are incorporated as on-line arithmetic operators in the output ports of the switches as virtual arithmetic output channels, and storage as virtual input channels. The front-end carries out the event-driven simulation of the user program, and uses the arithmetic network for the hard floating-point work by means of virtual dataflows. We expect to reduce significantly the needs of main memory due to the execution model proposed, where variables are just virtual interconnections in the network or signals stored in the virtual channels. Moreover, we have the hypothesis that the problem size assigned to a microprocessor should allow maximum concurrency and it should not be oversized. This may lead to systems composed of microprocessors with main memory incorporated in 3D chips. We identified several challenges that a research to develop this microprocessor should address, and several hypothesis that should be demonstrated by means of scientific evidence.
A Blockchain-enabled Trustless Crowd-Intelligence Ecosystem on Mobile Edge Computing
Venue: arXiv cs.CY 2019
Year: 2019

Crowd-intelligence tries to gather, process, infer and ascertain massive useful information by utilizing the intelligence of crowds or distributed computers, which has great potential in Industrial Internet of Things (IIoT). A crowd-intelligence ecosystem involves three stakeholders, namely the platform, workers (e.g., individuals, sensors or processors), and task publisher. The stakeholders have no mutual trust but interest conflict, which means bad cooperation of them. Due to lack of trust, transferring raw data (e.g., pictures or video clips) between publisher and workers requires the remote platform center to serve as a relay node, which implies network congestion. First we use a reward-penalty model to align the incentives of stakeholders. Then the predefined rules are implemented using blockchain smart contract on many edge servers of the mobile edge computing network, which together function as a trustless hybrid human-machine crowd-intelligence platform. As edge servers are near to workers and publisher, network congestion can be effectively improved. Further, we proved the existence of the only one strong Nash equilibrium, which can maximize the interests of involved edge servers and make the ecosystem bigger. Theoretical analysis and experiments validate the proposed method respectively.
VEDAR: Accountable Behavioural Change Detection
Venue: arXiv cs.SI 2019
Year: 2019

With exponential increase in the availability oftelemetry / streaming / real-time data, understanding contextualbehavior changes is a vital functionality in order to deliverunrivalled customer experience and build high performance andhigh availability systems. Real-time behavior change detectionfinds a use case in number of domains such as social networks,network traffic monitoring, ad exchange metrics etc. In streamingdata, behavior change is an implausible observation that does notfit in with the distribution of rest of the data. A timely and preciserevelation of such behavior changes can give us substantialinformation about the system in critical situations which can bea driving factor for vital decisions. Detecting behavior changes instreaming fashion is a difficult task as the system needs to processhigh speed real-time data and continuously learn from data alongwith detecting anomalies in a single pass of data. In this paperwe introduce a novel algorithm called Accountable BehaviorChange Detection (VEDAR) which can detect and elucidate thebehavior changes in real-time and operates in a fashion similarto human perception. We have bench marked our algorithmon open source anomaly detection datasets. We have benchmarked our algorithm by comparing its performance on opensource anomaly datasets against industry standard algorithmslike Numenta HTM and Twitter AdVec (SH-ESD). Our algorithmoutperforms above mentioned algorithms for behaviour changedetection, efficacy is given in section V.
Learning Topological Representation for Networks via Hierarchical Sampling
Venue: arXiv cs.SI 2019
Year: 2019

The topological information is essential for studying the relationship between nodes in a network. Recently, Network Representation Learning (NRL), which projects a network into a low-dimensional vector space, has been shown their advantages in analyzing large-scale networks. However, most existing NRL methods are designed to preserve the local topology of a network, they fail to capture the global topology. To tackle this issue, we propose a new NRL framework, named HSRL, to help existing NRL methods capture both the local and global topological information of a network. Specifically, HSRL recursively compresses an input network into a series of smaller networks using a community-awareness compressing strategy. Then, an existing NRL method is used to learn node embeddings for each compressed network. Finally, the node embeddings of the input network are obtained by concatenating the node embeddings from all compressed networks. Empirical studies for link prediction on five real-world datasets demonstrate the advantages of HSRL over state-of-the-art methods.
Openbots
Venue: arXiv cs.CY 2019
Year: 2019

Social bots have recently gained attention in the context of public opinion manipulation on social media platforms. While a lot of research effort has been put into the classification and detection of such (semi-)automated programs, it is still unclear how sophisticated those bots actually are, which platforms they target, and where they originate from. To answer these questions, we gathered repository data from open source collaboration platforms to identify the status-quo as well as trends of publicly available bot code. Our findings indicate that most of the code on collaboration platforms is of supportive nature and provides modules of automation instead of fully fledged social bot programs. Hence, the cost (in terms of additional programming effort) for building social bots with the goal of topic-specific manipulation is higher than assumed and that methods in context of machine- or deep-learning currently only play a minor role. However, our approach can be applied as multifaceted knowledge discovery framework to monitor trends in public bot code evolution to detect new developments and streams.
Analysis of Coauthorship Network in Political Science using Centrality Measures
Venue: arXiv cs.SI 2019
Year: 2019

In recent era, networks of data are growing massively and forming a shape of complex structure. Data scientists try to analyze different complex networks and utilize these networks to understand the complex structure of a network in a meaningful way. There is a need to detect and identify such a complex network in order to know how these networks provide communication means while using the complex structure. Social network analysis provides methods to explore and analyze such complex networks using graph theories, network properties and community detection algorithms. In this paper, an analysis of coauthorship network of Public Relation and Public Administration subjects of Microsoft Academic Graph (MAG) is presented, using common centrality measures. The authors belong to different research and academic institutes present all over the world. Cohesive groups of authors have been identified and ranked on the basis of centrality measures, such as betweenness, degree, page rank and closeness. Experimental results show the discovery of authors who are good in specific domain, have a strong field knowledge and maintain collaboration among their peers in the field of Public Relations and Public Administration.
SFCM-R: A novel algorithm for the hamiltonian sequence problem
Venue: arXiv cs.DS 2019
Year: 2019

A hamiltonian sequence is a path walk $P$ that can be a hamiltonian path or hamiltonian circuit. Determining whether such hamiltonian sequence exists in a given graph \G is a NP-Complete problem. In this paper, a novel algorithm for hamiltonian sequence problem is proposed. The proposed algorithm assumes that $G$ has potential forbidden minors that prevent a potential hamiltonian sequence $P^\prime$ from being a hamiltonian sequence. The algorithm's goal is to degenerate such potential forbidden minors in a two-phrase process. In first phrase, the algorithm passes through $G$ in order to construct a potential hamiltonian sequence $P^\prime$ with the aim of degenerating these potential forbidden minors. The algorithm, in turn, tries to reconstruct $P^\prime$ in second phrase by using a goal-oriented approach.
On the Performance of NOMA-based Cooperative Relaying with Receive Diversity
Venue: arXiv cs.IT 2019
Year: 2019

Non-orthogonal multiple access (NOMA) is widely recognized as a potential multiple access (MA) technology for efficient spectrum utilization in the fifth-generation (5G) wireless standard. In this paper, we present the achievable sum rate analysis of a cooperative relaying system (CRS) using NOMA with two different receive diversity schemes - selection combining (SC), where the antenna with highest instantaneous signal-to-noise ratio (SNR) is selected, and maximal-ratio combining (MRC). We also present the outage probability and diversity analysis for the CRS-NOMA system. Analytical results confirm that the CRS-NOMA system outperforms the CRS with conventional orthogonal multiple access (OMA) by achieving higher spectral efficiency at high transmit SNR and achieves a full diversity order.