OpenRaiser/Intern-Atlas huggingface dataset:
Entity Resolution and Federated Learning get a Federated Resolution
Venue: arXiv cs.DB 2018
Year: 2018

Consider two data providers, each maintaining records of different feature sets about common entities. They aim to learn a linear model over the whole set of features. This problem of federated learning over vertically partitioned data includes a crucial upstream issue: entity resolution, i.e. finding the correspondence between the rows of the datasets. It is well known that entity resolution, just like learning, is mistake-prone in the real world. Despite the importance of the problem, there has been no formal assessment of how errors in entity resolution impact learning. In this paper, we provide a thorough answer to this question, answering how optimal classifiers, empirical losses, margins and generalisation abilities are affected. While our answer spans a wide set of losses --- going beyond proper, convex, or classification calibrated ---, it brings simple practical arguments to upgrade entity resolution as a preprocessing step to learning. One of these suggests that entity resolution should be aimed at controlling or minimizing the number of matching errors between examples of distinct classes. In our experiments, we modify a simple token-based entity resolution algorithm so that it indeed aims at avoiding matching rows belonging to different classes, and perform experiments in the setting where entity resolution relies on noisy data, which is very relevant to real world domains. Notably, our approach covers the case where one peer \textit{does not} have classes, or a noisy record of classes. Experiments display that using the class information during entity resolution can buy significant uplift for learning at little expense from the complexity standpoint.
Dedekind Zeta Zeroes and Faster Complex Dimension Computation
Venue: arXiv cs.CC 2018
Year: 2018

Thanks to earlier work of Koiran, it is known that the truth of the Generalized Riemann Hypothesis (GRH) implies that the dimension of algebraic sets over the complex numbers can be determined within the polynomial-hierarchy. The truth of GRH thus provides a direct connection between a concrete algebraic geometry problem and the P vs.NP Problem, in a radically different direction from the geometric complexity theory approach to VP vs. VNP. We explore more plausible hypotheses yielding the same speed-up. One minimalist hypothesis we derive involves improving the error term (as a function of the degree, coefficient height, and $x$) on the fraction of primes $p\!\leq\!x$ for which a univariate polynomial has roots mod $p$. A second minimalist hypothesis involves sharpening current zero-free regions for Dedekind zeta functions. Both our hypotheses allow failures of GRH but still enable complex dimension computation in the polynomial hierarchy.
Innovative Texture Database Collecting Approach and Feature Extraction Method based on Combination of Gray Tone Difference Matrixes, Local Binary Patterns,and K-means Clustering
Venue: arXiv cs.CV 2018
Year: 2018

Texture analysis and classification are some of the problems which have been paid much attention by image processing scientists since late 80s. If texture analysis is done accurately, it can be used in many cases such as object tracking, visual pattern recognition, and face recognition.Since now, so many methods are offered to solve this problem. Against their technical differences, all of them used same popular databases to evaluate their performance such asBrodatz or Outex, which may be made their performance biased on these databases. In this paper, an approach is proposed to collect more efficient databases of texture images. The proposed approach is included two stages. The first one is developing feature representation based on gray tone difference matrixes and local binary patterns features and the next one is consisted an innovative algorithm which is based on K-means clustering to collect images based on evaluated features. In order to evaluate the performance of the proposed approach, a texture database is collected and fisher rate is computed for collected one and well known databases. Also, texture classification is evaluated based on offered feature extraction and the accuracy is compared by some state of the art texture classification methods.
Deep Class-Wise Hashing: Semantics-Preserving Hashing via Class-wise Loss
Venue: arXiv cs.CV 2018
Year: 2018

Deep supervised hashing has emerged as an influential solution to large-scale semantic image retrieval problems in computer vision. In the light of recent progress, convolutional neural network based hashing methods typically seek pair-wise or triplet labels to conduct the similarity preserving learning. However, complex semantic concepts of visual contents are hard to capture by similar/dissimilar labels, which limits the retrieval performance. Generally, pair-wise or triplet losses not only suffer from expensive training costs but also lack in extracting sufficient semantic information. In this regard, we propose a novel deep supervised hashing model to learn more compact class-level similarity preserving binary codes. Our deep learning based model is motivated by deep metric learning that directly takes semantic labels as supervised information in training and generates corresponding discriminant hashing code. Specifically, a novel cubic constraint loss function based on Gaussian distribution is proposed, which preserves semantic variations while penalizes the overlap part of different classes in the embedding space. To address the discrete optimization problem introduced by binary codes, a two-step optimization strategy is proposed to provide efficient training and avoid the problem of gradient vanishing. Extensive experiments on four large-scale benchmark databases show that our model can achieve the state-of-the-art retrieval performance. Moreover, when training samples are limited, our method surpasses other supervised deep hashing methods with non-negligible margins.
Standardization of work: co-constructed practice
Venue: arXiv cs.CY 2018
Year: 2018

There is strong pressure to achieve greater uniformity, standardisation and application of best practices in the service professions, a sector which is growing in presence and importance. At the same time, there is a conflicting demand for the delivery of high quality or highly priced or knowledge intensive specialised or localised services. Our paper analyses information systems embedded efforts of standardising service work through an indepth interpretative study of an ongoing standardisation initiative within the field of nursing. Nursing provides a graphic illustration of the dilemmas involved in the standardisation of service work. In nursing, standardisation is commonly a feature of projects to improve both efficiency and quality in health care. In contrast to the dominant conception of standardisation as a largely topdown, imposed process, we offer a view of standardisation as incomplete, co-constructed with users and with significant unintended consequences. The paper contributes by developing a theoretical perspective for the standardisation of information-system-embedded service work and operational and practical implications for system design and health care management.
Social software and strategy
Venue: arXiv cs.CY 2018
Year: 2018

Aligning interests, motivating contributions to knowledge work, and giving direction to multiple business units and market initiatives represent daily challenges facing the strategist in most companies. The diversity of contexts within an organization has thus led critical management thinkers to suggest that in the presence of multiple initiatives and discourses, the introduction of a new technology has multiple unintended consequences for organization. New technology is regularly subject to power struggles, conflicting goals, and discrepant events (Markus, 1983; Barley, 1986; Orlikowski, 1992; Ciborra, 1996; Leonardi, 2008) which impact on how strategies are shaped within organizations.
Addressing the Free-Rider Problem in Public Transport Systems
Venue: arXiv cs.CY 2018
Year: 2018

Public transport network constitutes for an indispensable part of a city by providing mobility services to the general masses. To improve ease of access and reduce infrastructural investments, public transport authorities often adopt proof of payment system. Such a system operates by eliminating ticket controls when boarding the vehicle and subjecting the travelers to random ticket checks by affiliated personnel (controllers). Although cost efficient, such a system promotes free-riders, who deliberately decide to evade fares for the transport service. A recent survey by the association of European transport, estimates hefty income losses due to fare evasion, highlighting that free-riding is a serious problem that needs immediate attention. To this end, we highlight the attack vectors which can be exploited by free-riders by analyzing the crowdsourced data about the control-locations. Next, we propose a framework to generate randomized control-location traces by using generative adversarial networks (GANs) in order to minimize the attack vectors. Finally, we propose metrics to evaluate such a system, quantified in terms of increased risk and higher probability of being subjected to control checks across the city.
Dfuntest: A Testing Framework for Distributed Applications
Venue: arXiv cs.DC 2018
Year: 2018

New ideas in distributed systems (algorithms or protocols) are commonly tested by simulation, because experimenting with a prototype deployed on a realistic platform is cumbersome. However, a prototype not only measures performance but also verifies assumptions about the underlying system. We developed dfuntest - a testing framework for distributed applications that defines abstractions and test structure, and automates experiments on distributed platforms. Dfuntest aims to be jUnit's analogue for distributed applications; a framework that enables the programmer to write robust and flexible scenarios of experiments. Dfuntest requires minimal bindings that specify how to deploy and interact with the application. Dfuntest's abstractions allow execution of a scenario on a single machine, a cluster, a cloud, or any other distributed infrastructure, e.g. on PlanetLab. A scenario is a procedure; thus, our framework can be used both for functional tests and for performance measurements. We show how to use dfuntest to deploy our DHT prototype on 60 PlanetLab nodes and verify whether the prototype maintains a correct topology.
Automated software vulnerability detection with machine learning
Venue: arXiv cs.SE 2018
Year: 2018

Thousands of security vulnerabilities are discovered in production software each year, either reported publicly to the Common Vulnerabilities and Exposures database or discovered internally in proprietary code. Vulnerabilities often manifest themselves in subtle ways that are not obvious to code reviewers or the developers themselves. With the wealth of open source code available for analysis, there is an opportunity to learn the patterns of bugs that can lead to security vulnerabilities directly from data. In this paper, we present a data-driven approach to vulnerability detection using machine learning, specifically applied to C and C++ programs. We first compile a large dataset of hundreds of thousands of open-source functions labeled with the outputs of a static analyzer. We then compare methods applied directly to source code with methods applied to artifacts extracted from the build process, finding that source-based models perform better. We also compare the application of deep neural network models with more traditional models such as random forests and find the best performance comes from combining features learned by deep models with tree-based models. Ultimately, our highest performing model achieves an area under the precision-recall curve of 0.49 and an area under the ROC curve of 0.87.
Massive Multiple Input Massive Multiple Output for 5G Wireless Backhauling
Venue: arXiv cs.IT 2018
Year: 2018

In this paper, we propose a new technique for the future fifth generation cellular network wireless backhauling. We show that hundreds of bits per second per Hertz (bits per second per Hz) of spectral efficiency can be attained at a high carrier frequency (such as 26 GHz) between large antenna arrays deployed along structures (such as lamp posts) that are close and roughly parallel to each other. Hundreds of data streams are spatially multiplexed through a short range and line of sight massive multiple input massive multiple output propagation channel thanks to a new low complexity spatial multiplexing scheme, called block discrete Fourier transform based spatial multiplexing with maximum ratio transmission. Its performance in real and existing environments is assessed using accurate ray-tracing tools and antenna models. In the best simulated scenario, 1.6 kbits per second per Hz of spectral efficiency is attained, corresponding to 80% of Singular Value Decomposition performance, with a transmitter and a receiver that are 200 and 10,000 times less complex, respectively.
Bias in OLAP Queries: Detection, Explanation, and Removal
Venue: arXiv cs.DB 2018
Year: 2018

On line analytical processing (OLAP) is an essential element of decision-support systems. OLAP tools provide insights and understanding needed for improved decision making. However, the answers to OLAP queries can be biased and lead to perplexing and incorrect insights. In this paper, we propose HypDB, a system to detect, explain, and to resolve bias in decision-support queries. We give a simple definition of a \emph{biased query}, which performs a set of independence tests on the data to detect bias. We propose a novel technique that gives explanations for bias, thus assisting an analyst in understanding what goes on. Additionally, we develop an automated method for rewriting a biased query into an unbiased query, which shows what the analyst intended to examine. In a thorough evaluation on several real datasets we show both the quality and the performance of our techniques, including the completely automatic discovery of the revolutionary insights from a famous 1973 discrimination case.
Building Better Quality Predictors Using "$ε$-Dominance"
Venue: arXiv cs.SE 2018
Year: 2018

Despite extensive research, many methods in software quality prediction still exhibit some degree of uncertainty in their results. Rather than treating this as a problem, this paper asks if this uncertainty is a resource that can simplify software quality prediction. For example, Deb's principle of $ε$-dominance states that if there exists some $ε$ value below which it is useless or impossible to distinguish results, then it is superfluous to explore anything less than $ε$. We say that for "large $ε$ problems", the results space of learning effectively contains just a few regions. If many learners are then applied to such large $ε$ problems, they would exhibit a "many roads lead to Rome" property; i.e., many different software quality prediction methods would generate a small set of very similar results. This paper explores DART, an algorithm especially selected to succeed for large $ε$ software quality prediction problems. DART is remarkable simple yet, on experimentation, it dramatically out-performs three sets of state-of-the-art defect prediction methods. The success of DART for defect prediction begs the questions: how many other domains in software quality predictors can also be radically simplified? This will be a fruitful direction for future work.
A New Simulation Algorithm for Absorbing Receiver in Molecular Communication
Venue: arXiv cs.ET 2018
Year: 2018

The simulation of diffusion-based molecular communication systems with absorbing receivers often requires a high computational complexity to produce accurate results. In this work, a new a priori Monte Carlo (APMC) algorithm is proposed to precisely simulate the molecules absorbed at a spherical receiver when the simulation time step length is relatively large. This algorithm addresses the limitations of the current refined Monte Carlo (RMC) algorithm, since the RMC algorithm provides accurate simulation only for a relatively small time step length. The APMC algorithm is demonstrated to achieve a higher simulation efficiency than the existing algorithms by finding that the APMC algorithm, for a relatively large time step length, absorbs the fraction of molecules expected by analysis, while other algorithms do not.
Hierarchical Reinforcement Learning: Approximating Optimal Discounted TSP Using Local Policies
Venue: arXiv cs.LG 2018
Year: 2018

In this work, we provide theoretical guarantees for reward decomposition in deterministic MDPs. Reward decomposition is a special case of Hierarchical Reinforcement Learning, that allows one to learn many policies in parallel and combine them into a composite solution. Our approach builds on mapping this problem into a Reward Discounted Traveling Salesman Problem, and then deriving approximate solutions for it. In particular, we focus on approximate solutions that are local, i.e., solutions that only observe information about the current state. Local policies are easy to implement and do not require substantial computational resources as they do not perform planning. While local deterministic policies, like Nearest Neighbor, are being used in practice for hierarchical reinforcement learning, we propose three stochastic policies that guarantee better performance than any deterministic policy.
VERSE: Versatile Graph Embeddings from Similarity Measures
Venue: arXiv cs.SI 2018
Year: 2018

Embedding a web-scale information network into a low-dimensional vector space facilitates tasks such as link prediction, classification, and visualization. Past research has addressed the problem of extracting such embeddings by adopting methods from words to graphs, without defining a clearly comprehensible graph-related objective. Yet, as we show, the objectives used in past works implicitly utilize similarity measures among graph nodes. In this paper, we carry the similarity orientation of previous works to its logical conclusion; we propose VERtex Similarity Embeddings (VERSE), a simple, versatile, and memory-efficient method that derives graph embeddings explicitly calibrated to preserve the distributions of a selected vertex-to-vertex similarity measure. VERSE learns such embeddings by training a single-layer neural network. While its default, scalable version does so via sampling similarity information, we also develop a variant using the full information per vertex. Our experimental study on standard benchmarks and real-world datasets demonstrates that VERSE, instantiated with diverse similarity measures, outperforms state-of-the-art methods in terms of precision and recall in major data mining tasks and supersedes them in time and space efficiency, while the scalable sampling-based variant achieves equally good results as the non-scalable full variant.
Hybrid Forecasting of Chaotic Processes: Using Machine Learning in Conjunction with a Knowledge-Based Model
Venue: arXiv cs.LG 2018
Year: 2018

A model-based approach to forecasting chaotic dynamical systems utilizes knowledge of the physical processes governing the dynamics to build an approximate mathematical model of the system. In contrast, machine learning techniques have demonstrated promising results for forecasting chaotic systems purely from past time series measurements of system state variables (training data), without prior knowledge of the system dynamics. The motivation for this paper is the potential of machine learning for filling in the gaps in our underlying mechanistic knowledge that cause widely-used knowledge-based models to be inaccurate. Thus we here propose a general method that leverages the advantages of these two approaches by combining a knowledge-based model and a machine learning technique to build a hybrid forecasting scheme. Potential applications for such an approach are numerous (e.g., improving weather forecasting). We demonstrate and test the utility of this approach using a particular illustrative version of a machine learning known as reservoir computing, and we apply the resulting hybrid forecaster to a low-dimensional chaotic system, as well as to a high-dimensional spatiotemporal chaotic system. These tests yield extremely promising results in that our hybrid technique is able to accurately predict for a much longer period of time than either its machine-learning component or its model-based component alone.
Artificial neural network based modelling approach for municipal solid waste gasification in a fluidized bed reactor
Venue: arXiv cs.LG 2018
Year: 2018

In this paper, multi-layer feed forward neural networks are used to predict the lower heating value of gas (LHV), lower heating value of gasification products including tars and entrained char (LHVp) and syngas yield during gasification of municipal solid waste (MSW) during gasification in a fluidized bed reactor. These artificial neural networks (ANNs) with different architectures are trained using the Levenberg-Marquardt (LM) back-propagation algorithm and a cross validation is also performed to ensure that the results generalise to other unseen datasets. A rigorous study is carried out on optimally choosing the number of hidden layers, number of neurons in the hidden layer and activation function in a network using multiple Monte Carlo runs. Nine input and three output parameters are used to train and test various neural network architectures in both multiple output and single output prediction paradigms using the available experimental datasets. The model selection procedure is carried out to ascertain the best network architecture in terms of predictive accuracy. The simulation results show that the ANN based methodology is a viable alternative which can be used to predict the performance of a fluidized bed gasifier.
A System for the Generation of Synthetic Wide Area Aerial Surveillance Imagery
Venue: arXiv cs.OH 2018
Year: 2018

The development, benchmarking and validation of aerial Persistent Surveillance (PS) algorithms requires access to specialist Wide Area Aerial Surveillance (WAAS) datasets. Such datasets are difficult to obtain and are often extremely large both in spatial resolution and temporal duration. This paper outlines an approach to the simulation of complex urban environments and demonstrates the viability of using this approach for the generation of simulated sensor data, corresponding to the use of wide area imaging systems for surveillance and reconnaissance applications. This provides a cost-effective method to generate datasets for vehicle tracking algorithms and anomaly detection methods. The system fuses the Simulation of Urban Mobility (SUMO) traffic simulator with a MATLAB controller and an image generator to create scenes containing uninterrupted door-to-door journeys across large areas of the urban environment. This `pattern-of-life' approach provides three-dimensional visual information with natural movement and traffic flows. This can then be used to provide simulated sensor measurements (e.g. visual band and infrared video imagery) and automatic access to ground-truth data for the evaluation of multi-target tracking systems.