OpenRaiser/Intern-Atlas huggingface dataset:
On The Communication Complexity of Linear Algebraic Problems in the Message Passing Model
Venue: arXiv cs.CC 2014
Year: 2014

We study the communication complexity of linear algebraic problems over finite fields in the multi-player message passing model, proving a number of tight lower bounds. Specifically, for a matrix which is distributed among a number of players, we consider the problem of determining its rank, of computing entries in its inverse, and of solving linear equations. We also consider related problems such as computing the generalized inner product of vectors held on different servers. We give a general framework for reducing these multi-player problems to their two-player counterparts, showing that the randomized $s$-player communication complexity of these problems is at least $s$ times the randomized two-player communication complexity. Provided the problem has a certain amount of algebraic symmetry, which we formally define, we can show the hardest input distribution is a symmetric distribution, and therefore apply a recent multi-player lower bound technique of Phillips et al. Further, we give new two-player lower bounds for a number of these problems. In particular, our optimal lower bound for the two-player version of the matrix rank problem resolves an open question of Sun and Wang. A common feature of our lower bounds is that they apply even to the special "threshold promise" versions of these problems, wherein the underlying quantity, e.g., rank, is promised to be one of just two values, one on each side of some critical threshold. These kinds of promise problems are commonplace in the literature on data streaming as sources of hardness for reductions giving space lower bounds.
ADHA: Automatic Data layout framework for Heterogeneous Architectures
Venue: arXiv cs.DC 2014
Year: 2014

Data layouts play a crucial role in determining the performance of a given application running on a given architecture. Existing parallel programming frameworks for both multicore and heterogeneous systems leave the onus of selecting a data layout to the programmer. Therefore, shifting the burden of data layout selection to optimizing compilers can greatly enhance programmer productivity and application performance. In this work, we introduce {\ADHA}: a two-level hierarchal formulation of the data layout problem for modern heterogeneous architectures. We have created a reference implementation of ADHA in the Heterogeneous Habanero-C (H2C) parallel programming system. ADHA shows significant performance benefits of up to 6.92$\times$ compared to manually specified layouts for two benchmark programs running on a CPU+GPU heterogeneous platform.
D4D-Senegal: The Second Mobile Phone Data for Development Challenge
Venue: arXiv cs.CY 2014
Year: 2014

The D4D-Senegal challenge is an open innovation data challenge on anonymous call patterns of Orange's mobile phone users in Senegal. The goal of the challenge is to help address society development questions in novel ways by contributing to the socio-economic development and well-being of the Senegalese population. Participants to the challenge are given access to three mobile phone datasets. This paper describes the three datasets. The datasets are based on Call Detail Records (CDR) of phone calls and text exchanges between more than 9 million of Orange's customers in Senegal between January 1, 2013 to December 31, 2013. The datasets are: (1) antenna-to-antenna traffic for 1666 antennas on an hourly basis, (2) fine-grained mobility data on a rolling 2-week basis for a year with bandicoot behavioral indicators at individual level for about 300,000 randomly sampled users, (3) one year of coarse-grained mobility data at arrondissement level with bandicoot behavioral indicators at individual level for about 150,000 randomly sampled users
Redundancy of Exchangeable Estimators
Venue: arXiv cs.IT 2014
Year: 2014

Exchangeable random partition processes are the basis for Bayesian approaches to statistical inference in large alphabet settings. On the other hand, the notion of the pattern of a sequence provides an information-theoretic framework for data compression in large alphabet scenarios. Because data compression and parameter estimation are intimately related, we study the redundancy of Bayes estimators coming from Poisson-Dirichlet priors (or "Chinese restaurant processes") and the Pitman-Yor prior. This provides an understanding of these estimators in the setting of unknown discrete alphabets from the perspective of universal compression. In particular, we identify relations between alphabet sizes and sample sizes where the redundancy is small, thereby characterizing useful regimes for these estimators.
An Empirical Study on Android for Saving Non-shared Data on Public Storage
Venue: arXiv cs.CR 2014
Year: 2014

With millions of apps that can be downloaded from official or third-party market, Android has become one of the most popular mobile platforms today. These apps help people in all kinds of ways and thus have access to lots of user's data that in general fall into three categories: sensitive data, data to be shared with other apps, and non-sensitive data not to be shared with others. For the first and second type of data, Android has provided very good storage models: an app's private sensitive data are saved to its private folder that can only be access by the app itself, and the data to be shared are saved to public storage (either the external SD card or the emulated SD card area on internal FLASH memory). But for the last type, i.e., an app's non-sensitive and non-shared data, there is a big problem in Android's current storage model which essentially encourages an app to save its non-sensitive data to shared public storage that can be accessed by other apps. At first glance, it seems no problem to do so, as those data are non-sensitive after all, but it implicitly assumes that app developers could correctly identify all sensitive data and prevent all possible information leakage from private-but-non-sensitive data. In this paper, we will demonstrate that this is an invalid assumption with a thorough survey on information leaks of those apps that had followed Android's recommended storage model for non-sensitive data. Our studies showed that highly sensitive information from billions of users can be easily hacked by exploiting the mentioned problematic storage model. Although our empirical studies are based on a limited set of apps, the identified problems are never isolated or accidental bugs of those apps being investigated. On the contrary, the problem is rooted from the vulnerable storage model recommended by Android. To mitigate the threat, we also propose a defense framework.
Composing security protocols: from confidentiality to privacy
Venue: arXiv cs.CR 2014
Year: 2014

Security protocols are used in many of our daily-life applications, and our privacy largely depends on their design. Formal verification techniques have proved their usefulness to analyse these protocols, but they become so complex that modular techniques have to be developed. We propose several results to safely compose security protocols. We consider arbitrary primitives modeled using an equational theory, and a rich process algebra close to the applied pi calculus. Relying on these composition results, we are able to derive some security properties on a protocol from the security analysis performed on each sub-protocol individually. We consider parallel composition and the case of key-exchange protocols. Our results apply to deal with confidentiality but also privacy-type properties (e.g. anonymity, unlinkability) expressed using a notion of equivalence. We illustrate the usefulness of our composition results on protocols from the 3G phone application and electronic passport.
Process-Oriented Parallel Programming with an Application to Data-Intensive Computing
Venue: arXiv cs.PL 2014
Year: 2014

We introduce process-oriented programming as a natural extension of object-oriented programming for parallel computing. It is based on the observation that every class of an object-oriented language can be instantiated as a process, accessible via a remote pointer. The introduction of process pointers requires no syntax extension, identifies processes with programming objects, and enables processes to exchange information simply by executing remote methods. Process-oriented programming is a high-level language alternative to multithreading, MPI and many other languages, environments and tools currently used for parallel computations. It implements natural object-based parallelism using only minimal syntax extension of existing languages, such as C++ and Python, and has therefore the potential to lead to widespread adoption of parallel programming. We implemented a prototype system for running processes using C++ with MPI and used it to compute a large three-dimensional Fourier transform on a computer cluster built of commodity hardware components. Three-dimensional Fourier transform is a prototype of a data-intensive application with a complex data-access pattern. The process-oriented code is only a few hundred lines long, and attains very high data throughput by achieving massive parallelism and maximizing hardware utilization.
Artificial Life and the Web: WebAL Comes of Age
Venue: arXiv cs.NE 2014
Year: 2014

A brief survey is presented of the first 18 years of web-based Artificial Life ("WebAL") research and applications, covering the period 1995-2013. The survey is followed by a short discussion of common methodologies employed and current technologies relevant to WebAL research. The paper concludes with a quick look at what the future may hold for work in this exciting area.
Decoding of Differential AG Codes
Venue: arXiv cs.IT 2014
Year: 2014

The interpolation-based decoding that was developed for general evaluation AG codes is shown to be equally applicable to general differential AG codes. A performance analysis of the decoding algorithm, which is parallel to that of its companion algorithm, is reported. In particular, the decoding capacities of evaluation AG codes and differential AG codes are seen to be nicely interrelated. As an interesting special case, a decoding algorithm for classical Goppa codes is presented.
Generation of pedagogical content based on the learning style of learners in a dynamic adaptive hypermedia environment
Venue: arXiv cs.CY 2014
Year: 2014

Several researches in psychology and science of education affirm the impact of learning style on the learning process and encourage its taking into account in the teaching strategies in order to facilitate the task for learners and improve their results. This article deals with the relationship between the characteristics of the learner, the teaching materials and the context in which takes place the learning in order to allow a better adaptivity. The latter is ensured thanks to the most important element of our system, which is the generator of course, the latter allows you to offer a hypermedia virtual, therefore the pages and the links will be built dynamically, taking into account the learning style and the cognitive status of the learner. As well and after having completed a first filter on the fragments, in order to select those corresponding to the course, there will be applied a second filter to select the fragments corresponding to the learning style of the learner to retain only those in accordance with responsive to the level of knowledge required.
Detecting Flow Anomalies in Distributed Systems
Venue: arXiv cs.CY 2014
Year: 2014

Deep within the networks of distributed systems, one often finds anomalies that affect their efficiency and performance. These anomalies are difficult to detect because the distributed systems may not have sufficient sensors to monitor the flow of traffic within the interconnected nodes of the networks. Without early detection and making corrections, these anomalies may aggravate over time and could possibly cause disastrous outcomes in the system in the unforeseeable future. Using only coarse-grained information from the two end points of network flows, we propose a network transmission model and a localization algorithm, to detect the location of anomalies and rank them using a proposed metric within distributed systems. We evaluate our approach on passengers' records of an urbanized city's public transportation system and correlate our findings with passengers' postings on social media microblogs. Our experiments show that the metric derived using our localization algorithm gives a better ranking of anomalies as compared to standard deviation measures from statistical models. Our case studies also demonstrate that transportation events reported in social media microblogs matches the locations of our detect anomalies, suggesting that our algorithm performs well in locating the anomalies within distributed systems.
The U-curve optimization problem: improvements on the original algorithm and time complexity analysis
Venue: arXiv cs.LG 2014
Year: 2014

The U-curve optimization problem is characterized by a decomposable in U-shaped curves cost function over the chains of a Boolean lattice. This problem can be applied to model the classical feature selection problem in Machine Learning. Recently, the U-Curve algorithm was proposed to give optimal solutions to the U-curve problem. In this article, we point out that the U-Curve algorithm is in fact suboptimal, and introduce the U-Curve-Search (UCS) algorithm, which is actually optimal. We also present the results of optimal and suboptimal experiments, in which UCS is compared with the UBB optimal branch-and-bound algorithm and the SFFS heuristic, respectively. We show that, in both experiments, $\proc{UCS}$ had a better performance than its competitor. Finally, we analyze the obtained results and point out improvements on UCS that might enhance the performance of this algorithm.
Requirements engineering current practice and capability in small and medium software development enterprises in New Zealand
Venue: arXiv cs.SE 2014
Year: 2014

This paper presents research on current industry practices with respect to requirements engineering as implemented within software development companies in New Zealand. A survey instrument is designed and deployed. The results are analysed and compared against what is internationally considered "best practice" and previous New Zealand and Australian studies. An attempt is made to assess the requirements engineering capability of New Zealand companies using both formal and informal frameworks.
Mining developer communication data streams
Venue: arXiv cs.SE 2014
Year: 2014

This paper explores the concepts of modelling a software development project as a process that results in the creation of a continuous stream of data. In terms of the Jazz repository used in this research, one aspect of that stream of data would be developer communication. Such data can be used to create an evolving social network characterized by a range of metrics. This paper presents the application of data stream mining techniques to identify the most useful metrics for predicting build outcomes. Results are presented from applying the Hoeffding Tree classification method used in conjunction with the Adaptive Sliding Window (ADWIN) method for detecting concept drift. The results indicate that only a small number of the available metrics considered have any significance for predicting the outcome of a build.
A review of Plesiochronous Digital Hierarchy (PDH) and Synchronous Digital Hierarchy (SDH)
Venue: arXiv cs.NI 2014
Year: 2014

With the advancement in telecommunications, packet traffic is rapidly becoming the mainstream of data traffic. The use and deployment of Synchronous Digital Hierarchy (SDH) networks for interconnection has gained traction worldwide due to its flexibility and standard for interconnecting multiple vendors, low operating cost and the high quality of service it provides. Plesiochronous Digital Hierarchy (PDH) on the other hand has been used before the introduction of the SDH standard and it also provides a means to transport large quantity of data via digital equipment such as radio wave systems, optic fibre and microwaves. In this paper, we shall discuss both PDH and SDH technologies and identify some of the features of both and the issues in PDH that brought about the introduction of the SDH technology.
A Genetic Algorithm for Software Design Migration from Structured to Object Oriented Paradigm
Venue: arXiv cs.SE 2014
Year: 2014

The potential benefit of migrating software design from Structured to Object Oriented Paradigm is manifolded including modularity, manageability and extendability. This design migration should be automated as it will reduce the time required in manual process. Our previous work has addressed this issue in terms of optimal graph clustering problem formulated by a quadratic Integer Program (IP). However, it has been realized that solution to the IP is computationally hard and thus heuristic based methods are required to get a near optimal solution. This paper presents a Genetic Algorithm (GA) for optimal clustering with an objective of maximizing intra-cluster edges whereas minimizing the inter-cluster ones. The proposed algorithm relies on fitness based parent selection and cross-overing cluster elements to reach an optimal solution step by step. The scheme was implemented and tested against a set of real and synthetic data. The experimental results show that GA outperforms our previous works based on Greedy and Monte Carlo approaches by 40% and 49.5%.
M-best solutions for a class of fuzzy constraint satisfaction problems
Venue: arXiv cs.AI 2014
Year: 2014

The article considers one of the possible generalizations of constraint satisfaction problems where relations are replaced by multivalued membership functions. In this case operations of disjunction and conjunction are replaced by maximum and minimum, and consistency of a solution becomes multivalued rather than binary. The article studies the problem of finding d most admissible solutions for a given d. A tractable subclass of these problems is defined by the concepts of invariants and polymorphisms similar to the classic constraint satisfaction approach. These concepts are adapted in two ways. Firstly, the correspondence of "invariant-polymorphism" is generalized to (min,max) semirings. Secondly, we consider non-uniform polymorphisms, where each variable has its own operator, in contrast to the case of one operator common for all variables. The article describes an algorithm that finds $d$ most admissible solutions in polynomial time, provided that the problem is invariant with respect to some non-uniform majority operator. It is essential that this operator needs not to be known for the algorithm to work. Moreover, even a guarantee for the existence of such an operator is not necessary. The algorithm either finds the solution or discards the problem. The latter is possible only if the problem has no majority polymorphism.
The package HarmonicSums: Computer Algebra and Analytic aspects of Nested Sums
Venue: arXiv cs.SC 2014
Year: 2014

This paper summarizes the essential functionality of the computer algebra package HarmonicSums. On the one hand HarmonicSums can work with nested sums such as harmonic sums and their generalizations and on the other hand it can treat iterated integrals of the Poincare and Chen-type, such as harmonic polylogarithms and their generalizations. The interplay of these representations and the analytic aspects are illustrated by concrete examples.
NeatSort - A practical adaptive algorithm
Venue: arXiv cs.DS 2014
Year: 2014

We present a new adaptive sorting algorithm which is optimal for most disorder metrics and, more important, has a simple and quick implementation. On input $X$, our algorithm has a theoretical $Ω(|X|)$ lower bound and a $\mathcal{O}(|X|\log|X|)$ upper bound, exhibiting amazing adaptive properties which makes it run closer to its lower bound as disorder (computed on different metrics) diminishes. From a practical point of view, \textit{NeatSort} has proven itself competitive with (and often better than) \textit{qsort} and any \textit{Random Quicksort} implementation, even on random arrays.
Which Sustainable Software Practices Do Scientists Find Most Useful?
Venue: arXiv cs.SE 2014
Year: 2014

We studied scientists who attended two-day workshops on basic software skills to determine which tools and practices they found most useful. Our pre- and post-workshop surveys showed increases in self-reported familiarity, while our interviews showed that participants found learning Python more useful than learning the Unix shell, that they found pointers to further resources very valuable, and that background material---the "why" behind the skills---was also very valuable.
A Data Propagation Model for Wireless Gossiping
Venue: arXiv cs.NI 2014
Year: 2014

Wireless sensor networks require communication protocols for efficiently propagating data in a distributed fashion. The Trickle algorithm is a popular protocol serving as the basis for many of the current standard communication protocols. In this paper we develop a mathematical model describing how Trickle propagates new data across a network consisting of nodes placed on a line. The model is analyzed and asymptotic results on the hop count and end-to-end delay distributions in terms of the Trickle parameters and network density are given. Additionally, we show that by only a small modification of the Trickle algorithm the expected end-to-end delay can be greatly decreased. Lastly, we demonstrate how one can derive the exact hop count and end-to-end delay distributions for small network sizes.
Feature Engineering for Knowledge Base Construction
Venue: arXiv cs.DB 2014
Year: 2014

Knowledge base construction (KBC) is the process of populating a knowledge base, i.e., a relational database together with inference rules, with information extracted from documents and structured sources. KBC blurs the distinction between two traditional database problems, information extraction and information integration. For the last several years, our group has been building knowledge bases with scientific collaborators. Using our approach, we have built knowledge bases that have comparable and sometimes better quality than those constructed by human volunteers. In contrast to these knowledge bases, which took experts a decade or more human years to construct, many of our projects are constructed by a single graduate student. Our approach to KBC is based on joint probabilistic inference and learning, but we do not see inference as either a panacea or a magic bullet: inference is a tool that allows us to be systematic in how we construct, debug, and improve the quality of such systems. In addition, inference allows us to construct these systems in a more loosely coupled way than traditional approaches. To support this idea, we have built the DeepDive system, which has the design goal of letting the user "think about features---not algorithms." We think of DeepDive as declarative in that one specifies what they want but not how to get it. We describe our approach with a focus on feature engineering, which we argue is an understudied problem relative to its importance to end-to-end quality.