OpenRaiser/Intern-Atlas huggingface dataset:
A Heuristic Description of Fast Fourier Transform
Venue: arXiv cs.NA 2011
Year: 2011

Fast Fourier Transform (FFT) is an efficient algorithm to compute the Discrete Fourier Transform (DFT) and its inverse. In this paper, we pay special attention to the description of complex-data FFT. We analyze two common descriptions of FFT and propose a new presentation. Our heuristic description is helpful for students and programmers to grasp the algorithm entirely and deeply.
A Matricial Algorithm for Polynomial Refinement
Venue: arXiv cs.IT 2011
Year: 2011

In order to have a multiresolution analysis, the scaling function must be refinable. That is, it must be the linear combination of 2-dilation, $\mathbb{Z}$-translates of itself. Refinable functions used in connection with wavelets are typically compactly supported. In 2002, David Larson posed the question in his REU site, "Are all polynomials (of a single variable) finitely refinable?" That summer the author proved that the answer indeed was true using basic linear algebra. The result was presented in a number of talks but had not been typed up until now. The purpose of this short note is to record that particular proof.
Optimal Forwarding in Delay Tolerant Networks with Multiple Destinations
Venue: arXiv cs.NI 2011
Year: 2011

We study the trade-off between delivery delay and energy consumption in a delay tolerant network in which a message (or a file) has to be delivered to each of several destinations by epidemic relaying. In addition to the destinations, there are several other nodes in the network that can assist in relaying the message. We first assume that, at every instant, all the nodes know the number of relays carrying the packet and the number of destinations that have received the packet. We formulate the problem as a controlled continuous time Markov chain and derive the optimal closed loop control (i.e., forwarding policy). However, in practice, the intermittent connectivity in the network implies that the nodes may not have the required perfect knowledge of the system state. To address this issue, we obtain an ODE (i.e., a deterministic fluid) approximation for the optimally controlled Markov chain. This fluid approximation also yields an asymptotically optimal open loop policy. Finally, we evaluate the performance of the deterministic policy over finite networks. Numerical results show that this policy performs close to the optimal closed loop policy.
Cellular Automata Using Infinite Computations
Venue: arXiv cs.DM 2011
Year: 2011

This paper presents an application of the Infinite Unit Axiom, introduced by Yaroslav Sergeyev, (see [11] - [14]) to the development of one-dimensional cellular automata. This application allows the establishment of a new and more precise metric on the space of definition for one-dimensional cellular automata, whereby accuracy of computations is increased. Using this new metric, open disks are defined and the number of points in each disk is computed. The forward dynamics of a cellular automaton map are also studied via defined equivalence classes. Using the Infinite Unit Axiom, the number of configurations that stay close to a given configuration under the shift automaton map can now be computed.
TopicViz: Semantic Navigation of Document Collections
Venue: arXiv cs.HC 2011
Year: 2011

When people explore and manage information, they think in terms of topics and themes. However, the software that supports information exploration sees text at only the surface level. In this paper we show how topic modeling -- a technique for identifying latent themes across large collections of documents -- can support semantic exploration. We present TopicViz, an interactive environment for information exploration. TopicViz combines traditional search and citation-graph functionality with a range of novel interactive visualizations, centered around a force-directed layout that links documents to the latent themes discovered by the topic model. We describe several use scenarios in which TopicViz supports rapid sensemaking on large document collections.
Un modello di struttura dinamica per ebook scolastici
Venue: arXiv cs.DL 2011
Year: 2011

This article proposes a model of e-books for schools based on a graph in which nodes represent individual subjects of a teaching program to a the relatively low granularity, which facilitates their aggregation and re-usability, and edges represent prerequisites between subjects (and, indeed, between nodes). On this graph we will develop a series of simple algorithms that allow both teachers and students to assemble an interactive and personalized ebook that, respecting the prerequisites, it will be significant from the point of methodological and stylistic sense. Therefore, teachers and students do not have available a set of unrelated units neither a limited set a few pre-packaged learning paths, as it is typical of some solutions on the Web, but rather will have a network of topics that can be serialized in a combinatorial vast number of alternatives, and therefore can create as many custom ebook, but guaranteed from the point of view of the scientific perspective.
Incorporating Agile with MDA Case Study: Online Polling System
Venue: arXiv cs.SE 2011
Year: 2011

Nowadays agile software development is used in greater extend but for small organizations only, whereas MDA is suitable for large organizations but yet not standardized. In this paper the pros and cons of Model Driven Architecture (MDA) and Extreme programming have been discussed. As both of them have some limitations and cannot be used in both large scale and small scale organizations a new architecture has been proposed. In this model it is tried to opt the advantages and important values to overcome the limitations of both the software development procedures. In support to the proposed architecture the implementation of it on Online Polling System has been discussed and all the phases of software development have been explained.
Probabilistic Planning via Heuristic Forward Search and Weighted Model Counting
Venue: arXiv cs.AI 2011
Year: 2011

We present a new algorithm for probabilistic planning with no observability. Our algorithm, called Probabilistic-FF, extends the heuristic forward-search machinery of Conformant-FF to problems with probabilistic uncertainty about both the initial state and action effects. Specifically, Probabilistic-FF combines Conformant-FFs techniques with a powerful machinery for weighted model counting in (weighted) CNFs, serving to elegantly define both the search space and the heuristic function. Our evaluation of Probabilistic-FF shows its fine scalability in a range of probabilistic domains, constituting a several orders of magnitude improvement over previous results in this area. We use a problematic case to point out the main open issue to be addressed by further research.
Query-time Entity Resolution
Venue: arXiv cs.DB 2011
Year: 2011

Entity resolution is the problem of reconciling database references corresponding to the same real-world entities. Given the abundance of publicly available databases that have unresolved entities, we motivate the problem of query-time entity resolution quick and accurate resolution for answering queries over such unclean databases at query-time. Since collective entity resolution approaches --- where related references are resolved jointly --- have been shown to be more accurate than independent attribute-based resolution for off-line entity resolution, we focus on developing new algorithms for collective resolution for answering entity resolution queries at query-time. For this purpose, we first formally show that, for collective resolution, precision and recall for individual entities follow a geometric progression as neighbors at increasing distances are considered. Unfolding this progression leads naturally to a two stage expand and resolve query processing strategy. In this strategy, we first extract the related records for a query using two novel expansion operators, and then resolve the extracted records collectively. We then show how the same strategy can be adapted for query-time entity resolution by identifying and resolving only those database references that are the most helpful for processing the query. We validate our approach on two large real-world publication databases where we show the usefulness of collective resolution and at the same time demonstrate the need for adaptive strategies for query processing. We then show how the same queries can be answered in real-time using our adaptive approach while preserving the gains of collective resolution. In addition to experiments on real datasets, we use synthetically generated data to empirically demonstrate the validity of the performance trends predicted by our analysis of collective entity resolution over a wide range of structural characteristics in the data.
Chain: A Dynamic Double Auction Framework for Matching Patient Agents
Venue: arXiv cs.GT 2011
Year: 2011

In this paper we present and evaluate a general framework for the design of truthful auctions for matching agents in a dynamic, two-sided market. A single commodity, such as a resource or a task, is bought and sold by multiple buyers and sellers that arrive and depart over time. Our algorithm, Chain, provides the first framework that allows a truthful dynamic double auction (DA) to be constructed from a truthful, single-period (i.e. static) double-auction rule. The pricing and matching method of the Chain construction is unique amongst dynamic-auction rules that adopt the same building block. We examine experimentally the allocative efficiency of Chain when instantiated on various single-period rules, including the canonical McAfee double-auction rule. For a baseline we also consider non-truthful double auctions populated with zero-intelligence plus"-style learning agents. Chain-based auctions perform well in comparison with other schemes, especially as arrival intensity falls and agent valuations become more volatile.
Exploiting Subgraph Structure in Multi-Robot Path Planning
Venue: arXiv cs.AI 2011
Year: 2011

Multi-robot path planning is difficult due to the combinatorial explosion of the search space with every new robot added. Complete search of the combined state-space soon becomes intractable. In this paper we present a novel form of abstraction that allows us to plan much more efficiently. The key to this abstraction is the partitioning of the map into subgraphs of known structure with entry and exit restrictions which we can represent compactly. Planning then becomes a search in the much smaller space of subgraph configurations. Once an abstract plan is found, it can be quickly resolved into a correct (but possibly sub-optimal) concrete plan without the need for further search. We prove that this technique is sound and complete and demonstrate its practical effectiveness on a real map. A contending solution, prioritised planning, is also evaluated and shown to have similar performance albeit at the cost of completeness. The two approaches are not necessarily conflicting; we demonstrate how they can be combined into a single algorithm which outperforms either approach alone.
A General Theory of Additive State Space Abstractions
Venue: arXiv cs.AI 2011
Year: 2011

Informally, a set of abstractions of a state space S is additive if the distance between any two states in S is always greater than or equal to the sum of the corresponding distances in the abstract spaces. The first known additive abstractions, called disjoint pattern databases, were experimentally demonstrated to produce state of the art performance on certain state spaces. However, previous applications were restricted to state spaces with special properties, which precludes disjoint pattern databases from being defined for several commonly used testbeds, such as Rubiks Cube, TopSpin and the Pancake puzzle. In this paper we give a general definition of additive abstractions that can be applied to any state space and prove that heuristics based on additive abstractions are consistent as well as admissible. We use this new definition to create additive abstractions for these testbeds and show experimentally that well chosen additive abstractions can reduce search time substantially for the (18,4)-TopSpin puzzle and by three orders of magnitude over state of the art methods for the 17-Pancake puzzle. We also derive a way of testing if the heuristic value returned by additive abstractions is provably too low and show that the use of this test can reduce search time for the 15-puzzle and TopSpin by roughly a factor of two.
A Bigraph Relational Model
Venue: arXiv cs.LO 2011
Year: 2011

In this paper, we present a model based on relations for bigraphical reactive systems [Milner09]. Its defining characteristics are that validity and reaction relations are captured as traces in a multi-set rewriting system. The relational model is derived from Milner's graphical definition and directly amenable to implementation.
An Improved Implementation and Abstract Interface for Hybrid
Venue: arXiv cs.LO 2011
Year: 2011

Hybrid is a formal theory implemented in Isabelle/HOL that provides an interface for representing and reasoning about object languages using higher-order abstract syntax (HOAS). This interface is built around an HOAS variable-binding operator that is constructed definitionally from a de Bruijn index representation. In this paper we make a variety of improvements to Hybrid, culminating in an abstract interface that on one hand makes Hybrid a more mathematically satisfactory theory, and on the other hand has important practical benefits. We start with a modification of Hybrid's type of terms that better hides its implementation in terms of de Bruijn indices, by excluding at the type level terms with dangling indices. We present an improved set of definitions, and a series of new lemmas that provide a complete characterization of Hybrid's primitives in terms of properties stated at the HOAS level. Benefits of this new package include a new proof of adequacy and improvements to reasoning about object logics. Such proofs are carried out at the higher level with no involvement of the lower level de Bruijn syntax.
A Distributed Intrusion Detection System Using Cooperating Agents
Venue: arXiv cs.CR 2011
Year: 2011

The current intrusion detection systems have a number of problems that limit their configurability, scalability and efficiency. There have been some propositions about distributed architectures based on multiple independent agents working collectively for intrusion detection. However, these distributed intrusion detection systems are not fully distributed as most of them centrally analyze data collected from distributed nodes which may lead to a single point of failure. In this paper, a distributed intrusion detection architecture is presented that is based on autonomous and cooperating agents without any centralized analysis components. The agents cooperate by using a hierarchical communication of interests and data, and the analysis of intrusion data is made by the agents at the lowest level of the hierarchy. This architecture provides significant advantages in scalability, flexibility, extensibility, fault tolerance, and resistance to compromise. A proof-of-concept prototype is developed and experiments have been conducted on it. The results show the effectiveness of the system in detecting intrusive activities.
Making the long code shorter, with applications to the Unique Games Conjecture
Venue: arXiv cs.CC 2011
Year: 2011

The long code is a central tool in hardness of approximation, especially in questions related to the unique games conjecture. We construct a new code that is exponentially more efficient, but can still be used in many of these applications. Using the new code we obtain exponential improvements over several known results, including the following: 1. For any eps > 0, we show the existence of an n vertex graph G where every set of o(n) vertices has expansion 1 - eps, but G's adjacency matrix has more than exp(log^delta n) eigenvalues larger than 1 - eps, where delta depends only on eps. This answers an open question of Arora, Barak and Steurer (FOCS 2010) who asked whether one can improve over the noise graph on the Boolean hypercube that has poly(log n) such eigenvalues. 2. A gadget that reduces unique games instances with linear constraints modulo K into instances with alphabet k with a blowup of K^polylog(K), improving over the previously known gadget with blowup of 2^K. 3. An n variable integrality gap for Unique Games that that survives exp(poly(log log n)) rounds of the SDP + Sherali Adams hierarchy, improving on the previously known bound of poly(log log n). We show a connection between the local testability of linear codes and small set expansion in certain related Cayley graphs, and use this connection to derandomize the noise graph on the Boolean hypercube.
Clique cover and graph separation: New incompressibility results
Venue: arXiv cs.DS 2011
Year: 2011

The field of kernelization studies polynomial-time preprocessing routines for hard problems in the framework of parameterized complexity. Although a framework for proving kernelization lower bounds has been discovered in 2008 and successfully applied multiple times over the last three years, establishing kernelization complexity of many important problems remains open. In this paper we show that, unless NP is a subset of coNP/poly and the polynomial hierarchy collapses up to its third level, the following parameterized problems do not admit a polynomial-time preprocessing algorithm that reduces the size of an instance to polynomial in the parameter: - EDGE CLIQUE COVER, parameterized by the number of cliques, - DIRECTED EDGE/VERTEX MULTIWAY CUT, parameterized by the size of the cutset, even in the case of two terminals, - EDGE/VERTEX MULTICUT, parameterized by the size of the cutset, and - k-WAY CUT, parameterized by the size of the cutset. The existence of a polynomial kernelization for EDGE CLIQUE COVER was a seasoned veteran in open problem sessions. Furthermore, our results complement very recent developments in designing parameterized algorithms for cut problems by Marx and Razgon [STOC'11], Bousquet et al. [STOC'11], Kawarabayashi and Thorup [FOCS'11] and Chitnis et al. [SODA'12].
On Identity Testing of Tensors, Low-rank Recovery and Compressed Sensing
Venue: arXiv cs.CC 2011
Year: 2011

We study the problem of obtaining efficient, deterministic, black-box polynomial identity testing algorithms for depth-3 set-multilinear circuits (over arbitrary fields). This class of circuits has an efficient, deterministic, white-box polynomial identity testing algorithm (due to Raz and Shpilka), but has no known such black-box algorithm. We recast this problem as a question of finding a low-dimensional subspace H, spanned by rank 1 tensors, such that any non-zero tensor in the dual space ker(H) has high rank. We obtain explicit constructions of essentially optimal-size hitting sets for tensors of degree 2 (matrices), and obtain quasi-polynomial sized hitting sets for arbitrary tensors (but this second hitting set is less explicit). We also show connections to the task of performing low-rank recovery of matrices, which is studied in the field of compressed sensing. Low-rank recovery asks (say, over the reals) to recover a matrix M from few measurements, under the promise that M is rank <=r. We also give a formal connection between low-rank recovery and the task of sparse (vector) recovery: any sparse-recovery algorithm that exactly recovers vectors of length n and sparsity 2r, using m non-adaptive measurements, yields a low-rank recovery scheme for exactly recovering nxn matrices of rank <=r, making 2nm non-adaptive measurements. Furthermore, if the sparse-recovery algorithm runs in time τ, then the low-rank recovery algorithm runs in time O(rn^2+nτ). We obtain this reduction using linear-algebraic techniques, and not using convex optimization, which is more commonly seen in compressed sensing algorithms. By using a dual Reed-Solomon code, we are able to (deterministically) construct low-rank recovery schemes taking 4nr measurements over the reals, such that the measurements can be all rank-1 matrices, or all sparse matrices.
Maximum Bounded Rooted-Tree Packing Problem
Venue: arXiv cs.CC 2011
Year: 2011

Given a graph and a root, the Maximum Bounded Rooted-Tree Packing (MBRTP) problem aims at finding K rooted-trees that span the largest subset of vertices, when each vertex has a limited outdegree. This problem is motivated by peer-to-peer streaming overlays in under-provisioned systems. We prove that the MBRTP problem is NP-complete. We present two polynomial-time algorithms that computes an optimal solution on complete graphs and trees respectively.
Using Automated Dependency Analysis To Generate Representation Information
Venue: arXiv cs.DL 2011
Year: 2011

To preserve access to digital content, we must preserve the representation information that captures the intended interpretation of the data. In particular, we must be able to capture performance dependency requirements, i.e. to identify the other resources that are required in order for the intended interpretation to be constructed successfully. Critically, we must identify the digital objects that are only referenced in the source data, but are embedded in the performance, such as fonts. This paper describes a new technique for analysing the dynamic dependencies of digital media, focussing on analysing the process that underlies the performance, rather than parsing and deconstructing the source data. This allows the results of format-specific characterisation tools to be verified independently, and facilitates the generation of representation information for any digital media format, even when no suitable characterisation tool exists.
Towards "Intelligent Compression" in Streams: A Biased Reservoir Sampling based Bloom Filter Approach
Venue: arXiv cs.IR 2011
Year: 2011

With the explosion of information stored world-wide,data intensive computing has become a central area of research.Efficient management and processing of this massively exponential amount of data from diverse sources,such as telecommunication call data records,online transaction records,etc.,has become a necessity.Removing redundancy from such huge(multi-billion records) datasets resulting in resource and compute efficiency for downstream processing constitutes an important area of study. "Intelligent compression" or deduplication in streaming scenarios,for precise identification and elimination of duplicates from the unbounded datastream is a greater challenge given the realtime nature of data arrival.Stable Bloom Filters(SBF) address this problem to a certain extent.However,SBF suffers from a high false negative rate(FNR) and slow convergence rate,thereby rendering it inefficient for applications with low FNR tolerance.In this paper, we present a novel Reservoir Sampling based Bloom Filter,(RSBF) data structure,based on the combined concepts of reservoir sampling and Bloom filters for approximate detection of duplicates in data streams.Using detailed theoretical analysis we prove analytical bounds on its false positive rate(FPR),false negative rate(FNR) and convergence rates with low memory requirements.We show that RSBF offers the currently lowest FN and convergence rates,and are better than those of SBF while using the same memory.Using empirical analysis on real-world datasets(3 million records) and synthetic datasets with around 1 billion records,we demonstrate upto 2x improvement in FNR with better convergence rates as compared to SBF,while exhibiting comparable FPR.To the best of our knowledge,this is the first attempt to integrate reservoir sampling method with Bloom filters for deduplication in streaming scenarios.