OpenRaiser/Intern-Atlas huggingface dataset:
An Efficient Preprocessing Methodology for Discovering Patterns and Clustering of Web Users using a Dynamic ART1 Neural Network
Venue: arXiv cs.NE 2011
Year: 2011

In this paper, a complete preprocessing methodology for discovering patterns in web usage mining process to improve the quality of data by reducing the quantity of data has been proposed. A dynamic ART1 neural network clustering algorithm to group users according to their Web access patterns with its neat architecture is also proposed. Several experiments are conducted and the results show the proposed methodology reduces the size of Web log files down to 73-82% of the initial size and the proposed ART1 algorithm is dynamic and learns relatively stable quality clusters.
Get the Most out of Your Sample: Optimal Unbiased Estimators using Partial Information
Venue: arXiv cs.DB 2011
Year: 2011

Random sampling is an essential tool in the processing and transmission of data. It is used to summarize data too large to store or manipulate and meet resource constraints on bandwidth or battery power. Estimators that are applied to the sample facilitate fast approximate processing of queries posed over the original data and the value of the sample hinges on the quality of these estimators. Our work targets data sets such as request and traffic logs and sensor measurements, where data is repeatedly collected over multiple {\em instances}: time periods, locations, or snapshots. We are interested in queries that span multiple instances, such as distinct counts and distance measures over selected records. These queries are used for applications ranging from planning to anomaly and change detection. Unbiased low-variance estimators are particularly effective as the relative error decreases with the number of selected record keys. The Horvitz-Thompson estimator, known to minimize variance for sampling with "all or nothing" outcomes (which reveals exacts value or no information on estimated quantity), is not optimal for multi-instance operations for which an outcome may provide partial information. We present a general principled methodology for the derivation of (Pareto) optimal unbiased estimators over sampled instances and aim to understand its potential. We demonstrate significant improvement in estimate accuracy of fundamental queries for common sampling schemes.
A semi-quantitative equivalence for abstracting from fast reactions
Venue: arXiv cs.CE 2011
Year: 2011

Semantic equivalences are used in process algebra to capture the notion of similar behaviour, and this paper proposes a semi-quantitative equivalence for a stochastic process algebra developed for biological modelling. We consider abstracting away from fast reactions as suggested by the Quasi-Steady-State Assumption. We define a fast-slow bisimilarity based on this idea. We also show congruence under an appropriate condition for the cooperation operator of Bio-PEPA. The condition requires that there is no synchronisation over fast actions, and this distinguishes fast-slow bisimilarity from weak bisimilarity. We also show congruence for an operator which extends the reactions available for a species. We characterise models for which it is only necessary to consider the matching of slow transitions and we illustrate the equivalence on two models of competitive inhibition.
Profiling parallel Mercury programs with ThreadScope
Venue: arXiv cs.PL 2011
Year: 2011

The behavior of parallel programs is even harder to understand than the behavior of sequential programs. Parallel programs may suffer from any of the performance problems affecting sequential programs, as well as from several problems unique to parallel systems. Many of these problems are quite hard (or even practically impossible) to diagnose without help from specialized tools. We present a proposal for a tool for profiling the parallel execution of Mercury programs, a proposal whose implementation we have already started. This tool is an adaptation and extension of the ThreadScope profiler that was first built to help programmers visualize the execution of parallel Haskell programs.
Dynamic Maintenance of Half-Space Depth for Points and Contours
Venue: arXiv cs.CG 2011
Year: 2011

Half-space depth (also called Tukey depth or location depth) is one of the most commonly studied data depth measures because it possesses many desirable properties for data depth functions. The data depth contours bound regions of increasing depth. For the sample case, there are two competing definitions of contours: the rank-based contours and the cover-based contours. In this paper, we present three dynamic algorithms for maintaining the half-space depth of points and contours: The first maintains the half-space depth of a single point in a data set in $O(\log n)$ time per update (insertion/deletion) and overall linear space. By maintaining such a data structure for each data point, we present an algorithm for dynamically maintaining the rank-based contours in $O(n\cdot\log n)$ time per update and overall quadratic space. The third dynamic algorithm maintains the cover-based contours in $O(n\cdot \log^2 n)$ time per update and overall quadratic space. We also augment our first algorithm to maintain the local cover-based contours at data points while maintaining the same complexities. A corollary of this discussion is a strong structural result of independent interest describing the behavior of dynamic cover-based contours near data points.
Daily Deals: Prediction, Social Diffusion, and Reputational Ramifications
Venue: arXiv cs.SI 2011
Year: 2011

Daily deal sites have become the latest Internet sensation, providing discounted offers to customers for restaurants, ticketed events, services, and other items. We begin by undertaking a study of the economics of daily deals on the web, based on a dataset we compiled by monitoring Groupon and LivingSocial sales in 20 large cities over several months. We use this dataset to characterize deal purchases; glean insights about operational strategies of these firms; and evaluate customers' sensitivity to factors such as price, deal scheduling, and limited inventory. We then marry our daily deals dataset with additional datasets we compiled from Facebook and Yelp users to study the interplay between social networks and daily deal sites. First, by studying user activity on Facebook while a deal is running, we provide evidence that daily deal sites benefit from significant word-of-mouth effects during sales events, consistent with results predicted by cascade models. Second, we consider the effects of daily deals on the longer-term reputation of merchants, based on their Yelp reviews before and after they run a daily deal. Our analysis shows that while the number of reviews increases significantly due to daily deals, average rating scores from reviewers who mention daily deals are 10% lower than scores of their peers on average.
On Clustering on Graphs with Multiple Edge Types
Venue: arXiv cs.SI 2011
Year: 2011

We study clustering on graphs with multiple edge types. Our main motivation is that similarities between objects can be measured in many different metrics. For instance similarity between two papers can be based on common authors, where they are published, keyword similarity, citations, etc. As such, graphs with multiple edges is a more accurate model to describe similarities between objects. Each edge/metric provides only partial information about the data; recovering full information requires aggregation of all the similarity metrics. Clustering becomes much more challenging in this context, since in addition to the difficulties of the traditional clustering problem, we have to deal with a space of clusterings. We generalize the concept of clustering in single-edge graphs to multi-edged graphs and investigate problems such as: Can we find a clustering that remains good, even if we change the relative weights of metrics? How can we describe the space of clusterings efficiently? Can we find unexpected clusterings (a good clustering that is distant from all given clusterings)? If given the ground-truth clustering, can we recover how the weights for edge types were aggregated? %In this paper, we discuss these problems and the underlying algorithmic challenges and propose some solutions. We also present two case studies: one based on papers on Arxiv and one based on CIA World Factbook.
A New Rate Region for General Interference Channel (Improved HK Region)
Venue: arXiv cs.IT 2011
Year: 2011

In this paper (a) after detailed investigation of the previous equivalent rate regions for general interference channel, i.e., the Han-Kobayashi (HK) and the Chong-Motani-Garg (CMG) regions, we define modified CMG region the equivalency of which with the HK region is readily seen; (b) we make two novel changes in the HK coding. First, we allow the input auxiliary random variables to be correlated and, second, exploit the powerful technique of random binning instead of the HK -CMG superposition coding, thereby establishing a new rate region for general interference channel, as an improved version of the HK region; (c) we make a novel change in the CMG coding by allowing the message variables to be correlated and obtain an equivalent form for our new region in (b), as an improved version of the CMG region. Then, (d) in order to exactly demarcate the regions, by considering their different easily comparable versions, we compare our region to the HK and CMG regions. Specifically, using a simple dependency structure for the correlated auxiliary random variables, based on the Wyner and Gacs-Korner common information between dependent variables, we show that the HK and the CMG regions are special cases of our new region. Keywords. Interference channel, Correlated auxiliary random variables, Common information, Superposition coding, Binning scheme
Digital Libraries, Conceptual Knowledge Systems, and the Nebula Interface
Venue: arXiv cs.DL 2011
Year: 2011

Concept Analysis provides a principled approach to effective management of wide area information systems, such as the Nebula File System and Interface. This not only offers evidence to support the assertion that a digital library is a bounded collection of incommensurate information sources in a logical space, but also sheds light on techniques for collaboration through coordinated access to the shared organization of knowledge.
The Stability of The Longley-Rice Irregular Terrain Model for Typical Problems
Venue: arXiv cs.NI 2011
Year: 2011

In this paper, we analyze the numerical stability of the popular Longley-Rice Irregular Terrain Model (ITM). This model is widely used to plan wireless networks and in simulation-validated research and hence its stability is of fundamental importance to the correctness of a large amount of work. We take a systematic approach by first porting the reference ITM implementation to a multiprecision framework and then generating loss predictions along many random paths using real terrain data. We find that the ITM is not unstable for common numerical precisions and practical prediction scenarios.
Generalizing Boolean Satisfiability III: Implementation
Venue: arXiv cs.AI 2011
Year: 2011

This is the third of three papers describing ZAP, a satisfiability engine that substantially generalizes existing tools while retaining the performance characteristics of modern high-performance solvers. The fundamental idea underlying ZAP is that many problems passed to such engines contain rich internal structure that is obscured by the Boolean representation used; our goal has been to define a representation in which this structure is apparent and can be exploited to improve computational performance. The first paper surveyed existing work that (knowingly or not) exploited problem structure to improve the performance of satisfiability engines, and the second paper showed that this structure could be understood in terms of groups of permutations acting on individual clauses in any particular Boolean theory. We conclude the series by discussing the techniques needed to implement our ideas, and by reporting on their performance on a variety of problem instances.
Perseus: Randomized Point-based Value Iteration for POMDPs
Venue: arXiv cs.AI 2011
Year: 2011

Partially observable Markov decision processes (POMDPs) form an attractive and principled framework for agent planning under uncertainty. Point-based approximate techniques for POMDPs compute a policy based on a finite set of points collected in advance from the agents belief space. We present a randomized point-based value iteration algorithm called Perseus. The algorithm performs approximate value backup stages, ensuring that in each backup stage the value of each point in the belief set is improved; the key observation is that a single backup may improve the value of many belief points. Contrary to other point-based methods, Perseus backs up only a (randomly selected) subset of points in the belief set, sufficient for improving the value of each belief point in the set. We show how the same idea can be extended to dealing with continuous action spaces. Experimental results show the potential of Perseus in large scale POMDP problems.
CIXL2: A Crossover Operator for Evolutionary Algorithms Based on Population Features
Venue: arXiv cs.NE 2011
Year: 2011

In this paper we propose a crossover operator for evolutionary algorithms with real values that is based on the statistical theory of population distributions. The operator is based on the theoretical distribution of the values of the genes of the best individuals in the population. The proposed operator takes into account the localization and dispersion features of the best individuals of the population with the objective that these features would be inherited by the offspring. Our aim is the optimization of the balance between exploration and exploitation in the search process. In order to test the efficiency and robustness of this crossover, we have used a set of functions to be optimized with regard to different criteria, such as, multimodality, separability, regularity and epistasis. With this set of functions we can extract conclusions in function of the problem at hand. We analyze the results using ANOVA and multiple comparison statistical tests. As an example of how our crossover can be used to solve artificial intelligence problems, we have applied the proposed model to the problem of obtaining the weight of each network in a ensemble of neural networks. The results obtained are above the performance of standard methods.
Optiplan: Unifying IP-based and Graph-based Planning
Venue: arXiv cs.AI 2011
Year: 2011

The Optiplan planning system is the first integer programming-based planner that successfully participated in the international planning competition. This engineering note describes the architecture of Optiplan and provides the integer programming formulation that enabled it to perform reasonably well in the competition. We also touch upon some recent developments that make integer programming encodings significantly more competitive.
$2^{\log^{1-\eps} n}$ Hardness for Closest Vector Problem with Preprocessing
Venue: arXiv cs.CC 2011
Year: 2011

We prove that for an arbitrarily small constant $\eps>0,$ assuming NP$\not \subseteq$DTIME$(2^{{\log^{O(1/\eps)} n}})$, the preprocessing versions of the closest vector problem and the nearest codeword problem are hard to approximate within a factor better than $2^{\log ^{1-\eps}n}.$ This improves upon the previous hardness factor of $(\log n)^δ$ for some $δ> 0$ due to \cite{AKKV05}.
A radial version of the Central Limit Theorem
Venue: arXiv cs.IT 2011
Year: 2011

In this note, we give a probabilistic interpretation of the Central Limit Theorem used for approximating isotropic Gaussians in [1].
Bandits with an Edge
Venue: arXiv cs.LG 2011
Year: 2011

We consider a bandit problem over a graph where the rewards are not directly observed. Instead, the decision maker can compare two nodes and receive (stochastic) information pertaining to the difference in their value. The graph structure describes the set of possible comparisons. Consequently, comparing between two nodes that are relatively far requires estimating the difference between every pair of nodes on the path between them. We analyze this problem from the perspective of sample complexity: How many queries are needed to find an approximately optimal node with probability more than $1-δ$ in the PAC setup? We show that the topology of the graph plays a crucial in defining the sample complexity: graphs with a low diameter have a much better sample complexity.
Visualizing Domain Ontology using Enhanced Anaphora Resolution Algorithm
Venue: arXiv cs.IR 2011
Year: 2011

Enormous explosion in the number of the World Wide Web pages occur every day and since the efficiency of most of the information processing systems is found to be less, the potential of the Internet applications is often underutilized. Efficient utilization of the web can be exploited when similar web pages are rigorously, exhaustively organized and clustered based on some domain knowledge (semantic-based) .Ontology which is a formal representation of domain knowledge aids in such efficient utilization. The performance of almost all the semantic-based clustering techniques depends on the constructed ontology, describing the domain knowledge . The proposed methodology provides an enhanced pronominal anaphora resolution, one of the key aspects of semantic analysis in Natural Language Processing for obtaining cross references within a web page providing better ontology construction. The experimental data sets exhibits better efficiency of the proposed method compared to earlier traditional algorithms.
Linking Search Space Structure, Run-Time Dynamics, and Problem Difficulty: A Step Toward Demystifying Tabu Search
Venue: arXiv cs.AI 2011
Year: 2011

Tabu search is one of the most effective heuristics for locating high-quality solutions to a diverse array of NP-hard combinatorial optimization problems. Despite the widespread success of tabu search, researchers have a poor understanding of many key theoretical aspects of this algorithm, including models of the high-level run-time dynamics and identification of those search space features that influence problem difficulty. We consider these questions in the context of the job-shop scheduling problem (JSP), a domain where tabu search algorithms have been shown to be remarkably effective. Previously, we demonstrated that the mean distance between random local optima and the nearest optimal solution is highly correlated with problem difficulty for a well-known tabu search algorithm for the JSP introduced by Taillard. In this paper, we discuss various shortcomings of this measure and develop a new model of problem difficulty that corrects these deficiencies. We show that Taillards algorithm can be modeled with high fidelity as a simple variant of a straightforward random walk. The random walk model accounts for nearly all of the variability in the cost required to locate both optimal and sub-optimal solutions to random JSPs, and provides an explanation for differences in the difficulty of random versus structured JSPs. Finally, we discuss and empirically substantiate two novel predictions regarding tabu search algorithm behavior. First, the method for constructing the initial solution is highly unlikely to impact the performance of tabu search. Second, tabu tenure should be selected to be as small as possible while simultaneously avoiding search stagnation; values larger than necessary lead to significant degradations in performance.
Lossless data compression on GPGPU architectures
Venue: arXiv cs.DS 2011
Year: 2011

Modern graphics processors provide exceptional computa- tional power, but only for certain computational models. While they have revolutionized computation in many fields, compression has been largely unnaffected. This paper aims to explain the current issues and possibili- ties in GPGPU compression. This is done by a high level overview of the GPGPU computational model in the context of compression algorithms; along with a more in-depth analysis of how one would implement bzip2 on a GPGPU architecture.
Decision-Theoretic Planning with non-Markovian Rewards
Venue: arXiv cs.AI 2011
Year: 2011

A decision process in which rewards depend on history rather than merely on the current state is called a decision process with non-Markovian rewards (NMRDP). In decision-theoretic planning, where many desirable behaviours are more naturally expressed as properties of execution sequences rather than as properties of states, NMRDPs form a more natural model than the commonly adopted fully Markovian decision process (MDP) model. While the more tractable solution methods developed for MDPs do not directly apply in the presence of non-Markovian rewards, a number of solution methods for NMRDPs have been proposed in the literature. These all exploit a compact specification of the non-Markovian reward function in temporal logic, to automatically translate the NMRDP into an equivalent MDP which is solved using efficient MDP solution methods. This paper presents NMRDPP (Non-Markovian Reward Decision Process Planner), a software platform for the development and experimentation of methods for decision-theoretic planning with non-Markovian rewards. The current version of NMRDPP implements, under a single interface, a family of methods based on existing as well as new approaches which we describe in detail. These include dynamic programming, heuristic search, and structured methods. Using NMRDPP, we compare the methods and identify certain problem features that affect their performance. NMRDPPs treatment of non-Markovian rewards is inspired by the treatment of domain-specific search control knowledge in the TLPlan planner, which it incorporates as a special case. In the First International Probabilistic Planning Competition, NMRDPP was able to compete and perform well in both the domain-independent and hand-coded tracks, using search control knowledge in the latter.
Multi-Hypothesis CRF-Segmentation of Neural Tissue in Anisotropic EM Volumes
Venue: arXiv cs.CV 2011
Year: 2011

We present an approach for the joint segmentation and grouping of similar components in anisotropic 3D image data and use it to segment neural tissue in serial sections electron microscopy (EM) images. We first construct a nested set of neuron segmentation hypotheses for each slice. A conditional random field (CRF) then allows us to evaluate both the compatibility of a specific segmentation and a specific inter-slice assignment of neuron candidates with the underlying observations. The model is solved optimally for an entire image stack simultaneously using integer linear programming (ILP), which yields the maximum a posteriori solution in amortized linear time in the number of slices. We evaluate the performance of our approach on an annotated sample of the Drosophila larva neuropil and show that the consideration of different segmentation hypotheses in each slice leads to a significant improvement in the segmentation and assignment accuracy.
Power of Randomization in Automata on Infinite Strings
Venue: arXiv cs.FL 2011
Year: 2011

Probabilistic Büchi Automata (PBA) are randomized, finite state automata that process input strings of infinite length. Based on the threshold chosen for the acceptance probability, different classes of languages can be defined. In this paper, we present a number of results that clarify the power of such machines and properties of the languages they define. The broad themes we focus on are as follows. We present results on the decidability and precise complexity of the emptiness, universality and language containment problems for such machines, thus answering questions central to the use of these models in formal verification. Next, we characterize the languages recognized by PBAs topologically, demonstrating that though general PBAs can recognize languages that are not regular, topologically the languages are as simple as ω-regular languages. Finally, we introduce Hierarchical PBAs, which are syntactically restricted forms of PBAs that are tractable and capture exactly the class of ω-regular languages.