Using landing pages for sponsored search ad selection We explore the use of the landing page content in sponsored search ad selection. Specifically, we compare the use of the ad's intrinsic content to augmenting the ad with the whole, or parts, of the landing page. We explore two types of extractive summarization techniques to select useful regions from the landing pages: out-of-context and in-context methods. Out-of-context methods select salient regions from the landing page by analyzing the content alone, without taking into account the ad associated with the landing page. In-context methods use the ad context (including its title, creative, and bid phrases) to help identify regions of the landing page that should be used by the ad selection engine. In addition, we introduce a simple yet effective unsupervised algorithm to enrich the ad context to further improve the ad selection. Experimental evaluation confirms that the use of landing pages can significantly improve the quality of ad selection. We also find that our extractive summarization techniques reduce the size of landing pages substantially, while retaining or even improving the performance of ad retrieval over the method that utilize the entire landing page. Web search is the gateway to the Internet for billions of users daily. When the user issues a query to the search engine, two separate searches are evaluated: the search over the corpus of pre-crawled web pages is called web search; the advertisements that are displayed at the top and the side of the web search results are retrieved by sponsored search. Sponsored search provides revenue for the search engine and brings users to numerous advertiser sites.. Web search and sponsored search differ in a few key aspects. Sponsored search is evaluated over a set of ads that promote products and services. As it is customary in the advertising world, the textual content visible to the user (ad creative), is generated by the advertiser to maximize the response of the target audience. In web search on the other hand, the snippet shown on the search result page is generated automatically by the summarization mechanism of the search engine. Another important difference is in the way the ads and the web results are selected. While the web pages are selected based on their content, the ad selection depends heavily on the use of the ad bid phrase -a query that the advertiser has specified as suitable for the ad. In the early days of the sponsored search marketplace, this mechanism allowed for simple ad selection where the whole burden (and control) is shifted to the advertiser. However, with the development of the sponsored search market, it become quickly apparent that the advertisers cannot possibly find all the queries that could be relevant to their advertisements.. To alleviate this problem the search engines allow for advanced match where ad can be selected even if their bid phrase does not match the query. The advanced match problem corresponds closer to the web search problem. Recent advanced match approaches use search techniques for ad selection by evaluating the query over a corpus of documents that are created from the ads [7, 25] . One of the key difficulties in this ad retrieval approach is that the ads are much shorter than documents in most other search applications [25] .. In this paper, we explore the use of the landing page in ad retrieval for sponsored search. We contrast the use of the content of the ad creative with the use of the whole, or parts, of the landing page. Our study was partly motivated by a preliminary examination of a set of textual ads and their landing pages, which indicated that over 30% of the landing pages are not at all, or very remotely, related to the ads. Thus our intuition suggests that indiscriminate use the content of such landing pages in the ad selection would decrease the precision of the ad retrieval.. We explore two types of extractive summarization techniques to select useful regions from the landing pages: outof-context and in-context methods. Out-of-context methods select regions from the landing page by simply analyzing the landing page itself, without taking the ad context into account. The ad context is composed of the creative, bid phrase, title and any other information about the ad that can be computed offline (i.e., prior to query processing). In-context methods use the ad context to help select which regions of the landing page should be used by the ad selection engine. In addition, we introduce a simple yet effective unsupervised algorithm motivated by compositional vector space models [21, 22, 14] based on compositional semantics [23, 21] in order to enrich the ad context and enhance the ad selection. Experimental results demonstrate that selective use of landing pages can significantly improve the quality of ad selection. We also find that our extractive summarization techniques reduce the size of landing pages substantially, thereby reducing the amount of data that needs to be indexed, while retaining or even improving the performance of ad retrieval over the method that utilize the entire landing page.. The contributions of this paper are threefold:. • We quantify the benefit of using the landing page for ad selection in sponsored search. In particular, we examine a number of different extractive summarization techniques to make the best use of landing pages.. • We propose a simple yet effective unsupervised algorithm using compositional vector space models to enrich the ad context. We then present two different ways in which the enriched ad context can be utilized to enhance the ad selection.. • We report experimental results that show that our proposed methods for selecting landing page regions have up to 8.5% performance improvement in Discounted Cumulative Gain (DCG), measured over a productionlevel ad selection system. In this paper, we explore a number of extractive summarization techniques for landing pages in order to enhance sponsored search ad retrieval. We contrast two hypotheses -in-context and out-of-context summarization of landing pages with respect to the advertisement intent, and find that in-context summarization techniques are more effective for improving sponsored search. Empirical results show that applying extractive summarization techniques to landing pages can reduce the amount of data that needs to be indexed significantly, while retaining or even improving the performance of ad retrieval over the method that utilize the entire landing page.. Our work is the first to utilize compositional vector space models in the context of ad retrieval. We explore a range of compositional vector operations that combine co-occurrence vectors to enrich the succinct advertisement. We then show two different ways in which the enriched ad context can be utilized. First, it helps to extract more useful regions in the landing page with respect to the ad intent. Second, the enriched ad context can be a useful resource on its own to reduce the vocabulary mismatch.. In the future, we plan to extend extractive summarization techniques presented in this paper in order to reliably detect the deceptive advertisements that link to spam or broken landing pages. We discuss the related work from three different aspects; sponsored search and content match (Section 5.1), web page summarization (Section 5.2), and applications of compositional vector space models (Section 5.3). A large part of the $30 billion Web advertising market consists of textual ads, the ubiquitous short text messages usually marked as "sponsored links". There are two main channels for distributing such ads. Sponsored search places ads on the result pages of a Web search engine, where ads are selected to be relevant to the search query (see [11] for a brief history of the subject). All major Web search engines (Google, Microsoft, Yahoo!) support sponsored ads and act simultaneously as a Web search engine and an ad search engine. Content match (or contextual advertising) places ads on third-party Web pages. Today, almost all of the for-profit non-transactional Web sites (without direct sales) rely at least to some extent on contextual advertising revenue. Content match supports sites that range from individual bloggers and small niche communities, to large publishers such as major newspapers.. In this paper we focus on sponsored search. However, content match ads are identical to the sponsored search ads and we believe that using landing page content for ad selection would be applicable to content match as well.. Sponsored search is an interplay of three entities: The advertiser provides the supply of ads. As in traditional advertising, the goal of the advertisers can be broadly defined as promotion of products or services. The search engine provides "real estate" for placing ads (i.e., allocates space on search results pages), and selects ads that are relevant to the user's query. Users visit the Web pages and interact with the ads.. The prevalent pricing model for textual ads is that the advertisers pay for every click on the advertisement (pay-perclick or PPC). The amount paid by the advertiser for each sponsored search click is usually determined by an auction process [9] . The advertisers place bids on a search phrase, and their position in the column of ads displayed on the search results page is determined by their bid. Thus, each ad is annotated with one or more bid phrases. In addition to the bid phrase, an ad also contains a title usually displayed in bold font, and a creative, which is the few lines of text, usually shorter than 120 characters, displayed on the page. Naturally, each ad contains a URL to the advertised Web page, called the landing page. A recent study, analyzed the types of landing pages [3] and we classified the landing pages into three main categories: homepage, which are top-level pages on the advertisers' Web site; search transfer, which are dynamically generated search result pages on the advertiser's site; and category browse, which are subsections of the advertiser's site, generally related to the user query.. In this work we explore the use of landing pages in the context of an ad retrieval system that is based on information retrieval principles, as reported in [7] . The input to our system is a search (or "Web") query, and the output is a set of ads that are relevant to this query. We represent the ads and the queries in a vector space model using their unigrams and phrases as features. The query-ads similarity is computed using cosine between the angle between their vectors. Assuming that the query vector and the ad vectors are pre-normalized using L2 norm, the scoring function is a simple dot product:. where w f,ad and w f,query are the weights of the feature f in the ad and the query accordingly. For weighting of the features we use a section-aware variant of tf-idf where each region of the ad is given a tf multiplier. This weighting scheme can be naturally extended to incorporate new regions of the ad and the query, as the one we explore in this paper -the landing page. The ads are processed and indexed in an inverted index of ads that at runtime is used to evaluate similarity queries by a document-at-the-time algorithm. For more details we refer the reader to [7] . To validate and compare the proposed approaches, we use a data set sampled from the sponsored search traffic of a major search engine. The query-ad pairs in this sample were evaluated for relevance by professional editorial staff. We evaluate our methods by augmenting the existing ad selection mechanisms to use the landing page features and rerank the judged pairs. In the following we first give more details about the data set, and then present the evaluation results.

O B I O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I B I O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B I O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Sliding Windows and Persistence: An Application of Topological Methods to Signal Analysis Abstract. We develop in this paper a theoretical framework for the topological study of time series data. Broadly speaking, we describe geometrical and topological properties of sliding window embeddings, as seen through the lens of persistent homology. In particular, we show that maximum persistence at the point-cloud level can be used to quantify periodicity at the signal level, prove structural and convergence theorems for the resulting persistence diagrams, and derive estimates for their dependency on window size and embedding dimension. We apply this methodology to quantifying periodicity in synthetic data sets, and compare the results with those obtained using state-of-the-art methods in gene expression analysis. We call this new method SW1PerS which stands for Sliding Windows and 1-dimensional Persistence Scoring. Signal analysis is an enormous field. There are many methods to study signals and many applications of that study. Given its importance, one might conclude that there is little opportunity left for the development of totally new approaches to signals. Yet in this paper we provide a new way to find periodicity and quasiperiodicity in signals. The method is based on sliding windows (also known as time-delay reconstruction), which have been used extensively in both engineering applications and in dynamical systems. But it adds a new element not applied before, which comes from the new field of computational topology [12] .. Persistent homology is a topological method for measuring the shapes of spaces and the features of functions. One of the most important applications of persistent homology is to point clouds [3] , where shape is usually interpreted as the geometry of some implicit underlying object near which the point cloud is sampled. The simplest non-trivial example of this idea is a point cloud which has the shape of a circle, and this shape is captured with 1-dimensional persistence. The challenge in applying the method is that noise can reduce the persistence, and not enough points can prevent the circular shape from appearing. It's also a challenge to deal with the fact that features come on all scale-levels and can be nested or in more complicated relationships. But this is what persistent homology is all about.. The idea of applying 1D persistence to study time series arose in our study of gene expression data [11, 23] . The first of these papers studied a variety of existing methods for finding periodicity in gene expression patterns. The motivation of that work was the search for gene regulatory networks (more precisely, possible nodes of gene regulatory networks) that control periodic processes in cells such as the cell division cycle, circadian rhythms, metabolic cycles and periodic patterning in biological development (lateral roots and somites). The methods studied in [11] were derived from a number of fields including astronomy, geometry, biology and statistics, and all were based on a direct study of the underlying signal in either physical or frequency space. The most successful methods are based on finding cosine-like behavior, a rather limited definition of periodicity.. In this paper and in [23] we look instead at the shape of the sliding window point cloud, a totally different approach. Of course the geometry of point clouds derived from other kinds of data like images has been studied before [4, 16] , but the current approach is quite different. Our method understands periodicity as repetition of patterns, whatever these may be, and quantifies this reoccurrence as the degree of circularity/roundness in the generated point-cloud. Thus, it is fundamentally agnostic.. 1.1. Previous Work. The sliding window, or time-delay embedding, has been used mostly in the study of dynamical systems to understand the nature of their attractors. Takens' theorem [28] gives conditions under which a smooth attractor can be reconstructed from the observations of a function, with bounds related to those of the Whitney Embedding Theorem. This methodology has in turn been employed to test for non-linearity and chaotic behavior in the dynamics of ECG-EKG, EEG and MEG [25, 26] .. It has been recently demonstrated by de Silva et. al. [10] that combining timedelay embeddings with topological methods provides a framework for parametrizing periodic systems.. Kantz and Schreiber provide in [16, Chapter 1] a good source of examples of time delay embeddings used in real-world data sets.. 1.2. Our Contribution. In the above applications, little of the topology and none of the geometry of the resulting sliding window embedding has ever been used. The novelty of our approach lies in our use of this geometry and topology through persistent homology. We make this possible by showing that maximum persistence, as a measure of "roundness" of the point-cloud, occurs when the window size corresponds to the natural frequency of the signal. This means that 1D persistence is an effective quantifier of periodicity and quasi-periodicity and can be used to infer properties of the signal.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O


Continuous bottleneck tree partitioning problems We study continuous partitioning problems on tree network spaces whose edges and nodes are points in Euclidean spaces. A continuous partition of this space into p connected components is a collection of p subtrees, such that no pair of them intersect at more than one point, and their union is the tree space. An edge-partition is a continuous partition deÿned by selecting p − 1 cut points along the edges of the underlying tree, which is assumed to have n nodes. These cut points induce a partition into p subtrees (connected components). The objective is to minimize (maximize) the maximum (minimum) "size" of the components (the min-max (max-min) problem). When the size is the length of a subtree, the min-max and the max-min partitioning problems are NP-hard. We present O(n 2 log(min(p; n))) algorithms for the edge-partitioning versions of the problem. When the size is the diameter, the min-max problems coincide with the continuous p-center problem. We describe O(n log 3 n) and O(n log 2 n) algorithms for the maxmin partitioning and edge-partitioning problems, respectively, where the size is the diameter of a component. ? There are numerous papers in the literature studying discrete bottleneck tree partitioning problems, deÿned by deleting p − 1 edges from a given tree graph and looking at the set of p discrete subtrees induced by the partition. (Of course, in the discrete cases p ¡ n.) For example the reader is referred to [2, 3, [11] [12] [13] [14] 17, 22, 23] for the most recent studies. In this paper we focus on continuous partitioning problems on the metric space of the continuum set of points on the edges of a tree in Euclidean space, induced by the edge lengths. The objective is to partition this space (a tree network space) into p connected components (closed subtrees), optimizing the maximum or minimum "size" of the components. Typically, the size will refer to the length or the diameter of the component. (Informally, the length of a component is the sum of the lengths of its edges, and its diameter is the length of a longest simple path of the component.) In a continuous partition we require that no pair of components intersect at more than one point, and the union of the components is the tree network space. We are aware of only a few papers dealing with continuous tree partitioning problems. For example, in the continuous p-center problem on a tree the objective is to split the above space into p connected components, minimizing the maximum of the diameter (radius) of the components. Polynomial algorithms solving this problem appear in [8, 9, 20] . On the other hand, if we wish to minimize (maximize) the maximum (minimum) length of the components the problem is NP-hard, since the Partition problem is a special case [16] . (Consider the case of a tree consisting of n − 1 leaves, each one of them connected by an edge to a central node, and let p = 2.) The recent paper by Agnetis et al. [1] considers the model of partitioning a tree space into p subsets, not necessarily connected, of equal length, while minimizing the maximum diameter of the subsets. They provide polynomial algorithms only for the cases where p = 2 and 3.. In view of the above NP-hardness result, in this paper we focus mainly on a class of continuous partitions, where the p − 1 cut points, splitting the space into p components are restricted to be on the edges of the tree. (See the next section for an exact deÿnition of a cut.) We call such partitions continuous edge-partitions. As it will be clear from the deÿnition, not every partition is an edge-partition. (For example, splitting a ÿve node star tree, with one center node connected to four leaves, into two components, each having exactly two leaves can be achieved by a continuous partition, but not by a continuous edge-partition. See Fig. 1 at the end of Section 3.) Such a model has been recently discussed by Becker et al. [4] [5] [6] . They study the continuous max-min tree edge-partitioning problem, where the objective is to maximize the minimum length of the components. Assuming rational data, the authors present an O(n 2 p 2 + np 3 ) algorithm to solve the model. We are unaware of papers dealing with the continuous min-max tree partitioning problem. We adapt the general approach in [8, 9] , to improve upon the above results. Speciÿcally, for any real data we present O(n 2 log(min(p; n))) algorithms to solve both the continuous max-min and min-max tree length edge-partitioning problems.. We also consider partitioning and edge-partitioning bottleneck problems involving the diameters of the components. The min-max model, where the goal is to minimize the maximum of the diameters of the p components is the continuous p-center problem, for which an O(n log 2 n) algorithm is already known, [10, 20] . For the maxmin problems where the goal is to maximize the minimum of the diameters of the p components, we obtain the following results. For the edge-partitioning version we derive an O(n log 2 n) algorithm, while the partitioning version is solved in O(n log 3 n) time.. node v i (v j ) only, and the other consisting of the (closed) edge (v i ; v j ). (Again, note that v i (v j ) is in both parts of the split.) v i (v j ) is called the marker of the cut, and the edge (v i ; v j ) is its direction. If the marker of the node cut is not at a leaf of the tree it also partitions the tree into two subtrees of positive length, intersecting at the marker only. Note that an interior cut is uniquely deÿned by an interior point of an edge, while a node cut is characterized by a node and an edge which is incident to that node. In particular, there are at most deg(v i ) distinct cuts associated with a node v i . (deg(v i ) is the number of edges incident to v i .) To illustrate consider the example in Fig. 1 . A continuous 2-partition splitting the tree into two subtrees, each having exactly two leaves is deÿned by the two subtrees induced respectively by the node sets {v 1 Since p−1 distinct cuts induce a partition of T into p subtrees, it is more convenient to add an artiÿcial cut so that the number of cuts will be equal to the number of subtrees. Thus, we augment the original tree T by an artiÿcial node v 0 , a super root, by connecting it to v 1 with an artiÿcial edge of unit length. We will assume without loss of generality that we select p cut points and one of the cuts must be the [v 1 ; v 0 ]-cut. These p cuts induce p subtrees on the original tree T . (The artiÿcial edge takes no part in the partition of the original tree T .) Let p be a positive integer, and let X p be a set of p distinct cut points, such that no node v i is the marker of more than deg(v i )−1 boundary cuts in X p . X p is called a p-cut. It deÿnes a continuous edge-partition of A(T ) into p closed connected components (subtrees) of positive length, {T 1 (X p ); : : : ; T p (X p )}. The intersection of each pair of components is either empty or consists of exactly one point, which is either a marker or an interior point of an edge. For convenience we will deÿne a one-to-one correspondence between the cut points and the subtrees. Each subtree T j (X p ), j = 1; : : : ; p, will correspond to the (unique) closest point to the root v 1 in T j (X p ) ∩ X p . We will also say that this cut point determines or induces the subtree T j (X p ).. Note that an edge-partition is a partition with the additional following property. If two distinct subtrees of the partition have a common node, then this node is a leaf of at least one of the subtrees.. In this paper we study bottleneck optimization problems deÿned by p-cuts. Specifically, for each p-cut X p we consider the lengths of the components, {l(T 1 (X p )); : : : ; l(T p (X p ))}.. In the continuous max-min tree length edge-partitioning problem, the objective is to ÿnd a p-cut X p maximizing l Min(X p ) = min{l(T 1 (X p )); : : : ; l(T p (X p ))}:. In the continuous min-max tree length edge-partitioning problem, the objective is to ÿnd a p-cut X p minimizing l Max(X p ) = max{l(T 1 (X p )); : : : ; l(T p (X p ))}:. As mentioned above, the continuous max-min tree length edge-partitioning problem has been recently discussed by Becker et al. [4] [5] [6] . Assuming rational data, the authors present O(n 2 p 2 +np 3 ) algorithm to solve the model. We are unaware of papers dealing with the continuous min-max tree length edge-partitioning problem.. We now deÿne the related problems involving the diameters of the subtrees in the partition, {d(T 1 (X p )); : : : ; d(T p (X p ))}.. In the continuous min-max tree diameter edge-partitioning problem, the objective is to ÿnd a p-cut X p minimizing. This is the continuous p-center problem on a tree. (Using location theory terminology it corresponds to the case where both the demand set and the supply set are equal to A(T ) [9] .) Polynomial algorithms solving this problem appear in [8, 9, 20] . The algorithm with the lowest known complexity is the O(n log 3 n) procedure by Megiddo and Tamir [20] . The complexity can be further improved to O(n log 2 n) by using the modiÿcation in Cole (1987) [10] . Note that the continuous p-center problem is actually deÿned as the partitioning (not necessarily edge-partitioning) problem, minimizing the maximum of the diameters of the components. However, it can easily be shown that for this model the two versions coincide.. In the continuous max-min tree diameter edge-partitioning problem, the objective is to ÿnd a p-cut X p maximizing. We also consider the partitioning version of the model, the continuous min-max tree diameter partitioning problem, where we look for a partition (not necessarily edge-partition) maximizing the minimum of the diameters. The two versions are not identical. We present O(n log 2 n) and O(n log 3 n) algorithms for the edge-partitioning and the partitioning versions respectively.. Remark. We note that due to the nature of the above four bottleneck objective functions, the maxima of l Min(X p ) and d Min(X p ), as well as the minima of l Max(X p ) and d Max(X p ) do exist. (This follows from the fact that even if we allow X p to be a multiset, or contain a node v i which is a marker of deg(v i ) boundary cuts, there is always a p-cut X p which dominates X p .). For convenience we summarize the notation introduced above that will be used throughout the paper.

O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


The

Price

of

Resiliency:

a

Case

Study

on

Sorting

with

Memory

Faults

We

address

the

problem

of

sorting

in

the

presence

of

faults

that

may

arbitrarily

corrupt

memory

locations,

and

investigate

the

impact

of

memory

faults

both

on

the

correctness

and

on

the

running

times

of

mergesort-based

algorithms.

To

achieve

this

goal,

we

develop

a

software

testbed

that

simulates

different

fault

injection

strategies,

and

perform

a

thorough

experimental

study

using

a

combination

of

several

fault

parameters.

Our

experiments

give

evidence

that

simple-minded

approaches

to

this

problem

are

largely

impractical,

while

the

design

of

more

sophisticated

resilient

algorithms

seems

really

worth

the

effort.

Another

contribution

of

our

computational

study

is

a

carefully

engineered

implementation

of

a

resilient

sorting

algorithm,

which

appears

robust

to

different

memory

fault

patterns.

A

standard

assumption

in

the

design

and

analysis

of

algorithms

is

that

the

contents

of

memory

locations

do

not

change

throughout

the

algorithm

execution

unless

they

are

explicitly

written

by

the

algorithm

itself.

This

assumption,

however,

may

not

necessarily

hold

for

very

large

and

inexpensive

memories

used

in

modern

computing

platforms.

The

trend

observed

in

the

design

of

today's

memory

technologies,

in

fact,

is

to

avoid

the

use

of

sophisticated

error

checking

and

correction

circuitry

that

would

impose

non-negligible

costs

in

terms

of

both

performance

and

money:

as

a

consequence,

memories

may

be

quite

error-prone

and

can

be

responsible

of

silent

data

corruptions.

Memory

failures

are

usually

classified

as

either

hard

errors

(i.e.,

permanent

physical

defects

whose

repair

requires

component

replacement),

or

soft

errors

(i.e.,

transient

faults

in

semiconductor

devices

and

recoverable

errors

in

disks

and

other

devices)

Soft

errors

are

a

particularly

big

concern

in

the

reliability

of

storage

systems

as

recently

discussed,

e.g.,

in

Such

errors

have

several

origins

and

exhibit

complicated

patterns.

Hardware

or

power

failures,

as

well

as

environmental

conditions

such

as

cosmic

rays

and

alpha

particles,

can

temporarily

affect

the

behavior

of

semiconductor

devices

resulting

in

unpredictable,

random,

independent

bit

flips.

According

to

memory

vendors

(see,

e.g.,

the

standard

soft

error

rates

at

sea

level

for

modern

memory

devices

are

between

and

failures

in

time

per

Mbit

(in

short,

FIT):

this

means

that

a

memory

chip

of

Mbit

is

likely

to

experience

between

and

faults

every

hours

of

use.

We

remark

that

the

intensity

of

cosmic

radiation

depends

strongly

on

altitude,

and

thus

the

failure

rate

considerably

increases

with

altitude,

being

even

two

orders

of

magnitude

larger

at

meters

Figure

illustrates

the

mean

time

between

failures

for

computing

platforms

with

different

memory

sizes

using

currently

available

technologies

(at

sea

level).

The

mean

time

between

failures

has

been

obtained

as

/(FIT

memory

size

in

Mbit)

and

gives

the

expected

number

of

hours

between

two

faults

as

a

function

of

the

memory

size

and

of

the

FIT

rate.

As

shown

by

the

chart,

this

number

decreases

as

the

memory

size

becomes

larger:

for

instance,

a

system

with

Terabytes

of

memory,

such

as

a

large

cluster

of

computing

platforms

with

a

few

Gigabytes

per

node,

is

likely

to

experience

one

bit

flip

every

few

minutes.

Multiple

bit

errors

can

also

occur

at

different

levels

of

the

memory

hierarchy.

For

instance,

a

systematic

analysis

of

data

corruption

recently

conducted

in

the

CERN

computer

center

has

shown

that

disks

can

experience

transient

errors

due

to

environmental

factors,

such

as

high

temperature,

and

to

usage

activities,

and

that

a

single

error

can

often

determine

the

corruption

of

large

regions

of

data,

up

to

kB.

This

can

be

a

serious

problem

for

algorithms

that

work

by

copying

data

from

disk

to

main

memory,

as

happens

in

all

those

applications

designed

to

cope

with

large

data

sets..

In

the

design

of

reliable

systems,

when

specific

hardware

for

fault

detection

is

not

available,

it

makes

sense

to

assume

that

the

algorithms

themselves

are

in

charge

of

dealing

with

memory

faults.

Informally,

we

say

that

an

algorithm

is

resilient

to

memory

faults

if,

despite

the

corruption

of

some

memory

values

before

or

during

its

execution,

the

algorithm

is

nevertheless

able

to

get

a

correct

output

at

least

on

the

set

of

uncorrupted

values.

We

note

that

classical

algorithms

are

typically

non-resilient:

if

an

algorithm

is

not

prepared

to

cope

with

memory

faults,

it

may

take

wrong

steps

upon

reading

corrupted

values

and

errors

may

propagate

over

throughout

its

execution.

Designing

resilient

algorithms

seems

important

in

a

variety

of

different

domains.

Many

large-scale

applications

that

require

the

processing

of

massive

data

sets

demand

large

memory

capacities

at

low

cost:

in

these

cases

even

very

few

memory

faults

may

jeopardize

the

correctness

of

the

underlying

algorithms.

For

instance,

the

inverted

indices

used

by

Web

search

engines

are

typically

maintained

sorted

for

fast

document

access:

for

such

large

data

structures,

even

a

small

failure

probability

can

result

in

bit

flips

in

the

index,

that

may

become

responsible

of

erroneous

answers

to

keyword

searches.

Similar

phenomena

have

been

observed

in

practice

Another

application

domain

is

related

to

avionics

systems,

which

make

typically

use

of

embedded

software

with

strong

reliability

concerns:

as

observed

above,

in

this

case

memories

may

incur

rather

large

numbers

of

faults,

that

can

seriously

compromise

safety

critical

applications.

Finally,

in

fault-based

cryptanalysis

some

optical

and

electromagnetic

perturbation

attacks

work

by

manipulating

the

non-volatile

memories

of

cryptographic

devices,

so

as

to

induce

very

timing-precise

controlled

faults

on

given

individual

bits:

this

forces

the

devices

to

output

wrong

ciphertexts

that

may

allow

the

attacker

to

determine

the

secret

keys

used

during

the

encryption.

Induced

memory

errors

have

been

effectively

used

in

order

to

break

cryptographic

protocols

smart

cards

and

other

security

processors

and

to

take

control

over

a

Java

Virtual

Machine

In

this

context

the

errors

are

introduced

by

a

malicious

adversary,

which

can

assume

some

knowledge

of

the

algorithm's

behavior.

We

remark

that,

in

all

the

above

applications,

both

the

faults

rate

and

the

faults

patterns

(i.e.,

random

or

adversarial

faults,

single

or

multiple

corruptions)

can

be

very

different.

In

this

paper

we

have

addressed

the

problem

of

sorting

in

the

presence

of

faults

that

may

arbitrarily

corrupt

memory

locations.

We

have

experimentallyinvestigated

the

impact

of

memory

faults

both

on

the

correctness

and

on

the

running

times

of

mergesort-based

sorting

algorithms,

including

the

algorithms

FAST

and

OPT

presented

in

a

naive

approach,

and

a

carefully

engineered

version

of

OPT,

named

OPT-NB,

introduced

in

this

paper.

We

have

performed

experiments

using

a

variety

of

fault

injection

strategies

and

different

instance

families.

The

full

software

package

used

in

our

study

is

publicly

available

at

the

URL:

Our

experiments

give

evidence

that

simple-minded

approaches

to

the

resilient

sorting

problem

are

largely

impractical,

while

the

design

of

more

sophisticated

algorithms

seems

worth

the

effort:

the

algorithms

presented

in

are

not

only

theoretically

efficient,

but

also

fast

in

practice.

In

particular,

our

engineered

implementation

of

OPT

is

robust

to

different

memory

fault

patterns

and

appears

to

be

the

algorithm

of

choice

for

most

parameter

choices.

Algorithm

FAST,

however,

might

be

preferable

for

rather

small

values

of

δ..

The

model

in

which

the

algorithms

under

investigation

have

been

designed

assumes

that

there

exist

an

upper

bound

δ

on

the

maximum

number

of

faults

that

may

happen

throughout

the

execution

of

the

algorithm.

However,

in

practice

algorithms

with

larger

execution

times

are

likely

to

incur

a

larger

number

of

memory

faults.

We

have

thus

adapted

the

algorithms

to

work

in

a

different

faults

per

unit

time

per

unit

memory

model.

Even

in

this

more

realistic

scenario,

we

have

observed

the

same

relative

performances

of

the

algorithms:

OPT

and

OPT-NB

appear

to

be

more

robust

than

FAST

and

can

tolerate

higher

fault

rates..

The

design

and

the

theoretical

analysis

of

resilient

algorithms

in

the

faults

per

unit

time

per

unit

memory

model

certainly

deserve

additional

investigation.

In

particular,

it

would

be

interesting

to

understand

whether

it

is

possible

to

obtain

fault-oblivious

resilient

algorithms,

i.e.,

resilient

algorithms

that

do

not

assume

any

knowledge

on

the

maximum

number

δ

of

memory

faults.

In

the

problem

of

designing

resilient

dictionaries

and

priority

queues

has

been

also

addressed:

an

experimental

study

of

the

performances

of

these

resilient

data

structures

would

represent

a

valuable

contribution.

Since

the

pioneering

work

of

von

Neumann

in

the

late

the

problem

of

computing

with

unreliable

information

has

been

investigated

in

a

variety

of

different

settings,

including

the

liar

model

fault-tolerant

sorting

networks

resiliency

of

pointer-based

data

structures

parallel

models

of

computation

with

faulty

memories

We

refer

the

interested

reader

to

and

for

an

overview.

In

Finocchi

and

Italiano

introduced

a

faulty-memory

random

access

machine,

i.e.,

a

random

access

machine

whose

memory

locations

may

suffer

from

memory

faults.

In

this

model,

an

adaptive

adversary

may

corrupt

up

to

δ

memory

words

throughout

the

execution

of

an

algorithm.

We

remark

that

δ

is

not

a

constant,

but

a

parameter

of

the

model.

The

adaptive

adversary

captures

situations

like

cosmic-rays

bursts,

memories

with

non-uniform

fault-probability,

and

hackers'

attacks

which

would

be

difficult

to

be

modeled

otherwise.

The

algorithm

cannot

distinguish

corrupted

values

from

correct

ones

and

can

exploit

only

safe

memory

words,

whose

content

never

gets

corrupted.

The

last

assumption

is

not

very

restrictive,

since

one

can

usually

afford

the

cost

of

storing

a

small

number

of

running

variables

in

a

more

expensive

and

more

reliable

memory

of

constant

size.

In

Finocchi

et

al.

presented

matching

upper

and

lower

bounds

for

resilient

sorting

and

searching

in

this

faulty-memory

model..

Let

n

be

the

number

of

keys

to

be

sorted.

In

Finocchi

and

Italiano

proved

that

any

resilient

O(n

log

n)

comparison-based

deterministic

algorithm

can

tolerate

the

corruption

of

at

most

O(

n

log

n

keys.

They

also

proved

that

one

can

sort

resiliently

in

O(n

log

n

δ

time:

this

yields

an

algorithm

(FAST)

whose

running

time

is

optimal

in

the

comparison

model

as

long

as

δ

O((n

log

n)

In

Finocchi

et

al.

closed

the

gap

between

the

upper

and

the

lower

bound,

designing

a

resilient

sorting

algorithm

(OPT)

with

running

time

O(n

log

n

δ

We

note

that

both

FAST

and

OPT

pay

only

an

additive

overhead

in

their

running

times

in

order

to

cope

with

memory

faults:

this

seems

theoretically

much

better

than

a

simple-minded

approach,

that

would

produce

a

multiplicative

overhead

of

(δ)

in

the

running

times..

Our

Results

In

this

paper

we

perform

a

thorough

experimental

evaluation

of

the

resilient

sorting

algorithms

presented

in

along

with

a

carefully

engineered

version

of

OPT

(named

OPT-NB).

In

order

to

study

the

impact

of

memory

faults

on

the

correctness

and

the

running

time

of

sorting

algorithms,

we

implemented

a

software

testbed

that

simulates

different

fault

injection

strategies.

Our

testbed

allows

us

to

control

the

number

of

faults

to

be

injected,

the

memory

location

to

be

altered,

and

the

fault

generation

time.

Since

our

study

is

not

tied

to

a

specific

application

domain,

and

since

fault

rate

and

fault

patterns

can

be

very

different

in

different

applications,

we

performed

experiments

using

a

variety

of

combinations

of

these

parameters

and

different

instance

families.

We

tried

to

determine,

for

instance,

for

which

values

of

δ

a

naive

approach

is

preferable

to

more

sophisticated

algorithms,

and,

similarly,

if

there

are

any

values

of

δ

for

which

algorithm

FAST

is

preferable

to

OPT

in

spite

of

the

theoretical

bounds

on

the

running

times..

As

a

first

contribution,

we

show

experimentally

that

even

very

few

memory

faults

that

hit

random

memory

locations

can

make

the

sequence

produced

by

a

non-resilient

sorting

algorithm

completely

disordered:

this

stresses

the

need

of

taking

care

explicitly

of

memory

faults

in

the

algorithm

implementation.

We

next

evaluate

the

running

time

overhead

of

FAST,

OPT,

and

OPT-NB.

Our

main

findings

can

be

summarized

as

follows..

A

simple-minded

approach

to

resiliency

is

largely

impractical:

it

yields

an

algorithm

(NAIVE)

which

may

be

up

to

hundreds

of

times

slower

than

its

non-resilient

counterpart.

We

remark

that

NAIVE

turned

out

to

be

very

slow

even

for

the

smallest

values

of

δ

used

in

our

experiments,

i.e.,

δ

or

The

design

of

more

sophisticated

resilient

algorithms

seems

worth

the

effort:

FAST,

OPT

and

OPT-NB

are

always

much

faster

than

NAIVE

(even

for

small

δ)

and

get

close

to

the

running

time

of

non-resilient

sorting

algorithms.

In

particular,

OPT-NB

is

typically

at

most

times

slower

than

its

non-resilient

counterpart

for

the

parameter

settings

considered

in

our

experiments..

Despite

the

theoretical

bounds,

FAST

can

be

superior

to

OPT

in

case

of

a

small

number

of

faults:

this

suggests

that

OPT

has

larger

implementation

constants.

However,

unlike

OPT,

the

performance

of

FAST

degrades

quickly

as

the

number

of

faults

becomes

larger.

In

particular,

the

experiments

suggest

that

for

large

values

of

δ

the

theoretical

analyses

of

both

FAST

and

OPT

predict

rather

accurately

their

practical

performances..

The

time

interval

in

which

faults

happen

may

influence

significantly

the

running

times

of

FAST,

while

seems

to

have

a

negligible

effect

on

the

running

times

of

OPT

and

OPT-NB..

Our

engineered

implementation

OPT-NB

typically

outperforms

its

competitors

and

seems

to

be

the

algorithm

of

choice

for

resilient

sorting

for

moderate

and

large

values

of

δ.

FAST

may

be

still

preferable

when

δ

is

rather

small..

All

the

algorithms

under

investigation

make

explicit

use

of

an

upper

bound

δ

on

the

number

of

faults

in

order

to

guarantee

correctness.

Since

it

is

not

always

possible

to

know

in

advance

the

number

of

memory

faults

that

will

occur

during

the

algorithm

execution,

we

analyzed

the

sensitivity

of

the

algorithms

with

respect

to

variations

of

δ,

showing

that

rounding

up

δ

(in

absence

of

a

good

estimate)

does

not

affect

significantly

the

performances

of

OPT

and

OPT-NB

even

when

δ

is

rather

large.

Finally,

we

considered

a

more

realistic

scenario,

called

faults

per

unit

time

per

unit

memory

model,

where

algorithms

with

larger

space

consumption

and

larger

execution

times

are

likely

to

incur

a

larger

number

of

memory

faults.

Even

in

this

scenario,

we

observed

the

same

relative

performances

of

the

algorithms:

OPT

and

OPT-NB

appear

to

be

more

robust

than

FAST

and

can

tolerate

higher

fault

rates.

However,

given

an

error

rate

σ

(per

unit

memory

and

per

unit

time),

for

all

the

resilient

algorithms

there

exists

an

upper

bound

on

the

largest

instance

that

can

be

faithfully

sorted

in

the

presence

of

faults

occurring

with

rate

σ

Organization

of

the

Paper

The

remainder

of

this

paper

is

organized

as

follows.

Section

describes

the

mergesort-based

resilient

algorithms

under

investigation

and

discusses

some

implementation

issues.

Section

presents

our

experimental

framework,

focusing

on

fault

injection

strategies

and

performance

indicators.

Our

experimental

findings

are

summarized

in

Sect.

and

concluding

remarks

are

listed

in

Sect.

Our

experiments

have

been

carried

out

on

a

workstation

equipped

with

two

Opteron

processors

with

GHz

clock

rate

and

bit

address

space,

GB

RAM,

MB

cache,

and

kB

data/instruction

cache.

The

workstation

runs

Linux

Kernel

All

programs

have

been

compiled

through

the

GNU

gcc

compiler

version

with

optimization

level

The

full

package,

including

algorithm

implementations,

memory

manager,

and

a

test

program,

is

publicly

available

at

the

URL:

Unless

stated

otherwise,

in

our

experiments

we

average

each

data

point

on

ten

different

instances,

and,

for

each

instance,

on

five

runs

using

different

fault

sequences

on

randomly

chosen

memory

locations.

Random

values

are

produced

by

the

rand()

pseudo-random

source

of

numbers

provided

by

the

ANSI

C

standard

library.

The

running

time

of

each

experiment

is

measured

by

means

of

the

standard

system

call

getrusage().

The

sorting

and

the

corrupting

threads

run

as

two

different

parallel

processes

on

our

biprocessor

architecture

and

operating

system:

concurrent

accesses

to

shared

memory

locations

(e.g.,

the

corruption

of

a

key

in

unreliable

memory)

are

solved

at

the

hardware

level

by

spending

only

a

few

CPU

cycles.

This

allows

us

to

get

a

confident

measure

of

the

algorithms'

running

time,

without

taking

into

account

also

the

time

spent

for

injecting

faults.

O O O O O O O O B O B I O O O O O B O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Hop-by-hop quality of service routing Based on the self-adaptive multiple constraints routing algorithm (SAMCRA), an improved and exact version of our QoS routing algorithm, the tunable accuracy multiple constraints routing algorithm (TAMCRA) we have investigated QoS routing in a hop-by-hop manner because it forms the basis of IP networking as e.g. in OSPF. In particular, we studied`hop-by-hop destination based only' (HbHDBO) QoS routing that ignores the source and previous path history (as in current IP routing). We demonstrate that an exact QoS algorithm assures the avoidance of routing loops in this HbHDBO setting. However, despite the use of an exact QoS routing algorithm as SAMCRA, the exact solution cannot be guaranteed with HbHDBO routing. Fortunately, large simulation results on various sizes of random graphs show that the overall quality of the HbHDBO QoS routing is remarkably good. Finally, we show that, by using active networking as opposed to current IP routing, exact QoS routing in a hop-by-hop way can be guaranteed. Ó We have investigated the problem of connectionless QoS routing. To limit the number of routing entries, we have investigated the possibility of forwarding packets based on the destination only. We have ®rst demonstrated that an exact algorithm as SAMCRA is loop free when used in hop-by-hop mode. Moreover, with a non-linear path length de®nition, an exact algorithm is not only sucient but also necessary to avoid loops. We have further shown that``hop-by-hop destination based only'' QoS routing, even with an exact routing algorithm (and contrary to`single metric routing' as in OSPF), does not guarantee that the exact end-to-end solution is found. Extensive simulations in the class of RGU have shown that deviations from the exact results are small and occur infrequently. Moreover, the maximum relative dierence in length can be bounded.. The intractability of exact QoS routing with multiple additive link measure (due to NP-completeness) is only one of the complicating factors in the QoS routing problem. Indeed, topologically SAMCRA does not scale much worse than Dijkstra. Moreover, simulations in RGU (up to N 1600 nodes) indicate that the minimum queue size needed to retrieve the exact result is much smaller than the worst case prediction since k min ( k max . In spite random graphs do not resemble actual communication networks, we believe that future topologies due to the increased importance of wireless and ad hoc (or imbedded) networks, may possess a larger degree of randomness both in topological structure as in the link weights. However, mainly the granularity of the constraints as exhibited by expression (2) for k max determines the complexity. Hence, a careful design of QoS routing (thus both algorithm and protocol) may bene®t from a coarse granularity in the constraints (i.e. decreasing k max ). Also the dynamic behavior of QoS routing (not considered here, but see Ref. [22] ) simpli®es as the number of topology updates triggered by signi®cant changes in the link metrics diminishes accordingly. Thus, we believe that SAMCRA together with a minimum acceptable k max , is a promising strategy to provide a notion (hard guarantees are shown to be impossible) of QoS in the hop-by-hop Internet of today.. Finally, if the current Internet mode of HbHDBO routing (ignoring the past path history) is relaxed, at the expense of more computations, hop-by-hop routing with active packets can be exact provided the length vector of the path traversed so far and the constraint vector is stored in each packet. This observation is directly veri®ed from the meta-code of SAMCRA. This is an important corollary that follows from non-linear path length de®nitions as Eq. (1) A:3. The proof relies on the inequality (A.1). Consider two paths P 1 and P 2 for which l(P 1 ) < l(P 2 ) and assume that, by adding a same link a to both paths, the paths P 3 and P 4 can be constructed.. 1. Let us ®rst focus on the case where the equality sign in Eq. (A.1) holds, typically if q 1 in Eq. (A.3). By construction, we have lP 3 lP 1 a and by Eq. (A.1) with the equality sign, lP 1 a lP 1 la and analogously, lP 4 lP 2 a lP 2 la: Since lP 1 < lP 2 , there holds that lP 3 < lP 4 or, the subpaths of shortest path with linear de®nition of path length are again shortest paths, leading to the well-known and intuitive result.. 2. When the inequality sign holds in Eq. (A.1), typically if q > 1 as readily veri®ed from Eq. (A.3), we arrive in a similar fashion to the set of inequalities lP 4 lP 2 a < lP 2 la; lP 3 lP 1 a < lP 1 la and lP 3 < lP 4 : However, from this set, it cannot be concluded whether lP 3 T lP 4 or lP 3 > lP 4 : It suces to show that the latter situation may exist in order to prove the corollary. This can be illustrated on the example graph of Fig. 7 . For the constraints (14, 11, 22) , the shortest path from node a to node i runs over node c, e and f as shown in Fig. 8 . According to the de®nition (1), the path length of P(i, f, e, c, a) equals 0.95 which means it satis®es all constraints. However, as illustrated in Fig. 9 , the shortest path from node a to node e does not traverse node c but node b. Ã

O B I I B O O O O O O B O O O O O O O O O O B O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Reducing Reversible Circuit Cost by Adding Lines Abstract-Additional lines are required to implement an irreversible function as a reversible circuit. The emphasis, particularly in automated synthesis methods, has been on using the minimal number of additional lines. In this paper, we show that circuit cost reductions can be achieved by adding additional lines. We present an algorithm for line addition that can be targeted to reducing the quantum cost of a circuit or the transistor count for a CMOS implementation. Experimental results show that the cost reduction can be significant even if (1) only a small number of lines (even one) is added and (2) other circuit optimizations have already been applied. The amazing developments in microelectronic technology and the looming limitations are well known and well documented. Both, transistor density and power dissipation are crucial issues for designing current high performance digital circuits. For these reasons, researchers expect that in 10-20 years current technologies will reach their limits (see e.g. [1] ). It is thus important to consider alternative technologies. Reversible logic offers one such alternative due to its applications in the domains of low-power design, quantum computation, and bio-computing.. Landauer and Bennett showed in [2] , [3] , that power dissipation is not only caused by non-ideal behavior of transistors and materials, but also by a more fundamental reason. Each time information is erased (e.g. when the irreversible AND operation is computed), energy is dissipated as well. Only if computation is information-lossless, can zero energy dissipation be achieved. This holds for reversible logic since only bijective operations are allowed (i.e. each input pattern is mapped to a unique output pattern). Practical reversible circuits have been built that exploit these properties [4] . In fact, these circuits were powered by their input signals only and did not need additional power lines.. Quantum computation is of interest, since it potentially allows exponential speed-up of computation for many important problems. This is because qubits, in contrast to traditional bits, can assume a probabilistic superposition of the base states 0 and 1. As a result, a set of qubits can represent multiple states at the same time enabling enormous computational speed-up. Quantum circuits have been developed working with 8, 16 , and, recently, 28 qubits (see e.g. [5] ). Reversible logic acts as a framework for quantum computation since all quantum gate operations are inherently reversible.. For the reasons cited above, reversible circuits and logic synthesis have in recent years become well studied topics. Many synthesis approaches for reversible logic have been proposed (e.g. [6] - [15] ). Except for very small cases, these methods do not produce minimal results. Thus, post-synthesis optimization is applied to reduce the cost of a circuit.. For example, template matching [11] , [16] , [17] is a search method which looks for gate sequences that can be replaced by alternative cascades of lower cost. The run-time increases with both the number of applied templates and the number of gates in the circuit and as a result can be quite high. But for many circuits substantial improvement are achieved.. As a second example, Zhong and Muzio [18] showed how analyzing cross-point faults can identify redundant control connections in reversible circuits. Removing such control lines reduces the cost of the circuit. However, the computation needed to determine such redundancies is extremely high.. In this paper, we propose a post-synthesis optimization technique which reduces the cost of the circuit by adding further signal lines to the circuit. The general idea is to use the new lines for "buffering" factors of gate control lines so that they can be reused by the other gates in the circuit. This reduces the size of these gates and thus decreases the cost of the circuit. Both, quantum cost (used in quantum circuits) and transistor count (used in CMOS implementations) are considered.. A fast algorithm is presented along with results that show it can be quite effective even if only a small number of lines, even 1, are added. Since we consider the addition of lines to a circuit (not to the functional specification), our approach is applicable to circuits designed by any automated synthesis method and in fact to circuits designed by hand.. Experiments show that applying our approach circuit cost can be reduced by up to 70% with adding only a single line. Thus, even for quantum realizations (where qubits and hence the number of circuit lines are limited) adding a further line is beneficial as it reduces the quantum cost significantly. So, the designer can trade off the additional expenses of a further circuit line for a significant reduction of quantum cost. For CMOS realizations, the observed reductions are more moderate. However, here the cost of adding a new line to the circuit is negligible, so that the proposed optimization is worthwhile for such circuits as well.. In previous work, it has already been observed that more circuit lines usually lead to lower (quantum) cost (see e.g. [19] or recently [20] ). Moreover, the authors of [21] even showed that some functions cannot be synthesized for certain gate libraries unless one additional line is added. However, in this paper these observations are exploited for the first time by proposing a constructive post-synthesis optimization approach for reversible logic.. The remainder of this paper is structured as follows: Section II provides the necessary background on reversible circuits for this paper. The general idea of the proposed approach is introduced in Section III while Section IV describes the algo-rithm in detail. Experimental results are reported in Section V and the paper concludes with observations and suggestions for further research in Section VI.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Assessment of the critical dimension prediction accuracy for the lumped parameter model for resist lithography Lithography modelling is widely used to predict the critical dimensions (CD) of patterned features after lithographic processing. A lot of full and simplified resist models are available. Previous works on full resist models have shown that the numerous model parameters are very difficult to set and often have a poor range of validity outside the dataset that have been used to generate them [Proc. SPIE 3678 (1999) 877; Proc. SPIE 4404 (2001) 99]. Simplified resist models are an alternative solution, easier to set, and they often provide a good simulation accuracy [Proc. SPIE 4691 (2002) 1266; Proc. SPIE 5040 (2003) 1536]. Among simplified models, lumped parameter model (LPM) is widely used for CD predictions. In this paper, we study the CD prediction accuracy of the LPM, using a comparison between experimental and simulated data. A systematic method is applied for LPM parameters extraction (contrast γ and effective thickness D eff ). This assessment shows that a single parameter set giving reasonable accuracy is not found. Moreover, the critical analysis of the model parameters shows that these LPM parameters have a poor physical meaning. We also point out that there is a fundamental disagreement between the LPM theory and experiments.

O O O B I O O O O B I I O O B B O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O B O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


INFERENCE OF EDGE REPLACEMENT GRAPH GRAMMARS We describe an algorithm and experiments for inference of edge replacement graph grammars. This method generates candidate recursive graph grammar productions based on isomorphic subgraphs which overlap by two nodes. If there is no edge between the two overlapping nodes, the method generates a recursive graph grammar production with a virtual edge. We guide the search for the graph grammar based on the size of the grammar and the portion of the graph described by the grammar. We show experiments where we generate graphs from known graph grammars, use our method to infer the grammar from the generated graphs, and then measure the error between the original and inferred grammars. Experiments show that the method performs well on several types of grammars, and specifically that error decreases with increased numbers of unique labels in the graph. There is overlap in the recurring patterns or motifs representing the building blocks of networks in nature. Palla et al. 12 point out the existence of an overlap between parts of graphs representing social networks and proteins. They call them overlapping communities. Most knowledge discovery and data mining approaches look for independent recurring patterns, but do not consider how these patterns can connect and overlap iteratively or recursively to generate arbitrary-sized relational data. Graph grammars provide a representation for such knowledge. In our method of graph grammar inference we search for overlap between isomorphic subgraphs of a graph. The overlap allows our method to propose recursive graphgrammar productions. The first approach was to search for overlap by a single node, which led to developing a system for inference of Node Replacement Recursive Graph Grammars 8 . In this paper we describe an approach that allows inference of Edge graph grammars introducing operations on a set of undirected, unlabeled graphs which guarantee that the inferred grammar can generate the input set of graphs. Oates, Doshi, and Huang 14 assume that the structure of a graph of a hyperedge replacement context free graph grammar is given. They are interested in inference of probabilities associated with every rule of a grammar. Nevill-Manning and Witten 11 developed SEQUITUR which works on strings, but their approach is similar to ours in the sense that it infers hierarchical structure by replacing substrings by grammar rules. The new, compressed string is searched for substrings which can be described by grammar rules, and they are then compressed with the grammar and the process continues iteratively. Similarly, in our approach we replace the part of a graph described by the inferred graph grammar with a single node, and we look for grammar rules on the compressed graph and repeat this process iteratively until the graph is fully compressed. Jonyer, Holder, and Cook 5, 6 developed an approach to infer node replacement graph grammars which describe graphs in the from of 'chains', where isomorphic copies of subgraphs are adjacent to each other in a chain connected by a single edge. Their algorithm starts by finding frequently occurring subgraphs in the input graphs. Frequent subgraphs are those that when replaced by single nodes minimize the description length of the graph. They check if isomorphic instances of the subgraphs that minimize the measure are connected by one edge. If they are, a production S→ PS is proposed, where P is the frequent subgraph. P and S are connected by one edge. Our approach is similar to Jonyer's in that we also start by finding frequently occurring subgraphs, but we test if the instances of the subgraphs overlap by one node. Jonyer's method of testing if subgraphs are adjacent by one edge limits his grammars to descriptions of "chains" of isomorphic subgraphs connected by one edge. Since an edge of a frequent subgraph connecting it to the other isomorphic subgraph can be included to the subgraph structure, testing subgraphs for overlap allows us to propose a class of grammars that have more expressive power than the graph structures covered by Jonyer's grammars. For example, testing for overlap allows us to propose grammars that can describe tree structures, while Jonyer's approach does not allow for tree grammars. We conducted experiments with Jonyer's approach, called SubdueGL, to illustrate the types of graph grammars it can find and its limitations. We generated graphs from the grammar and then used SubdueGL to infer this grammar. We show our results in Figure 1 . In this figure, above the productions, we indicated a percentage that signifies the probability with which we are using every production. In Figure 1 we show a grammar that generates squares and We described an algorithm for inference of edge replacement graph grammars. The performance of the algorithm depends on the number of distinct labels in the input graph. If there is only one label, the algorithm finds a two edge grammar. If we use three or more labels in the input graph, the inference error drops to zero or to a value close to zero in inference of grammars with a graph structure of a tree, cycle, Peterson graph, and tetrahedron. However, as we complicate the structure and increase the average degree of nodes and the ratio of the number of edges to number of nodes, the error increases. The highest error we had is with a complete graph. The closer the graph structure of the grammar is to a complete graph, the more unique labels we need to use in the graph to achieve the same level of average inference error. If we generate graphs from a graph grammar and then add nodes and edges to this graph, it does not influence significantly the inference error in the range of noise 0 to 0.5. There is little influence on error from the number of non-terminal edges in the Peterson graph grammar structure when the number of non-terminals changes from 1 to 4. 15 In this paper we described the approach to graph grammar inference which extends the class of learnable graph grammars. Node Replacement Recursive Graph Grammar inference was limited to the patterns where instances overlap on exactly one node. In the approach presented in this paper allowing instances to overlap on two nodes led to the definition of real and virtual non-terminal edges. With this approach we can infer the grammar generating chains of squares overlapping on one edge which was not possible with node replacement grammars. Patterns often overlap on two nodes in chemical structures, as we saw in the example of the previous section; therefore, we have a tool which can find and represent important patterns in the chemical domain. In our experiments we generate thirty graphs from a known grammar, and then we infer the grammar from every generated graph. We compute the average inference error over these thirty examples. The generated graphs have 40 to 60 nodes. Our generator can assign a random label to a node or an edge. We compare the original grammar and inferred grammar using the following measure of the error: Our definition of an error has two aspects. First, there is the structural difference between the inferred and the original graph used in the productions. Second, there is the difference between the number of non-terminals and the number of connection instructions. If there is no error, the number of non-terminals in the original grammar is the same as the number of connection instructions in the inferred grammar. We would like our error to be a value between 0 and 1; therefore, we normalize the error by having in the denominator the sum of the size of the graph used in the original grammar and the number of non-terminals. We do not allow an error to be larger than 1; therefore, we take the minimum of 1 and our measure as a final value. The restriction that the error is not larger than 1 prohibits unnecessary influence on the average error by inferred graph structures significantly larger than the graph used in the original grammar. We now describe several experiments showing different aspects of the edge replacement graph grammar inference system.

O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


On hierarchical modelling of motion for workflow analysis from overhead view Abstract Understanding human behaviour is a high level perceptual problem, one which is often dominated by the contextual knowledge of the environment, and where concerns such as occlusion, scene clutter and high within-class variations are commonplace. Nonetheless, such understanding is highly desirable for automated visual surveillance. We consider this problem in a context of a workflow analysis within an industrial environment. The hierarchical nature of the workflow is exploited to split the problem into 'activity' and 'task' recognition. In this, sequences of low level activities are examined for instances of a task while the remainder are labelled as background. An initial prediction of activity is obtained using shape and motion based features of the moving blob of interest. A sequence of these activities is further adjusted by a probabilistic analysis of transitions between activities using hidden Markov models (HMMs). In task detection, HMMs are arranged to handle the activities within each task. Two separate HMMs for task and background compete for an incoming sequence of activities. Imagery derived from a camera mounted overhead the target scene has been chosen over the more conventional oblique views (from the side) as this view does not suffer from as much occlusion, and it poses a manageable detection and tracking problem while still retaining powerful cues as to the workflow patterns. We evaluate our approach both in activity and task detection on a challenging dataset of surveillance of human operators in a car manufacturing plant. The experimental results show that our hierarchical approach can automatically segment the timeline and spatially localize a series of predefined tasks that are performed to complete a workflow. Recently, there has been an increased interest in the area of human activity and behaviour recognition. While many different subsets of activities are investigated, the conditions and overall specifications of the problem vary largely. One of the applications of this is in visual surveillance. The goal here is to detect a set of pre-defined tasks which occur within a workflow of a complex industrial environment. Manufacturing [7, 9, 19, 20] and office [3, 10, 17] , environments, medical operating rooms [12] , TV studios [13] and smart homes for the elderly [8] are amongst the settings wherein 'action' perception and understanding from video streams had been studied. Pinhanez and Bobick [13] were among the first to analyse the actions within a video stream. They used this information to build an 'intelligent studio' wherein the cameras were controlled automatically based on a predefined script and the visual data from the cameras themselves. Considering the sequential nature of the patterns, using a hidden Markov model is a justified and popular choice. Padoy et al. [12] studied the workflows in a simulated operating theater. They introduced the Workflow-HMM which is a form of two-layered hierarchical HMM. Nguyen et al. [10] , who experimented in an office environment, also used hierarchical HMMs, however, movement trajectories are used there to learn and recognize actions. Behera et al. [3] have also exploited qualitative spatial information to inform action recognition. In this, all possible pair-wise spatial relations among objects are represented in a relational state space and a probabilistic Latent Semantic Analysis is used to model workflow from this relational state space. Voulodimos et al. [20] and Oliver et al. [11] used HMMs as a means for low level sensory data or high level decision fusion. Criticizing HMMs as requiring well-defined states and robust features, Veres et al. [19] opt for a neural network: they used an echo state network to learn the patterns within an annotated global scene descriptor. Shi et al. [17] proposed the use of propagation networks to handle parallel tracks within one action. Kosmopoulos et al. [7] incorporated user's feedback into the learning process using a neural networks based method to dynamically correct erroneous classifications. While on a slightly different note, Nater et al. [9] have proposed a novel approach for unsupervised discovery of tasks within a workflow. In this, assuming a temporal consistency and cyclic repeated patterns, they have used Slow Feature Analysis method to learn and extract invariant components in the temporal signal.. In these works, the modelling of workflows and the recognition of actions are addressed as well as temporal and spatial segmentation of the sequences. However, the recognition of actions is often addressed separately from the problem of sequence segmentation. Temporal segmentation, as in detecting the start and the end of a sequence which makes up an action, is also the focus of a related line of work which is devoted to unusual activity or unusual scene detection with the aim of detecting sequences that have not been previously observed [1, 5, 8] . In many approaches the sequence to be classified starts with a person entering the field of view and it ends when they exit this field. Thereby every movement is interpreted toward a meaningful behaviour. While such examples largely simplify the problem, this is not normally the case in real scenarios. In this paper, the imagery derived from a camera monitoring a working cell in a car manufacturing plant is analysed. In this, the human operators may be involved in multiple consecutive actions. On the other hand they might perform no predefined actions at all. The other point of concern is the spatial segmentation of actions. The complex and cluttered environment which is analysed in this paper, poses an overwhelming detection and tracking problem as was experiences by [7, 19, 20] . They have thus opted for a global scene descriptor and abandoned the problem of spatial segmentation.

O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O