Finding related pages in the World Wide Web When using traditional search engines, users have to formulate queries to describe their information need. This paper discusses a di erent approach t o w eb searching where the input to the search process is not a set of query terms, but instead is the URL of a page, and the output is a set of related web pages. A related web page is one that addresses the same topic as the original page. For example, www.washingtonpost.com is a page related to www.nytimes.com, since both are online newspapers.We describe two algorithms to identify related web pages. These algorithms use only the connectivity information in the web (i.e., the links between pages) and not the content of pages or usage information. We h a ve implemented both algorithms and measured their runtime performance. To e v aluate the e ectiveness of our algorithms, we performed a user study comparing our algorithms with Netscape's \What's Related" service 12]. Our study showed that the precision at 10 for our two algorithms are 73% better and 51% better than that of Netscape, despite the fact that Netscape uses both content and usage pattern information in addition to connectivity information. Traditional web search engines take a query as input and produce a set of (hopefully) relevant pages that match the query terms. While useful in many circumstances, search engines have the disadvantage that users have to formulate queries that specify their information need, which is prone to errors. This paper discusses how to nd related web pages, a di erent approach to web searching. In our approach the input to the search process is not a set of query terms, but the URL of a page, and the output is a set of related web pages. A related web page is one that addresses the same topic as the original page, but is not necessarily semantically identical. For example, given www.nytimes.com, t h e tool should nd other newspapers and news organizations on the web. Of course, in contrast to search engines, our approach requires that the user has already found a page of interest.. Recent work in information retrieval on the web has recognized that the hyperlink structure can be very valuable for locating information 18, 3, 7, 23, 19, 25, 24, 6, 17, 5] . This assumes that if there is a link from page v and w, then the author of v recommends page w, and links often connect related pages. In this paper, we describe the Companion and Cocitation algorithms, two algorithms which use only the hyperlink structure of the web to identify related web pages. For example, Table 1 Table 2 : Comparison of results for the Companion and Netscape algorithms. A \1" means that the page was valuable, a \0" means that the page was not valuable, a \{" means that the page could not be accessed.. input (in this case, the results for the Cocitation algorithm are identical and the results for Netscape are very similar, although this is not always true). One of our goals was to design algorithms with high precision that are very fast and that do not require a large number of di erent kinds of input data. Since we h a ve a tool that gives us access to the hyperlink structure of the web (the Connectivity Server 4]), we focused on algorithms that only use connectivity information to identify related pages. Our algorithms use only the information about the links that appear on each page and the order in which t h e links appear. They neither examine the content of pages, nor do they examine patterns of how users tend to navigate among pages.. Our Companion algorithm is derived from the HITS algorithm proposed by Kleinberg for ranking search engine queries 17]. Kleinberg suggested that the HITS algorithm could be used for nding related pages as well, and provided anecdotal evidence that it might work well. In this paper, we extend the algorithm to exploit not only links but also their order on a page (see Section 2.1.1) and present the results of a user-study showing that the resulting algorithm works very well.. The Cocitation algorithm nds pages that are frequently cocited with the input URL u (that is, it nds other pages that are pointed to by m a n y other pages that all also point t o u).. Netscape Communicator Version 4.06 introduced a related pages service that is built into the browser 12] (see Section 2.3 for a more detailed discussion). On the browser screen, there is a \What's Related" button, which presents a menu of related web pages in some cases. The \What's Related" algorithm in Netscape is based on technology developed by Alexa, Inc., and computes its answers based on connectivity information, content information, and usage information 11].. To compare the performance of our two algorithms and Netscape's algorithm, we performed a user study on 59 URLs chosen by 18 volunteers. Our study results show that the precision at 10 computed over all 59 URLs of our two algorithms are 73% better and 51% better than Netscape's. Not all algorithms gave answers for all URLs in our study. If we restrict the comparison to only the 37 URLs for which all three algorithms returned answers, then the precision at 10 of our two algorithms are 40% and 22% better than Netscape's algorithm. This is surprising since our algorithms are based only on connectivity information.. Netscape's algorithm gives answers for about 17 million URLs 11], while our algorithms can give a n s w ers for a much larger set of URLs (we h a ve connectivity information on 180 million URLs). This is important because it means that we c a n g i v e related URL information for more URLs. Our algorithms are also fast: in our environment, both average less than 200 msec of computation per input URL.. The example shown in Table 1 is for a URL with a very high level of connectivity ( www.nytimes.com contains 47,751 inlinks in our Connectivity S e r v er), and all three algorithms generally perform quite well for well-connected URLs. Our algorithms can also work well when there is much less connectivity, a s s h o wn by the example in Table 2 . This table shows the answers for the Companion and Netscape algorithms for linas.org/linux/corba.html, o n e of the input URLs chosen by a user as part of our user study. Alongside each answer is the user's rating for each answer, with a '1' meaning that the user considered the page related, '0' meaning that the user considered the page unrelated, and '{' meaning that the user was unable to access the page at all. The original page was about CORBA implementations for Linux, and there were 123 pages pointing to this page in our Connectivity Server. Nine of the ten answers given by the Companion algorithm were deemed related by our user, while only one page from Netscape's set of answers was deemed related. Most of Netscape's answers were about the much broader topic of Linux, rather than speci cally about CORBA implementations on Linux.. Section 2 presents our algorithms in detail and describes Netscape's service, while Section 3 discusses the implementation of our algorithms. Section 4 describes the user study we performed and presents its results, and also provides a brief performance evaluation of our algorithms. Finally, Section 6 presents our conclusions. We have presented two di erent algorithms for nding related pages in the WWW. They significantly outperform Netscape's algorithm for nding related pages. The algorithms can beimplemented e ciently and are suitable for use within a large scale web service providing a related pages feature.. Our two algorithms can be extended to handle more than one input URL. In this case, the algorithms would compute pages that are related to all input URLs. We are currently exploring these extensions.. to Hannes Marais for developing WebL, a web scripting language 16]. Using WebL, we w ere able to quickly develop a prototype user interface for our related pages server. Krishna Bharat, Allan Heydon, and Hannes Marais provided useful feedback on earlier drafts of this paper. Finally, we would like to thank all the participants in our user study. Many researchers have proposed schemes for using the hyperlink structure of the web 18, 3, 7, 23, 1 9 , 2 5 , 2 4 , 6 , 1 7 , 5 ] . For the most part, this work does not discuss the nding of related pages, with four exceptions discussed below.. We know of only one previous work that expoits the order of links: Chakrabarti et al. 9] use the links and their order to categorize web pages and they show that the links that are near a given link in page order frequently point to pages on the same topic.. Previous authors have suggested using cocitation and other forms of connectivity to identify related web pages. Spertus observed that cocitation could indicate that two pages are related 23]. That is, if page A points to both pages B and C, then B and C might be related. Various researchers in the eld of bibliometrics have also observed this 15, 13, 14, 22] , and this observation forms the basis of our Cocitation algorithm. The notion of collaborative ltering, although it is based on user's recommendations rather than hyperlinks, also relies on this observation 21]. Pitkow and Pirolli 19] cluster web pages based on co-citation analysis. Terveen and Hill 25] use the connectivity structure of the web to nd groups of related web sites.. Our companion algorithm descended from the HITS algorithm developed by Kleinberg 17] . The HITS algorithm was originally proposed by K l e i n berg as a way of using connectivity structure to identify the most authoritative sources of information on a particular topic, where the topic was de ned by the combined link structure of a large number of starting nodes on the topic. Kleinberg also proposed that the HITS algorithm could be used to nd related pages when the topic was de ned by just a single node. The Companion algorithm used HITS algorithm as a starting point and extended and modi ed it in four main ways:. 1. Kleinberg suggested using the following graph to nd related pages: Take a xed number (say 200) of parents of the given URL and call the set consisting of the URL and these parents the start set. Now build the graph consisting of all nodes pointing to a node in the start set or pointed to by a node in the start set. This means that \grandparents" of u are included in the graph, while nodes that share a child with u are not included in the graph. We believe that the latter nodes are more likely to berelated to u than are the \grandparent" nodes.. Therefore our vicinity graph is structured to exclude grandparent nodes but to include nodes that share a child with u. 2. We exploit the order of the links on a page to determine which \siblings" of u to include.. When we added this feature, the precision of our algorithm improved noticably. 3. The original HITS algorithm did not have edge weights. We use edge weights to reduce the in uence of pages that all reside on one host, since Bharat and Henzinger have shown that edge weights improve the precision 5]. 4. We also merge nodes in our vicinity g r a p h that have a large numberof duplicate links. Duplicate nodes are not such a serious problem when using the HITS or imp algorithms to rank query results, since the start set consists of a large number of URLs. However, when forming the vicinity graph starting with just a single URL, the in uence of duplicate nodes is increased because duplicate nodes with a large numberofout links will quickly dominate the hub and authority computation.. Kleinberg also showed that HITS computes the principal eigenvector of the matrix AA T , where A is the adjacency matrix of the above graph, and suggested using non-principal eigenvectors for nding related pages. Finally, h e g a ve anecdotal evidence that HITS might w ork well.. Consecutively, a sequence of papers 10, 8] presented improvements on HITS and used it to populate a given hierarchy of categories. These improvements are not directly relevant to the task of nding related pages. In experimenting with these algorithms, we w ere fortunate to have access to Compaq's Connectivity Server 4]. The Connectivity S e r v er provides high speed access to the graph structure of 180 million URLs (nodes) and the links (edges) that connect them. The entire graph structure is kept in memory on a Compaq AlphaServer 4100 with 8 GB of main memory and dual 466 MHz Alpha processors. The random access patterns engendered by the connectivity-based algorithms described in this paper mean that it is important for most or all of the graph to t in main memory to prevent high levels of paging activity.. We implemented a multi-threaded server that accepts a URL uses either the Cocitation algorithm or the Companion algorithm to nd pages related to the given URL. Our server implementation consists of approximately 5500 lines of commented C code, of which approximately 1000 lines implement the Companion algorithm, 400 lines implement the Cocitation algorithm, and the remainder are shared code to perform tasks such as parsing HTTP query requests, printing results, and logging status messages. We link our server code directly with the Connectivity Server library, and access the connectivity information by mmapping the graph information into the address space of our server.. Our implementation of the Companion algorithm has been subjected to a moderate amount of performance tuning, mostly in designing the neighborhood graph data structures to improve data-cache performance. The implementation of the Cocitation algorithm has not been tuned extensively, although it does share a fair amount of code with the Companion algorithm, and this shared code has been tuned somewhat. To compare the di erent a p p r o a c hes, we performed a user study. We asked 18 volunteers to supply us with at least two URLs for which t h e y w anted to nd related pages. Our volunteers included 14 computer science professionals (mostly our colleagues at Compaq's research laboratories), as well as 4 people with other professional careers. We received 69 URLs and used each of the algorithms to determine the top 10 answers for each URL. We put the answers in random order and returned them to the volunteers for rating. The volunteers were instructed as follows:. We want to measure how well each algorithm performs. To measure performance we want to know the percentage of valuable URLs returned b y e ach algorithm. To b e valuable the URL must be b oth relevant to the topic you are i n t e r ested in and a high quality page. For example, if your URL was www.audi.com and you get back a newsgroup posting where somebody talks about his new Audi car, then the page was on topic, but probably not high quality. On the other hand, if you get www.jaguar.com as an answer, then it is up to you to decide whether this answer is on topic or not. Scoring: Companion 50 498 42 Cocitation 58 580 62 Netscape 40 364 29 Table 3 : Summary of all answers for the algorithms 0: Page was not valuable/useful 1: Page was valuable/useful {: Page could not be a c cessed (i.e. did not exist, or server was down) Please ignore the order in which the pages are returned. So if a later page contains similar content to an earlier page please rate the latter page as if you had not seen the earlier page. This will imply that we do not measure how \happy" you are with a set of answers returned b y an algorithm. Instead we measure how many valuable answers each algorithm gives.

O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Information maximization in face processing This perspective paper explores principles of unsupervised learning and how they relate to face recognition. Dependency coding and information maximization appear to be central principles in neural coding early in the visual system. These principles may be relevant to how we think about higher visual processes such as face recognition as well. The paper first reviews examples of dependency learning in biological vision, along with principles of optimal information transfer and information maximization. Next, we examine algorithms for face recognition by computer from a perspective of information maximization. The eigenface approach can be considered as an unsupervised system that learns the first-and second-order dependencies among face image pixels. Eigenfaces maximize information transfer only in the case where the input distributions are Gaussian. Independent component analysis (ICA) learns high-order dependencies in addition to first-and second-order relations, and maximizes information transfer for a more general set of input distributions. Face representations based on ICA gave better recognition performance than eigenfaces, supporting the theory that information maximization is a good strategy for high level visual functions such as face recognition. Finally, we review perceptual studies suggesting that dependency learning is relevant to human face perception as well, and present an information maximization account of perceptual effects such as the atypicality bias, and face adaptation aftereffects. r A line of research lead by Attneave and Barlow [1, 3, 7] argues that statistical dependencies in the sensory input contain structural information about the environment, and that a general strategy for sensory systems is to learn the expected dependencies. Fig. 1 illustrates the idea that the percept of structure is driven by the dependencies in the sensory input. The set of points in 1c was randomly selected from a Gaussian distribution. In 1d, half of the points were generated from a Gaussian distribution and the other half were generated by rotating those points 51 about the centroid of the distribution. The dependence between pairs of dots gives the image in 1d a percept of structure. Dependency coding and information maximization appear to be central principles in neural coding early in the visual system. This paper describes how these principles may be relevant to how we think about higher visual processes such as face recognition as well. Section 2 reviews some concepts from information theory. Section 3 describes how information maximization is achieved in two unsupervised learning rules, Hebbian learning and independent component analysis (ICA).. Section 4 examines algorithms for face recognition by computer from a perspective of information maximization. Principal component solutions can be learned in neural networks with simple Hebbian learning rules [53] . Hence the eigenface approach can be considered a form of Hebbian learning model, which performs information maximization under restricted conditions. ICA performs information maximization under more general conditions. The learning rule contains a Hebbian learning term, but it is between the input and the gradient of the output. Face representations derived from ICA gave better recognition performance than face representations based on principal component analysis (PCA). This suggests that information maximization in early processing is an effective strategy for face recognition by computer.. Section 5 presents an information maximization account of perceptual effects including other-race and typicality effects, and shows how face adaptation aftereffects (e.g. [36, 51, 81] ) are consistent with information maximization on short time scales. Two models of information maximization in adaptation are presented, one in which the visual system learns a high-order transfer functions that match the curves in the cumulative probability density, and another in which the cumulative probability density is approximated with the closest fitting sigmoid. These two models give different predictions for sensitivity postadaptation. The way information maximization reduces statistical dependence can be understood as follows: The equation for the joint entropy of the output Y is the sum of the individual entropies minus the mutual information between them [22] :. Inspection of equation Eq. (5) shows that maximizing H(Y) encourages I(y 1 ,y, y n ) to be small. Thus, maximizing the joint entropy of the output encourages the individual outputs to move towards statistical independence. The mutual information is guaranteed to reach a minimum when the nonlinear transfer function g matches the cumulative distributions of the independent signals responsible for the data in X, up to a scaling and translation (Nadal and Parga, 1994) . In most circumstances these distributions are unknown. However, for mixtures of super-Gaussian signals (meaning the probability distribution is more steeply peaked than a Gaussian), the logistic transfer function has been found sufficient to separate the signals into independent components [16] . As mentioned above, many natural signals including sound sources and Dependency coding and information maximization appear to be central principles in neural coding early in the visual system. Neural systems with limited dynamic range can increase the information that the response gives about the signal by placing the more steeply sloped portions of the transfer function in the regions of highest density, and shallower slopes at regions of low density. The function that maximizes information transfer is the one that matches the cumulative probability density of the input. There is a large body of evidence that neural codes in vision and other sensory modalities match the statistical structure of the environment, and hence maximize information about environmental signals to a degree. See [65] for a review. This paper described how these principles may be relevant to how we think about higher visual processes such as face recognition as well.. Here we examined algorithms for face recognition by computer from a perspective of information maximization. Principal component solutions can be learned in neural networks with simple Hebbian learning rules [53] . Hence the Eigenface approach can be considered a form of Hebbian learning model, which performs information maximization under restricted conditions. In particular, PCA maximizes information transfer in the case where all of the signal distributions are Gaussian. ICA performs information maximization for a more general set of input distributions. The ICA learning algorithm employed here was developed from the principle of optimal information transfer in neurons with sigmoidal transfer functions. The learning rule contains a Hebbian learning term, but it is between the input and the gradient of the output. Section 4 showed that face representations derived from ICA gave better recognition performance than face representations ARTICLE IN PRESS Fig. 13 . Information maximization is consistent with the W-shaped sensitivity function such as observed psychophysically following motion adaptation. (a) Possible short-term probability density following adaptation, given a previously uniform distribution. Here we model the effect of adaptation to influence the probability density in a local range near the adapting stimulus value, as in [66] . The increase in probability near the adapting stimulus value is balanced by a decrease elsewhere within the local range, producing a Mexican-hat shape. (b) Optimal transfer function derived from the cumulative density of (a). (c) Sensitivity predictions derived from the first derivative of (b). Note that sensitivity increases near the adapting stimulus but also decreases for the stimuli immediately surrounding the adapting stimulus.

B I O O O O O O O O O O B O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B O O B O O O O O O O O B I O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I O O O O O O O O B O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I O O O O O B O O B O B I I O O O O O O B I O O O O O O B I O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O B I O O O O O B I O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B I O O O O O O B I O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Optimal Reconstruction of Material Properties in Complex Multiphysics Phenomena We develop an optimization-based approach to the problem of reconstructing temperature-dependent material properties in complex thermo-fluid systems described by the equations for the conservation of mass, momentum and energy. Our goal is to estimate the temperature dependence of the viscosity coefficient in the momentum equation based on some noisy temperature measurements, where the temperature is governed by a separate energy equation. We show that an elegant and computationally efficient solution of this inverse problem is obtained by formulating it as a PDEconstrained optimization problem which can be solved with a gradient-based descent method. A key element of the proposed approach, the cost functional gradients are characterized by mathematical structure quite different than in typical problems of PDE-constrained optimization and are expressed in terms of integrals defined over the level sets of the temperature field. Advanced techniques of integration on manifolds are required to evaluate numerically such gradients, and we systematically compare three different methods. As a model system we consider a two-dimensional unsteady flow in a lid-driven cavity with heat transfer, and present a number of computational tests to validate our approach and illustrate its performance. In this work we propose and validate a computational approach to the reconstruction of material properties in complex multiphysics phenomena based on incomplete and possibly noisy measurements. The material properties we are interested in here are the transport coefficients characterizing diffusion processes such as the viscosity or the thermal conductivity, and we focus on problems in which these coefficients depend on the state variables in the system. By the "multiphysics" aspect we mean situations in which the material property used in one conservation equation is a function of a state variable governed by a different conservation equation, e.g., reconstruction of the temperature dependence of the viscosity coefficient used in the momentum equation, where the temperature is governed by a separate energy equation, which is the specific model problem investigated in this study. This research is motivated by questions arising in the computational analysis and optimization of advanced welding processes which involves modelling complex alloys in the liquid phase at high temperatures [1] .. Inverse problems of parameter estimation for partial differential equations (PDEs) have received significant attention in the literature, both regarding theoretical [2] and practical aspects [3] . However, most of the attention focused on problems in which the material properties are functions of the space variable (i.e., the independent variable in the problem). Such problems are, at least in principle, relatively well understood and represent the foundation of, for example, numerous imaging techniques in medicine [4] and earth sciences [5] . The problem considered here is in fact different, in that the material properties are sought as functions of the state (dependent) variables in the system which gives rise to a number of computational challenges absent in the "classical" parameter estimation problem. Other than the seminal work of Chavent and Lemonnier [6] , our earlier study [7] concerning a simplified model problem and a few investigations of fully discrete formulations surveyed in [7] , there does not seem to be much literature concerning computational methods for this type of parameter estimation problems. One way to solve such inverse problems is to formulate them as optimization problems and this is the approach we will follow focusing on the "optimize-then-discretize" paradigm in which the optimality conditions are formulated at the continuous (PDE) level and only then discretized. The goal of this investigation is to extend the approach formulated in [7] for a simple model to a realistic multiphysics problem involving time-dependent fluid flow in a two-dimensional (2D) domain. As will be shown below, a number of computational difficulties will need to be overcome in order to achieve this goal.. As a key contribution of this work, we address a number of computational challenges related to accurate and efficient evaluation of cost functional gradients which are critical to the implementation of the proposed approach. More specifically, these gradients are given in terms of integrals of expressions involving state and adjoint variables defined on a grid over contours given by the level sets of the temperature field. A number of techniques have been proposed for the numerical evaluation of integrals defined over manifolds defined by level-set functions. Some of them rely on regularized Dirac delta and Heaviside functions [8, 9] , or discretization of the Dirac delta function [10] [11] [12] . Similar approaches, based on approximations of the Dirac delta functions obtained using the level-set function and its gradient, were developed by Towers [13, 14] . The family of geometric approaches proposed by Min and Gibou in [15, 16] relies on a decomposition of the entire domain into simplices. We emphasize that the problem discussed in this work is in fact more complicated, as the computation of our cost functional gradients requires evaluation of the corresponding integrals for the level-set values spanning the entire state space of interest, hence there are also additional issues related to the discretization of the state space which were outside the scope of references [8] [9] [10] [11] [12] [13] [14] [15] [16] . Thus, in order to address these questions and assess the different trade-offs in the choice of the numerical parameters we will compare the computational performance of three different methods for the evaluation of cost functional gradients.. The structure of this paper is as follows: in the next Section we formulate our model problem, in the following Section we cast the problem of parameter estimation as an optimization problem, an adjoint-based gradient-descent algorithm is formulated in Section 4, in Section 5 we outline some regularization strategies needed in the presence of measurement noise, whereas in Section 6 we analyze three different numerical approaches to the evaluation of the cost functional gradients; extensive computational results are presented in Section 7 with discussion and conclusions deferred to Section 8.

O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


An Extended Growing Self-Organizing Map for Selection of Clusters in Sensor Networks Sensor networks consist of wireless enabled sensor nodes with limited energy. As sensors could be deployed in a large area, data transmitting and receiving are energy consuming operations. One of the methods to save energy is to reduce the communication distance of each node by grouping them in to clusters. Each cluster will have a cluster-head (CH), which will communicate with all the other nodes of that cluster and transmit the data to the remote base station. In this paper, we propose an extension to Growing Self-Organizing Map (GSOM) and describe the use of evolutionary computing technique to cluster wireless sensor nodes and to identify the cluster-heads. We compare the proposed method with clustering solutions based on Genetic Algorithm (GA), an extended version of Particle Swarm Optimisation (PSO) and four general purpose clustering algorithms. This could help to discover the clusters to reduce the communication energy used to transmit data when exact locations of all sensors are known and computational resources are centrally available. This method is useful in the applications where sensors are deployed in a controlled environment and are not moving. We have derived an energy minimisation model that is used as a criterion for clustering. The proposed method can also be used as a design tool to study and analyze the cluster formation for a given node placement. A sensor network is a network of several sensors, deployed in an ad hoc fashion to gather and transmit information. Sensor networks are becoming popular with the recent advancement in the area of low power electronics and radio frequency communication. Developments in sensor technology and communications suggest that future sensors can be extremely small as well as wireless enabled. The advances in sensor technology have a strong impact on the way data is sensed. Other factors that contribute to the growth of studies in sensor network are the ease of deployment, cost effectiveness, flexibility, and reliability. The major challenge in designing sensor networks is the amount of energy dissipated in the network. The energy used may be a combination of the processing energy, transmission energy, and receiving energy. In this paper, we described the energy model to minimize the energy dissipation based on the communication distance in a sensor network. We proposed the extension of GSOM called E-GSOM to cluster sensor nodes in a sensor network. We have simulated two evolutionary computing techniques including an extension to PSO for cluster formation.. However, E-GSOM provides the best performance, in addition it is the fastest algorithm among the seven algorithms tested. It is intended to incorporate the energy as E-GSOM's growing threshold instead of spread factor, such that we can restrict the size of the cluster and the number of clusters based on the energy available in the network. This will also help to attach new nodes to the network when energy loss exceeds the specified limit. The incorporated constraint should allow more rapid flexibility in handling the overall network. We would like to improve the energy model by considering the higher order radio model. We intended to work on the application with large problem space. For these kinds of applications, a node transmission range cannot cover the entire network. Thus, we are working on the clustering algorithm with transmission range restriction.

O O O B I O O O O O B I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A 9-bit multiple relaxation Lattice Boltzmann magnetohydrodynamic algorithm for 2D turbulence While a minimalist representation of 2D Magnetohydrodynamics (MHD) on a square lattice is a 9-bit scalar and 5-bit vector distribution functions, here we examine the effect of using the 9-bit vector distribution function on the effect of a magnetic field on the Kelvin-Helmholtz instability. While there is little difference in the simulation results between the 5-bit and the 9-bit vector distribution models in the vorticity, energy spectra.., the 9-bit model permits simulations with mean magnetic field a factor of approximately 2 greater than those attainable in the standard 5-bit model. Indeed a 9-bit single-relaxation model can attain such success over a 5-bit multiple-relaxation model at the same computational expense. Lattice Boltzmann (LB) algorithms [1] are an extremely successful computational technique for solving nonlinear collisional problems because of their simplicity of coding and their ideal parallelization. In principle, one is replacing a computationally difficult problem involving nonlinear convective terms by a linearized kinetic equation with simple advection and local collision operator. Since one is now solving on a lattice in kinetic space, one typically reduces the inherited extra memory/calculations by minimizing the number of kinetic velocities required in LB to recover the fluid equations in the long wavelength limit. In particular, such a minimalist 2D LB MHD model has been introduced by Dellar [2] . It consists of a 9-bit square lattice model for the evolution of the quasi-incompressible fluid velocity u and a 5-bit (square lattice) model for the evolution of the magnetic field B. This asymmetry is because the fluid velocity u arises as the 1 st moment but the magnetic field arises from the 0 th moment of their corresponding distributions. Here we consider in some detail the effect of utilizing the same 9-bit square lattice for the vector magnetic distribution function as for the particle distribution function on the magnetic stabilization of the Kelvin-Helmholtz instability [3, 4] . In considering the stabilization of a velocity jet we extend the collision operator for the evolution of the vector magnetic distribution to a multiple relaxation model (MRT) -a relaxation model introduced in 2002 by d'Humieres et. al [5] for LB for Navier-Stokes flows and which permitted numerically stable LB simulations for considerably higher flow velocities than could be achieved by the single relaxation rate (SRT). Similarly, we [6] have found that an MRT extension to the evolution of the 5-bit vector distribution function permitted considerably greater B-fields than under the SRT collision operator. [We note that MRT models for LB MHD have also been considered earlier by Dellar [9, 10] , but these papers were not concerned with the question we are investigating]. Here we will find that even a single relaxation rate (SRT) model of the 9-bit collision operator leads to stable simulations with magnetic field magnitudes on the order of 34% greater than those attainable from the 5-bit MRT simulations at fixed Reynolds numbers. Moreover, this can be achieved at the same computational expense (wallclock time). Moving to 9-bit MRT model only gains a further 6% in maximum B-field over the 9-bit SRT, but with a wallclock time increase of over 50%. These results have direct bearing on LB-MHD tokamak simulations with its much larger toroidal to poloidal magnetic fields. The LB-MHD algorithm is an ideal computational tool as one scales codes to exascale because of its impressive parallelization. This parallelization arises because the 2 major LB-MHD operations: (a) free-streaming distribution function data to nearby spatial nodes (a simple shift), and (b) local collisional relaxation of the distribution functions (requiring only local data at the nodes). Here we have applied the LB-MHD code to study the stabilization of the Kelvin-Helmholtz unstable 2D-fluid jet by an external parallel magnetic field. Since the vector distribution function, as originally introduced by Dellar [2] , can recover the magnetic field at its zeroth moment the underlying lattice for 2D problems can be restricted to the simple 5-bit model. This SRT model has then been extended by Dellar [9, 10] and Flint et. al. [6] to MRT where the collision-step is now performed in the moment space of the magnetic distribution function as it is also performed for the moment velocity space so as to take full advantage of the local collisional invariants. In this paper we have examined the consequences of moving to a 9-bit lattice for the magnetic velocity distribution function and applied the model to the stabilization of the Kelvin-Helmholz instability in a fluid jet. Since the strength of the stabilizing axial magnetic field is a critical parameter in the jet stabilization, we have examined the effect of lattice geometry and kinetic collisional relaxation rates on Bmax before numerical instabilities limit the simulation. We found that Bmax can be successfully increased by 34% in moving from the 5-bit MRT magnetic lattice to the 9-bit SRT lattice with no loss in the computational wallclock time required. However the further extension to 9-bit MRT magnetic lattice only permitted Bmax to be 40% greater than the 5-bit MRT while the computational wallclock now increased by over 50%.. The optimal choice of kinetic moments basis is still an open question. Our choice was predicated on making the choice of the next higher order moments -but with our current results on the importance of the 9-bit SRT (which does not require the introduction of a kinetic moment space for the magnetic distribution) it may be a mute point. We have also performed some parameter runs in the choice of the stabilizing moment relaxation rates on achievable Bmax . With 6 degrees of freedom in the kinetic moment relaxation rates only a somewhat superficial testing was performed. No particularly strong correlations were found. In some cases, forcing strong damping of some of the higher kinetic moments did not lead to stabilization while reducing some led to better results. Again, in light of the important gains in the 9-bit SRT magnetic lattice (at no extra computational costs) this seems to be a secondary issue.. The important question of whether these results will also hold in wall-bounded flows is left for a future study. It is well known that the accuracy and stability of an LB-code is dependent on the treatment of boundary conditions. This would introduce another set of free parameters that would possibly lead to further discussions on improvement in boundary condition implementations. Since our next major goal is to extend these MHD results from 2D to 3D (in periodic geometry) and examine the differences between 2D and 3D Kelvin-Helmholtz jet stabilization in an axial magnetic field with an eye to moving to magnetic fusion applications where the toroidal magnetic field is an order of magnitude greater than the poloidal field, we are leaving the effect of wall boundary conditions for a later study.

O O O O B I O O O B I O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Output-sensitive modular algorithms for polynomial matrix normal forms We give modular algorithms to compute row-reduced forms, weak Popov forms, and Popov forms of polynomial matrices, as well as the corresponding unimodular transformation matrices. Our algorithms improve on existing fraction-free algorithms. In each case, we define lucky homomorphisms, determine the appropriate normalization, as well as bound the number of homomorphic images required. The algorithms have the advantage that they are output-sensitive; that is, the number of homomorphic images required depends on the size of the output. Furthermore, there is no need to verify the result by trial division or multiplication. Our algorithms can be used to compute normalized one-sided greatest common divisors and least common multiples of polynomial matrices, along with irreducible matrix-fraction descriptions of matrix rational functions. When our algorithm is used to compute polynomial greatest common divisors, we obtain a new output-sensitive modular algorithm. We give modular algorithms to compute a row-reduced form and a weak Popov form of a polynomial matrix, improving on existing fraction-free algorithms. In each case we define lucky homomorphisms, determine the appropriate normalization, as well as bound the number of homomorphic images required. The algorithms have the advantage that they are output-sensitive, that is, the number of homomorphic images required depends on the size of the output. Furthermore, there is no need to verify the result by trial division or multiplication. Our algorithms can be used to compute normalized one-sided greatest common divisors and least common multiples of polynomial matrices along with irreducible matrix-fraction descriptions of matrix rational functions.. Let D be an integral domain and. m×n be polynomial matrix of degree N . We wish to compute polynomial matrices U(z) and T(z) such that. with U(z) unimodular and T(z) row-reduced or in weak Popov form. Roughly speaking, we want the leading row coefficient of T(z) to be of the same rank as F(z) in the case of row-reduced form, or triangular in the case of weak Popov form [6, 7] . We are interested in the modular computation of row-reduced and weak Popov forms for polynomial matrices, as well as the associated transformation matrix. There are traditionally three major issues that must be addressed: the problem of "unlucky" homomorphisms, the number of images required for the reconstruction of the result, and the normalization of the result to ensure a unique answer. Although these issues are well understood in the case of polynomial GCD computations, they are nontrivial in our case. The problem of computing these normal forms and their transformation matrices can be viewed as one of determining a basis for a particular module. The difficulties in devising a modular algorithm to compute a basis for a module or a vector space is well known. The lack of uniqueness, definitions of lucky homomorphisms, and appropriate normalizations are all far from obvious in any basis computation. We overcome these difficulties by reducing the problem to a linear algebra problem.

O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B I O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I I O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A fast k-means implementation using coresets In this paper we develop an efficient implementation for a k-means clustering algorithm. Our algorithm is a variant of KMHybrid [28, 20] , i.e. it uses a combination of Lloyd-steps and random swaps, but as a novel feature it uses coresets to speed up the algorithm. A coreset is a small weighted set of points that approximates the original point set with respect to the considered problem. The main strength of the algorithm is that it can quickly determine clusterings of the same point set for many values of k. This is necessary in many applications, since, typically, one does not know a good value for k in advance. Once we have clusterings for many different values of k we can determine a good choice of k using a quality measure of clusterings that is independent of k, for example the average silhouette coefficient. The average silhouette coefficient can be approximated using coresets.To evaluate the performance of our algorithm we compare it with algorithm KMHybrid [28] on typical 3D data sets for an image compression application and on artificially created instances. Our data sets consist of 300, 000 to 4.9 million points. We show that our algorithm significantly outperforms KMHybrid on most of these input instances. Additionally, the quality of the solutions computed by our algorithm deviates less than that of KMHybrid.We also computed clusterings and approximate average silhouette coefficient for k = 1, . . . , 100 for our input instances and discuss the performance of our algorithm in detail. Clustering is the computational task to partition a given input into subsets of equal characteristics. These subsets are usually called clusters and ideally consist of similar objects that are dissimilar to objects in other clusters. This way one can use clusters as a coarse representation of the data. We loose the accuracy of the original data set but we achieve simplification.. Clustering has many applications in different areas of computer sciences such as computational biology, machine learning, data mining and pattern recognition. Since the quality of a partition is rather problem dependent, there is no general clustering algorithm. Consequently, over the years many different clustering algorithms have been developed. These algorithms can be characterized as hierarchical algorithms or partitioning algorithms.. Hierarchical algorithms build a hierarchy of clusters, i.e. every cluster is subdivided into child clusters, which form a partition of their parent cluster. Depending how the hierarchy is built we distinguish between agglomerative (bottom-up) and divisible (top-down) clustering algorithms.. Partitioning algorithms try to compute a clustering directly. For example, they try to compute a clustering by iteratively swapping objects or groups of objects between the clusters or they try to identify dense areas containing many points.. The most prominent and widely used clustering algorithm is Lloyd's algorithm sometimes also referred to as the k-means algorithm. This algorithm requires the input set to be a set of points in the d-dimensional Euclidean space. Its goal is to find k cluster centers and a partitioning of the points such that the sum of squared distances to the nearest center is minimized. The algorithm is a heuristic that converges to a local optimum. The main benefit of Lloyd's algorithm is its simplicity and its foundation on analysis of variances. Also, it is relatively efficient. The drawbacks are that the user must specify the number of clusters in advance, the algorithm has difficulties to deal with outliers and clusters that differ significantly in size, density, and shape.. Our Contribution. In this paper we develop an efficient kmeans clustering algorithm called CoreMeans. The main new idea is that the algorithm uses coresets [11] to speed up the computation of the clustering. Such a coreset is a small weighted set of points that for every set of k centers has roughly the same cost as the original (unweighted) point set [5, 15, 11, 14] . Our algorithm first computes a small coreset of the input points and then runs variant of KMHybrid [28, 20] , which is a combination of Lloyd's algorithm and random swaps. Then the algorithm doubles the size of the coreset and runs for a few steps on this coreset. This process is done until the coreset coincides with the whole point set. The coreset computation is supported by a quadtree (or higher dimensional equivalent) based data structure. This data structure can also be used to speed up nearest neighbor queries.. We compare our algorithm with algorithm KMHybrid [28, 20] , which uses a combination of Lloyd's algorithm with random swaps and simulated annealing. On most of the input instances our algorithm significantly outperforms KMHybrid, especially for low dimensional instances. For high dimensional instances our algorithm finds good solutions faster but KMHybrid's solution after a few seconds is slightly better. If we want to compute a clustering for one value of k the running time of both algorithms is often dominated by the setup time to compute auxiliary data structures. In this case CoreMeans benefits from its smaller setup time.. However, in many applications we do not know the right value of k in advance. In such a case one has to compute clusterings for many different values of k. Then one can use a quality measure independent of k to find out the best clustering. A prominent quality measure for such a scenario is the average silhouette coefficient. Unfortunately, computing the average silhouette coefficient for one clustering takes quadratic time, which is not feasible for point sets of medium and large size. However, we would like to compute the sihouette coefficient for many values of k. In this situation we can see the real strength of coresets. Using coresets it is possible to find clusterings and compute their average silhouette coefficient for large point sets and many values of k. For example, we computed clusterings for k = 1, . . . , 100 and approximated their average silhouette coefficient for a set of more than 4.9 million points in 3D consisting of the RGB values of an image in a few seconds on one core of an Intel Pentium D dual core processor with 3 GHz core frequency. In higher dimensions we did the same computations for an (artificially created) point set of 300, 000 points in 20 dimensions for all values of k between 1 and 100 in less than 8 minutes. Without coresets, the computation of the sihouette coefficient for one value of k takes several hours. Summarizing we can say that CoreMeans performs very well especially for small values of k. When we compare the computation time of CoreMeans with KMHybrid we see that for one clustering the running time of both algorithms is typically dominated by the setup time. However, KMHybrid is much more volatile and might require a number of runs to guarantee a good solution. The main strength of CoreMeans is that it can very quickly compute relatively good solutions and that one can approximate the average silhouette coefficient from the coreset. In this paper we presented an efficient implementation of a kmeans clustering algorithm using coresets. Our algorithm performs very well compared to KMHybrid [28] for small dimension and small to medium k. The quality of the solutions varies less than that of KMHybrid, which implies that we need fewer runs to guarantee a good solution. The main strength of our algorithm is to quickly find relatively good approximations for many values of k, for example when a good value for k is not known in advance. In this case, we can also use the coresets to compute the average clustering coefficient and thus to find a good choice of k. It is beyond the scope of this section to give a comprehensive overview of the clustering literature. Instead we will give a brief overview of the most important developments with focus on partitioning algorithms. We will also briefly summarize the recent work on coresets. For a more comprehensive overview of the work in clustering we refer to the surveys/books [12, 18, 6 ]. An overview of coreset constructions is given in [2] .. k-Means Clustering. The most popular algorithm for the kmeans clustering problem is Lloyd's algorithm [10, 23, 25] . It is known that this algorithm converges against a local optimum [32] . Recently, a number of very efficient implementations of this algorithm have been developed [3, 20, 19, 29, 30, 31] . These algorithms reduce the time needed to compute the nearest neighbors in a Lloyd's iteration, which is the most time consuming step of the algorithm. In the Euclidean space there are many (1 + )-approximation algorithms for the k-means clustering problem [5, 9, 15, 17, 22, 24] . Also for the k-means problem in metric spaces efficient constant factor approximation algorithms are known [26, 20] .. The quality of random sampling in metric spaces has been analyzed for some clustering problems including the metric and the Euclidean k-median [27, 8] . The analysis can be easily extended to the k-means clustering problem.. A testbed for k-means clustering algorithms has been given in [28] . We compare our implementation with algorithm KMHybrid, which is the fastest of 4 implementations described in [28] . We implemented our algorithm using C++. The code was compiled using gcc version 3.4.4 using optimization level 2 (-O2). We compare our algorithm to KMHybrid [20] . KMHybrid was compiled using the same compiler and also with optimization level 2. We ran it using the standard settings given by the developers.. We ran our experiments on an Intel Pentium D dual core processor with two cores. The algorithm used one core with core frequency 3 GHz. The computer has 2 GByte RAM.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Abductive Network Committees for Improved Classification of Medical Data Objectives: To introduce abductive network classifier committees as an ensemble method for improving classification accuracy in medical diagnosis. While neural networks allow many ways to introduce enough diversity among member models to improve performance when forming a committee, the self-organizing, automatic-stopping nature, and learning approach used by abductive networks are not very conducive for this purpose. We explore ways of overcoming this limitation and demonstrate improved classification on three standard medical datasets.Two standard 2-class medical datasets (Pima Indians Diabetes and Heart Disease) and a 6-class dataset (Dermatology) were used to investigate ways of training abductive networks with adequate independence, as well as methods of combining their outputs to form a network that improves performance beyond that of single models.Results: Two-or three-member committees of models trained on completely or partially different subsets of training data and using simple output combination methods achieve improvements between 2 and 5 percentage points in the classification accuracy over the best single model developed using the full training set.Varying model complexity alone gives abductive network models that are too correlated to ensure enough diversity for forming a useful committee. Diversity achieved through training member networks on independent subsets of the training data outweighs limitations of the smaller training set for each, resulting in net gain in committee performance. As such models train faster and can be trained in parallel, this can also speed up classifier development. Data mining and machine learning techniques for classification, association detection, sequential/temporal pattern recognition, and clustering/segmentation offer new and effective approaches to handle the data overload problem in medical informatics. They automatically discover patterns in medical data to provide support for the decision-making process in many health care areas, including screening, diagnosis, prognosis, monitoring, therapy, survival analysis, and hospital management. Tools used for performing such functions include: Bayesian and nearest-neighbor classifiers, rule induction methods, decision trees, genetic algorithms, fuzzy logic, and artificial neural networks (ANNs). The latter tool has been proposed for various medical applications, including diagnostic systems, biochemical analysis, and image analysis. In spite of the wide use of ANNs as a modeling tool, their opaque (black box) nature has limited their acceptance in medicine [1] . Learned knowledge is concealed in a maze of connections and weights, making it difficult to provide justifications and explanations often sought by physicians [2] . Techniques to improve the comprehensibility of neural networks, such as rule extraction [3] , have been used in medicine [4] . Other limitations with neural networks include the difficulty in determining optimum network topology and training parameters. There are many choices to be made in determining numerous critical design parameters with little guidance available [5] , and designers often resort to trial and error approaches which can be tedious and time consuming.. Such design parameters include the number and size of the hidden layers, the type of neuron transfer functions for the various layers, the learning rate, momentum coefficient, and stopping criteria to avoid over-fitting and ensure adequate generalization with new data.. Compared to neural networks, the alternative approach of self-organizing abductive or polynomial networks based on the group method of data handling (GMDH) algorithm offers the advantages of more automated and faster model development requiring little or no user intervention [6] . The method offers automatic selection of relevant input variables, automatic configuration of model structures, and faster convergence without the problem of getting stuck in local minima [7] . With the resulting model represented as a hierarchy of polynomial expressions, derived analytical relationships can provide better insight into the modeled phenomena and allow comparison with previously used empirical models. The technique automatically avoids overfitting through using a criterion for penalizing complexity [8] , without requiring a dedicated validation dataset; thus leaving more training data for use in actual training. Medical applications of GMDH-based techniques include modeling obesity [9] , analysis of school health surveys [10] , drug detection from EEG measurements [11] , medical image recognition [12] , and screening for delayed gastric emptying [13] . A simplified abductive network model for the waist-to-hip ratio [9] automatically selects only two out 13 input variables, giving a manageable analytical relationship and allowing greater insight into the data. A neural network model would provide no information as to which of the 13 inputs are most influential or how they affect the model output.. The importance of classification accuracy for medical diagnosis cannot be over-emphasized. In screening applications, for example, a high percentage of false negatives increases the risk of real patients not receiving the thorough investigation they need. On the other hand, a larger portion of false positives causes unnecessary inconvenience and increases the load on medical resources. In quest for higher classification accuracies and improved diagnosis, the concept of committee (ensemble) classifiers has been adopted in medicine, e.g. [14] [15] [16] [17] [18] [19] [20] . With this approach, a number of classifiers are used simultaneously and their outputs combined to produce the final predicted committee output, see Fig. 1 . The output combination module in Fig. 1 often performs simple functions on the outputs of individual members, such as majority voting or weighted averaging, without involving the input vectors of attributes [21] . Alternatively, a gating network may use the input vectors to determine the optimum weighting factors for each case to be classified [22] .. In the stacked generalization approach, the combiner takes the form of another higher-level network trained on the outputs of individual members to generate the committee classification output [23] . When member classifiers are independent, the resulting diversity in the decision making process is expected to boost generalization performance, thus improving the accuracy, robustness, and reliability of classification. Obviously, combining the outputs of several identical classifiers produces no gain, and improvement is expected only when members err in different ways so that errors may cancel out [24] . It can be shown [25] , that the mean squared error in the averaged committee output contains as a component the covariance of the outputs of individual committee members, therefore individual members should ideally be uncorrelated or even negatively correlated. Krogh and Vedelsby [26] have shown that the committee error can be expressed as two terms, one measuring the average generalization error of individual members and the other measuring the diversity or disagreement among the members. An ideal committee would therefore consist of highly accurate classifiers that disagree as much as possible. In the 'committee of experts' approach, members are developed using different machine learning techniques that adopt different ways to build decision boundaries for the classification problem at hand, such as neural networks, nearest neighbor classifiers, classification and regression trees (CART), etc. This allows training adequate individual models on the full training dataset available while ensuring a good degree of diversity among them. However, in many situations a committee is restricted to use only one machine learning technology. Neural networks allow great diversity in the available architectures (multi-layer perceptron (MLP), radial basis function (RBF), etc.), learning algorithms (back propagation, simulated annealing, etc.), and in the parameters that can be varied during training (e.g. network topology, neuron transfer functions, initial random weights, learning rate, momentum, stopping criteria, etc.). This allows many possibilities for constructing individual committee members that are reasonably independent using the same training data.. Although neural network committees have been reported for many applications, there appears to be no mention of GMDH-based abductive (or polynomial) network committees in the literature.. Due to the self-organizing and self-stopping nature of such networks, the absence of initial random weights, and the little room for user intervention during training, there is less diversity in the models that can be synthesized using the same training data. This paper investigates ways of obtaining diverse committee members and demonstrates that committees made up of abductive networks trained on independent subsets of the available training data can give better classification performance than individual committee members and single models that utilize the full training dataset. In effect, this may lead to better utilization of a given training dataset through splitting it into n subsets and forming an n-member committee from models trained on those subsets. Individual member networks are expected to train faster than the single model. They can also train in parallel, thus reducing the overall training time. Section 2 gives a brief introduction of the GMDH algorithm and the abductive network modeling tool used, and describes the approach adopted for constructing abductive network committees through training individual members and combining their outputs. Section 3 gives a brief outline of the three medical datasets used in the investigation. Section 4 presents classification results obtained using abductive network committees that employ various methods of combining outputs from individual members. The main objective here has been to demonstrate the effectiveness of abductive networks in improving performance over single models in spite of the limited scope for injecting diversity. Therefore, no special efforts were taken to optimize the performance of individual models or compare results with committees using other machine learning techniques.. Conclusions are made and suggestions for future work given in Section 5. Abductive networks can overcome the opacity and incomprehensibility limitations of neural networks that hamper their wide acceptability in medical diagnosis. This is achieved through automatic selection of only significant inputs and providing analytical model relationships. Other advantages include the more automated model synthesis through self-organization and selfstopping. However, when forming abductive network committees such advantages, coupled with the regression-based approach to learning, tend to limit the diversity that can be achieved among resulting models, thus degrading committee performance. We explored ways to overcome this limitation and demonstrated gain in classification accuracy on standard binary and multi-class medical datasets. Two-or three-member committees of models trained on completely or partially different datasets using simple output combination methods achieve improvements between 2 and 5 percentage points in the classification accuracy over the best single model developed using the full training set. For binary classification, best results were obtained with member outputs being combined by simple averaging or through weighting using a dynamic certainty measure.. Future work would investigate the effect of other ensembling techniques that use bootstrap sampling, such as bagging and boosting. Committee of Experiment 3 in Table 1 , using combination method d in Table 2 62.5 89.8 80.6 77.9 78.8 Table 6 . Confusion matrix showing best performance of committee C2 (Table 5) Table 1 .

O O O O O O O O O O O O O O O O O O O O O O B I O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B I O O O O B O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Semiparametric estimation in perturbed long memory series The estimation of the memory parameter in perturbed long memory series has recently attracted attention motivated especially by the strong persistence of the volatility in many financial and economic time series and the use of Long Memory in Stochastic Volatility (LMSV) processes to model such a behaviour. This paper discusses frequency domain semiparametric estimation of the memory parameter and proposes an extension of the log periodogram regression which explicitly accounts for the added noise, comparing it, asymptotically and in finite samples, with similar extant techniques. Contrary to the non linear log periodogram regression of Sun and Phillips (2003), we do not use a linear approximation of the logarithmic term which accounts for the added noise. A reduction of the asymptotic bias is achieved in this way and makes possible a faster convergence in long memory signal plus noise series by permitting a larger bandwidth. Monte Carlo results confirm the bias reduction but at the cost of a higher variability. An application to a series of returns of the Spanish Ibex35 stock index is finally included. The estimation of the memory parameter in perturbed long memory processes has recently received considerable attention motivated especially by the strong persistence found in the volatility of many financial and economic time series. Alternatively to the different extensions of ARCH and GARCH processes, the Long Memory in Stochastic Volatility (LMSV) has proved an useful tool to model such a strong persistent volatility. A logarithmic transformation of the squared series becomes a long memory process perturbed by an additive noise where the long memory signal corresponds to the volatility of the original series. As a result estimation of the memory parameter of the volatility component corresponds to a problem of estimation in a long memory signal plus noise model. Several estimation techniques have been proposed in this context (Harvey(1998) , Breidt et al.(1998) , Deo and Hurvich (2001) , Sun and Phillips (2003) , Arteche (2004) , Hurvich et al. (2005) ).. The perturbed long memory series recently considered in the literature are of the form,. where µ is a finite constant, u t is a weakly dependent process with a spectral density f u (λ) that is continuous on [−π, π], bounded above and away from zero, and y t is a long memory (LM) process characterized by a spectral density function satisfying. for a positive finite constant C, α ∈ [1, 2] and 0 < d 0 < 0.5. The LMSV model considers u t a non normal white noise but in a more general signal plus noise u t can be a serially weakly dependent process as in Sun and Phillips (2003) and Arteche (2004) . The constant α is a spectral smoothness parameter which determines the adequacy of the local specification of the spectral density of y t at frequencies around the origin. The interval 1 ≤ α ≤ 2 covers the most interesting situations. In parametric standard LM processes, such as the fractional ARIMA, α = 2 and α = 1 in the seasonal or cyclical long memory processes described in Arteche and Robinson (1999) if the long memory takes part at some frequency different from 0. The condition of positive memory 0 < d 0 < 0.5 is usually imposed when dealing with frequency domain estimation in perturbed long memory processes and guarantees the asymptotic equivalence between spectral densities of y t and z t . Otherwise the memory of z t corresponds to that of the noise (d 0 = 0). For u t uncorrelated with y t the spectral density of z t is. as λ → 0 and z t inherits the memory properties of y t in the sense that both share the same memory parameter d 0 . However the spectral smoothness parameter changes and for z t is min{2d 0 , α} = 2d 0 .. The semiparametric estimators considered in this paper are based on the minimization of some function of the difference between the periodogram and the local specification of the spectral density in (3) . The periodogram of z t does not approximate accurately Cλ −2d 0 and this causes a bias which translates into the different estimators. This is discussed in Section 2. As a result estimation techniques have been proposed that consider explicitly the added noise in the local specification of the spectral density of z t . They are described in Section 3. Section 4 proposes an estimator based on an extension of the log periodogram regression and establishes its asymptotic properties. Section 5 compares the "optimal" bandwidths defined as the minimizers of an approximation of the mean square error of the different semiparametric estimators considered. The performance in finite sample perturbed LM series is discussed in Section 6 by means of Monte Carlo. Section 7 shows an application to a series of returns of the Spanish Ibex35 stock index. Finally section 8 concludes. Technical details are placed in the Appendix.

B I O B I I O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O