Accelerating MATLAB Image Processing Toolbox functions on GPUs In this paper, we present our effort in developing an opensource GPU (graphics processing units) code library for the MATLAB Image Processing Toolbox (IPT). We ported a dozen of representative functions from IPT and based on their inherent characteristics, we grouped these functions into four categories: data independent, data sharing, algorithm dependent and data dependent. For each category, we present a detailed case study, which reveals interesting insights on how to efficiently optimize the code for GPUs and highlight performance-critical hardware features, some of which have not been well explored in existing literature. Our results show drastic speedups for the functions in the data-independent or data-sharing category by leveraging hardware support judiciously; and moderate speedups for those in the algorithm-dependent category by careful algorithm selection and parallelization. For the functions in the last category, finegrain synchronization and data-dependency requirements are the main obstacles to an efficient implementation on GPUs. With high volumes of graphics cards deployed in personal computers or workstations, graphics processor units (GPUs) become a parallel computing platform accessible to a wide range of users. Given their high memory bandwidth and teraflops computing capability, there is a strong incentive to use GPUs for general purpose computation (GPGPU) and there have been many successful reports in the literature on such efforts [9] . However, two main challenges remain for wide adoption of GPGPU. First, although GPU programming has been significantly simplified with the development of programming tools/environments such as AMD/ATI Brook+/CAL [16] , NVIDIA CUDA [22] , and a recently proposed open industry standard OpenCL [23] , developing high-performance GPU programs requires deep understanding of both application algorithms and GPU hardware architectures, making it elusive for many end users. Second, existing work on GPGPU usually focuses on one or a few specific algorithms and lacks of a systematic way of dealing with more generic classes of applications.. In this work, we promote wide adoption of GPGPU by leveraging MATLAB TM , one of the most commonly used platforms in scientific computing, and developing open-source GPU programs to replace the native implementation of MATLAB functions through the MEX interface [20] . This way, end users can easily experience the benefits of GPGPU with very little change in their MATLAB code. There have been some recent works on using GPUs to accelerate programs in MATLAB [21] [19] . Jacket [18] is a commercial product which is closely related to our work. It is a GPU engine for MATLAB and enables standard MATLAB code to run on NVIDIA CUDA-supported GPUs. In this work, we focus on the MATLAB Image Processing Toolbox (IPT), which in majority has not been covered by previous work. We selected a dozen of representative and performance-critical functions from IPT. Despite apparent resemblance between graphics and image processing in that they all work on a large number of pixels, many of the IPT functions exhibit significantly different characteristics from typical graphics processing. Based on the inherent parallelism and data dependency, we grouped these functions into four categories: data independent, data sharing, algorithm dependent, and data dependent. In each category, we present a detailed case study to reveal interesting insights in optimizing GPU code for algorithms in the category. In our work, we developed our GPU programs for both NVIDIA GTX 280 and AMD/ATI RV870 GPUs in order to exploit their unique hardware features for these IPT functions. Some of the performance-critical hardware features such as texture caches, to our knowledge, have not been well explored in the existing work.. Our experience with accelerating the data-independent IPT functions shows that they can easily achieve very good kernel speedups using GPUs. Some functions in this category, however, are limited by data bandwidth due to their low computational requirements per data element. Functions in the data-sharing category require the programmer to carefully balance thread-level parallelism and data/computation reuse, in order to overcome high latencies in accessing off-chip GPU memories. Due to their abundant parallelism these functions can also achieve drastic kernel speedups by effectively exploiting the GPU hardware features. Algorithm selection and the associated parallelization strategy are critical for algorithm-dependent function, in order to extract parallelism for GPUs. Nevertheless, we are able to achieve reasonably good kernel speedups from the functions in this category even with their inherent data dependencies and communication requirements. Finally, fine grain synchronization and data dependency requirements make the functions in the datadependent category a relatively poor fit for GPU implementation.. The main contributions of this work include (1) Develop highquality open-source library code to promote wide adoption of GPU as an effective accelerator for media processing; (2) Reveal interesting insights on how to efficiently parallelize a wide range of image processing algorithms; and (3) Identify performancecritical hardware features in different GPUs.. The remainder of the paper is organized as follows. In Section 2, we first highlight the impact of different GPU architectures on GPGPU programming and establish the experimental methodology to be used throughout the paper. In Section 3, we select representative and performance-critical functions from the MATLAB IPT and group them in four categories based on their characteristics. Then, we present a detailed case study and summarize the performance results as well as optimization strategy for each of the four categories in Sections 4-7, respectively. Finally, Section 8 concludes the paper. In summary, to optimize the functions in this category, we mainly focus on two directions. First, we aim to optimize the TEX:ALU ratio by leveraging various types of memory/ computation reuse, which may require utilization of different hardware structures in different GPUs. Second, we need to carefully balance data/computation reuse with thread-level parallelism so as to ensure there are enough threads to hide memory access latencies.. Another function we studied in this category is radon transform. One key optimization is the input/output block-tiling using shared memory, which helps to generate coalesced global memory writes. Another one is to use atomic operations to generate extra thread-level parallelism. Our performance results show 80x(84x) for overall speedup (kernel speedup) using CUDA on GTX280. Detailed information can be referred to the project website [15] .. From the two functions that we studied in this category, we can see that we need to carefully re-think the algorithm and quite often the resulting GPU algorithm is fundamentally different from the sequential CPU implementation. Furthermore, the optimization strategies are also algorithm dependent in terms of how to exploit thread-level parallelism. In this paper, we present our experience in developing high performance GPU code for a dozen functions from the MATLAB Image Processing Toolbox. Based on the algorithm characteristics, we group the functions into four categories: data independent, data sharing, algorithm dependent and data dependent. Then, we present detailed studies to establish optimization strategy for each category of functions. Our results show drastic speedups for the functions in the data-independent or data-sharing category; and moderate speedups for those in the algorithm-dependent category. For the functions in the last category, fine-grain synchronization/datadependency requirements make them a poor fit for GPU implementation. We use both NVIDIA GTX 280 and AMD/ATI RV870 GPUs in our experiments to exploit their unique hardware features.

O B B I O O O O O O O O O O O O O O O O O O O O O O O B B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O B B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Sybilproof reputation mechanisms Due to the open, anonymous nature of many P2P networks, new identities -or sybils -may be created cheaply and in large numbers. Given a reputation system, a peer may attempt to falsely raise its reputation by creating fake links between its sybils. Many existing reputation mechanisms are not resistant to these types of strategies.Using a static graph formulation of reputation, we attempt to formalize the notion of sybilproofness. We show that there is no symmetric sybilproof reputation function. For nonsymmetric reputations, following the notion of reputation propagation along paths, we give a general asymmetric reputation function based on flow and give conditions for sybilproofness. The large scale of many P2P networks and other online communities make it difficult for peers or users to assess the "trustworthiness" of other users, since a typical user's past history includes only a small fraction of the entire community. A common approach to this problem is the creation of a reputation system within the community. A reputation system attempts to aggregate the peers or users' collective experiences in order to allow a user to form an opinion about someone with whom he has not previously interacted [8] . Further, reputation may be used to assess and reward Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. "good" behavior, for example, a peer or user with high reputation may be rewarded with preferred queueing or preferred partnering.. In recent years, many people have come up with reputation mechanisms for various applications [4, 5, 9, 7, 1] , see also citations in [6] . However, much of this work has been ad-hoc -creating specific reputation mechanisms to combat some particular collection of adversarial strategies. In contrast, we would like to explore what conditions a reputation function must satisfy in order to be robust to adversarial strategies. This may allow a move towards defining a general class of robust reputation mechanisms. In this paper, we focus on the sybil attack.. In an online environment, new identities may be created with minimal cost, and users are not tied to unique identifiers. Therefore, a single user may create enough sybils to constitute a large fraction of the community [2] . In particular, a user may strategically create sybils for the purpose of boosting his own reputation. One example of this is the "link spamming" attack to PageRank [7] -when a single user attempts to boost his reputation by creating a large number of duplicate identities, who all recommend him.. Like many others [4, 9] , we base our analysis on the trust network (or "web of trust") that may be constructed between users, where edges represent direct trust (i.e. the outcome of actual interactions). To aid analysis, we restrict our attention to reputation mechanisms that depend only the current structure of the trust graph. In section two, we summarize our main results. In section 3, we define such a reputation function on a trust graph, and define the notion of sybilproofness in this framework. In section 4, we discuss conditions for sybilproofness. In the case of symmetric reputation functions (i.e. functions invariant to graph isomorphism), we show that no sybilproof mechanism exists. In the asymmetric case, we give a generalized flow-based reputation function and give some conditions for sybilproofness in that case. In section 5, we conclude with some open questions in this area. We have presented a possible framework for assessing a reputation mechanism's robustness to sybils. We have shown that no nonconstant, symmetric reputation function exists.. Further, we have given a collection of flow-based asymmetric reputation functions which are sybilproof, under some conditions.. There are many open questions related to this direction of work. For one, the flow-based reputation function defined on section 4.2 is unlikely to be the most general formulation of an asymmetric reputation function. Finding a more general formulation, along with necessary and sufficient conditions for sybilproofness would be useful. Furthermore, in this paper, we have focused strictly on a static graph model of reputation. One possible generalization of this framework is to allow reputation functions to depend on the state of the network at previous time steps as well as the current state of the network. It would also be interesting to extend the analysis here to explore necessary requirements for sybilproofness in a dynamic reputation model.

O B O O O O O O O O O O O O O O B O O O O O O O O O O B I O O O O O O O O B O O O B O O B O O B O O O O O O B O O O O B O O B O O O O O O O O O O O O O O O O B O O O O O O O O B B O O O O O B O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O


A generic approach for on-the-fly adding of context-aware features to existing websites More and more, mobile devices act as personal information managers and are able to obtain rich contextual information on the user's environment. Mobile, context-aware web applications can exploit this information to better address the needs of mobile users. Currently, such websites are either developed separately from their associated desktop-oriented version, or both versions are created simultaneously by employing methodologies that support multiplatform context-aware websites, requiring an extensive engineering effort. While these approaches provide a solution for developing new websites, they go past the plethora of existing websites. To address this issue, we present an approach for enhancing existing websites on-the-fly with context-aware features. We first discuss the requirements for such an adaptation process, and identify applicable adaptation methods to realize context-aware features. Next, we explain our generic approach, which is grounded in the use of semantic information extracted from existing websites. Finally, we present a concrete application of our approach that is based on the SCOUT framework for mobile and context-aware application development. In recent years, we have witnessed a rapid evolution of mobile device capabilities. With improved processing power and memory, better input capabilities and higher screen resolution, today's devices such as the iPhone or Blackberry are able to run powerful mobile web browsers. Combined with an increased availability of WiFi hotspots and high-speed cellular networks Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. HT '11, June 6-9, 2011 , Eindhoven, The Netherlands. Copyright 2011 ACM 978-1-4503-0256-2/11/06...$10.00.. (e.g., 3G/4G), this has lead to a continuous growth in market share for mobile browsers (according to a study by NetMarketShare, the market share has doubled in the last eleven months 1 ). As a result, there has been a significant shift in when, how and why websites are being used. In particular, mobile users often consult websites to quickly look up information needed for their current task (e.g., when does the next train to the university leave) or information related to their environment (e.g., where can I have lunch in this neighborhood). Although mobile users can use existing desktoporiented websites to obtain such information, it results in a suboptimal mobile experience, as it combines the limitations of desktop websites with those of mobile devices.. At the same time, modern mobile devices increasingly serve as fullyfledged personal information managers (e.g., contact information, agenda), while also supporting location and sensing technologies (e.g., location tracking, QR readers, RFID scanners). These new capabilities enable websites to tailor content and services to an individual user's contextual information, thus making them better suited for a mobile environment. Note that we consider the term "contextual information" here in its broadest sense: i.e., including personal information (e.g. preferences, characteristics, goals), device information (e.g., type, capabilities), and environment information (e.g., location, sensory data). Many of these websites are dedicated context-aware mobile versions developed next to the mainstream (desktop-oriented) website, or are realized using a multi-platform development ethodology. These strategies are time-consuming and costly, and only suitable for websites built from scratch.. We present an approach to enrich existing websites with contextaware features to better satisfy the needs of mobile users. The enrichment is done on-the-fly, while the user is consulting a webpage, so no expensive re-engineering of a website is necessary. Our main contribution is the automatic enhancement of existing websites: currently, no guidelines or frameworks to add contextaware features to existing websites are available.. We start by presenting related work (section 2). Then, we formulate the requirements for the approach (section 3). Subsequently, in section 4, we present adaptation methods for adding context-aware features that are suitable to be applied on-the-fly to existing websites in a mobile setting. We then present a three-step, semantics-based approach for adding context-aware features to existing websites (section 5 & 6) . This approach is motivated by the increasing availability of semantic information in websites, primarily due to the use of semantic annotation languages (e.g., RDFa, microformats). Such semantic information allows identifying relevant website content that can be enhanced with context-aware features.. Consequently, our approach consists of 1/ extracting this semantic information from the website, 2/ matching this information with the user's context, in order to identify relevant website content, and 3/ adapting the webpage to realize the desired context-aware features. In section 7, we show the feasibility of our approach by discussing a proof-of-concept implementation based on the SCOUT framework for mobile, context-aware application development. Finally, we present conclusions and future work (section 8). In this paper, we presented a generic approach for adding contextaware features to existing websites in a mobile setting. Our work considers three basic requirements, which take into account the mobile setting, and the fact that we are dealing with a priori unknown, third-party websites. We then present three methods for realizing context-aware features, which were synthesized from the literature on existing context-aware systems. Our generic approach consists of three steps: extracting semantics from the requested website, matching these semantics to the user's context, and applying adaptation to realize the desired context-aware features. Although our work is not solely dependent on the use of semantic annotation languages in websites (e.g., RDFa, micro-formats), they are a strong enabling factor. To the best of our knowledge, there is no other work described in literature that adds rich, semantics-based context-aware features to existing websites.. As a proof-of-concept for our approach, we implemented a concrete realization called COIN. This realization uses SCOUT, a framework for mobile context-aware application development, to access the user's contextual information, and exploits RDFa annotations to extract webpage semantics. In order to match these extracted semantics to the user's context, COIN applies a custom query-based matching process. Finally, the adaptations, realizing the desired context-aware features, are implemented by a generic JavaScript library, which is re-usable by third party developers following our approach.. Future work consists of implementing support for other annotation languages (such as microformats and eRDF) and experimenting with site-specific wrappers, in order to find out to what extent they can be employed to extract semantic information from non-annotated websites. Finally, we plan to investigate more user-friendly ways of specifying matching strategies, such as an extended API or a visual interface. Most existing websites have been designed for access from desktop computers with sufficient screen size, resources and input capabilities. In order for these websites to be usable in a mobile environment, webpages should be tailored to specific mobile devices [14, 34] as well as to a user's mobile context [29, 31] .. Manually authoring new mobile websites (or transforming existing desktop-oriented websites into mobile versions) is expensive and time-consuming, especially if the specific requirements of mobile users (e.g., context-awareness) are taken into account. For instance, authors need to prepare multiple versions of the same website, in order to suit different types of mobile devices (e.g., PDA, mobile phone). Manually customizing the website to the user's device and context can for example be achieved by using dynamic web templates (e.g., using languages such as PHP and JSP) and Cascading Style Sheets (CSS).. As part of the field of Adaptive Hypermedia, multi-platform context-aware methodologies have been developed that adapt a website according to the user, his device and/or context. For this purpose, the application design includes embedded conditions (e.g., in the form of queries [38] ) or context-matching expressions [32] , or is complemented with ECA (Event-Condition-Action) rules [16] or aspect-oriented specifications [8] , each of which reference the user's context in order to adapt the website to various contextual parameters (e.g., device, preferences). Although such systems allow for the adaptation specification to be hand-crafted towards the specific content and functionality of the website, they require extensive engineering at design time, and are not suitable for already deployed, existing websites. In [4] , context-aware browsing is employed to navigate through hypermedia: by walking around, information associated with the user's current context (i.e., location) is actively pushed to the user. In physical hypermedia [10] , hypermedia nodes represent real (physical) objects with associated digital information, which allows a user to directly view digital content related to a nearby physical object. As was the case before, such systems require an extensive engineering effort; also, they typically focus on linking content to location, and do not allow more complex matching of content to the user's general context.. On the other hand, so-called transcoding approaches enable the adaptation of arbitrary existing websites at runtime. Although they lack the accuracy of site-specific adaptation engineering (see above), they allow any webpage to be tailored to the user. In such an approach, a transcoding module transforms third-party webpages, for instance to view them on mobile devices [14, 34] . However, most transcoding approaches focus only on adapting to device properties (e.g., small screen), and do not take into account the user's full context. More recently, the MIMOSA [29] platform has been proposed, which takes the user's full context into account. This is work in progress, which (until now) only provides a HTML parsing utility accessible by modules implementing the adaptation logic. This means that modules can only adapt a small set of supported websites, as the knowledge on a website's HTML structure and contents needs to be hard-coded in the module before the website can be adapted. In contrast, our approach is based on the high-level semantics of the website, which allows adaptation strategies to be reused across arbitrary websites. Furthermore, MIMOSA does not consider requirements arising in a mobile environment while dealing with third-party websites.. Additionally, in most existing (automatic tailoring) approaches, adaptation only takes place once, either when the page is being generated (i.e., after an explicit user request) or after it has been received by the client. In [9] , an extension for WebML is presented, where adaptation can also be triggered based on changes in context (as is the case in our approach). Nevertheless, as for the AH systems discussed earlier, the adaptation logic needs to be explicitly engineered for each web application. In [35] , current technologies such as AJAX are exploited to dynamically replace certain parts of a webpage by their adapted counterparts, which is called "instant adaptation". However, only the general possibilities of using asynchronous technologies in AH systems are discussed, while we present a generic approach for on-the-fly webpage adaptation aiming at context-awareness.. Some works have proposed recommendations or guidelines for building mobile and/or context-aware applications. In [22] , guidelines for designing new websites for small screen devices are presented. In [31] , these guidelines are repeated, and complemented with a discussion of key aspects for the design of new mobile web applications, also briefly mentioning context-awareness. In [19] , guidelines are proposed for the interaction-and user interface design of context-aware applications. These guidelines focus on applications to be built from scratch, and partly overlap with the requirements formulated for our approach (see section 3). The W3C also formulated a number of best practices to create web content and applications suitable for mobile devices 2, 3 . All these works present guidelines that target new (web) applications; they do not address issues arising when adding context-awareness on-the-fly to already existing applications (in particular their a priori unknown content and structure). To the best of our knowledge, there currently exist no guidelines or generic approaches for on-the-fly adding of contextawareness to arbitrary existing websites.

O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Gradient clock synchronization We introduce the distributed gradient clock synchronization problem. As in traditional distributed clock synchronization, we consider a network of nodes equipped with hardware clocks with bounded drift. Nodes compute logical clock values based on their hardware clocks and message exchanges, and the goal is to synchronize the nodes' logical clocks as closely as possible, while satisfying certain validity conditions. The new feature of gradient clock synchronization (GCS for short) is to require that the skew between any two nodes' logical clocks be bounded by a nondecreasing function of the uncertainty in message delay (call this the distance) between the two nodes, and other network parameters. That is, we require nearby nodes to be closely synchronized, and allow faraway nodes to be more loosely synchronized. We contrast GCS with traditional clock synchronization, and discuss several practical motivations for GCS, mostly arising in sensor and ad-hoc networks. Our main result is that the worst case clock skew between two nodes at distance d or less from each other is (d+where D is the diameter 1 of the network. This means that clock synchronization is not a local property, in the sense that the clock skew between two nodes depends not only on the distance between the nodes, but also on the size of the network. Our lower bound implies, for example, that the TDMA protocol with a fixed slot granularity will fail as the network grows, even if the maximum degree of each node stays constant. Consider the classical distributed clock synchronization problem. A set of nodes communicate over a reliable network with bounded message delay. Each node is equipped with a hardware clock with bounded drift, that is, a timer running at roughly, but possibly not exactly, the rate of real time. Each node continuously computes logical clock values based on its hardware clock, and on messages exchanged with other nodes. The goal is to synchronize the nodes' logical clocks as closely as possible. To rule out trivial algorithms, the logical clocks must satisfy some validity conditions, for example, that they remain close to real time. This problem has been the subject of extensive research. Previous work in the area has focused on minimizing the clock skew between nodes and minimizing the amount of communication used by the synchronization algorithm [2, 12] , on tolerating various types of failures of the nodes and the network [5, 12] , and on proving lower bound results about clock skew and communication costs [3, 7, 9, 10] . In this paper, we introduce a new property for clock synchronization algorithms (CSA for short), the gradient property. Define the distance between two nodes to be the uncertainty in message delay between the nodes. Informally, the gradient property requires that the skew between two nodes forms a gradient with respect to the distance between the nodes. That is, nearby nodes should be closely synchronized, while faraway nodes may be more loosely synchronized.. We first contrast gradient clock synchronization with traditional clock synchronization. Let D be the diameter of the network. Then there exists a well-known lower bound result [7] saying that, for any CSA, the worst case clock skew between some pair of nodes in the network is (D). 2 Most CSAs (e.g., [12] ) achieve a worst case skew of O(D). However, these CSAs allow O(D) skew between any two nodes.. In particular, to our knowledge, in all existing CSAs, there exist executions in which a pair of nodes at O(1) distance from each other have O(D) skew. Thus, current CSAs do not satisfy the gradient property, because nearby nodes are not always well synchronized.. We now discuss some motivation for studying the gradient property. In many highly decentralized networks, such as sensor and ad-hoc networks, applications are local in nature. That is, only nearby nodes in the network need to cooperate to perform some task, and nodes that are far away interact much less frequently. Hence, only nearby nodes need to have highly synchronized clocks. As nodes get farther apart, they can tolerate greater clock skew. Thus, for these applications, the maximum acceptable clock skew between two nodes forms a gradient in their distance.. As an example, consider first the data fusion [11] problem in a sensor network. A group of distributed sensors collect data, then try to aggregate their data at one node to perform some signal processing on it. In order to conserve energy, the sensors form a communication tree. Starting from the leaves, each sensor sends its data to its parent sensor. When a parent sensor has received data from all its children, it "fuses" the data, that is, constructs a summary representation of the data, and sends the summary to its own parent. Since sensors typically measure real-world phenomena, times are associated with the sensor measurements. When fusing data, the children of a parent node must synchronize their clocks, so that the times of their readings are consistent and a fused reading will make sense. Hence, nearby nodes, which may be children of the same parent, need to have well synchronized clocks. But faraway nodes, which are not children of the same parent, can tolerate more poorly synchronized clocks.. Next, consider the target tracking problem in a sensor network. Suppose two sensor nodes want to measure the speed of an object. Each node records the time when the object crosses within its vicinity. Then the nodes exchange their time readings, and compute t, the difference in their readings. The amount of error in t is related to the clock skew between the nodes. The object's velocity is computed as v = d t , where d is the known Euclidean distance between the nodes. Suppose the nodes do not need to compute v exactly, but only to an accuracy of 1%. Since v = d t , then the larger the Euclidean distance is between the nodes, the more error is acceptable in t, while still computing v to 1% accuracy. Thus, the acceptable clock skew of the nodes forms a gradient. 3 What kind of gradient can be achieved by a clock synchronization algorithm? When the network consists of two nodes at distance d from each other, the tightest possible worst-case synchronization between the nodes is O(d). If there are more nodes, arranged in an arbitrary topology, is there a synchronization algorithm that ensures that the clock skew between all pairs of nodes is linear in their distance at all times? We show that no such algorithm exists. Our main result, stated in Theorem 1 of Sect. 5, is that if hardware clocks can drift, then given any clock synchronization algorithm and any D ≥ 1, there exists a network of diameter D, such that for any d ∈ [1, D] , there exists some execution of the CSA in which two nodes at most distance d apart in the network have (d + log D log log D ) clock skew. An implication of this result is that an application, such as TDMA [6] , that requires a fixed maximum skew between nearby nodes, cannot scale beyond networks of a certain diameter. We conjecture that the lower bound is nearly tight for small d, and that there exist CSAs which ensure that O(1) distance nodes always have O(log D) clock skew.. The rest of this paper is organized as follows. Section 2 describes previous work on clock synchronization and its relation to our work. Section 3 defines our model for clock synchronization, and Sect. 4 formally defines the gradient clock synchronization problem. We state our main lower bound and give an overview of its proof in Sect. 5. We prove two lemmas in Sects. 6 and 7, then prove the GCS lower bound in Sect. 8. Finally, we conclude in Sect. 9 with some remarks and open problems. We have introduced the gradient clock synchronization problem. We have shown the problem's usefulness in the context of sensor and ad-hoc networks, and have also noted that many current clock synchronization algorithms do not solve the problem. We proved that for any f -GCS algorithm, log D(N ) ). We also discussed some implications of this result.. Though our results show that nearby nodes may have large skew, we did not carefully analyze the amount of time for which the large skew may persist. However, it may be seen from the construction given in Sect. 8 that the ( log D log log D ) clock skew between neighboring nodes only lasts (1) amount of time. Furthermore, it took (D) amount of time to construct the high skew situation, suggesting that large skew between neighbors is a relatively rare occurrence. Algorithms which rely on gradient clock synchronization, such as TDMA, may exploit this fact. For example, a TDMA algorithm may adjust the granularity of the nodes' broadcast slots depending on the amount of clock skew between nodes. Nevertheless, performing such on-thefly adjustments and disseminating the new broadcast schedule throughout the network may prove challenging.

O B I O O O B O B I O O O O B B I O O O B O O O O B O O O O O O O B B O O O B O O B O O O O O O O O O O O O O O O O O O O O O O B O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B O O O O O O B O O O O O O O O O O B B I O O O O B O O O O O O O O O O O O O O O B O O O B I O O O O O O O O O O B I O O O O O O O O B O O O O O O O B O O B O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O B B I O O O O O O O O O B O O B O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O B B O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B I O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O B O O O O O O O O O O O O O O O O B B I O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B I O O O O O O O O B O O O O O O O O O O O O B B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B I O O O B O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O B I O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B I O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O B O B B O O B I O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B O O O O O O O O O O B I O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B B O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B O O O O O O O B O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B O O O


Genetic algorithm optimization of superresolution parameters Superresolution is the process of producing a high resolution image from a collection of low resolution images. This process has potential application in a wide spectrum of fields in which navigation, surveillance, and observation are important, yet in which target images have limited resolution. There have been numerous methods proposed and developed to implement superresolution, each with its own advantages and limitations. However, there is no standard method or software for superresolution. In this paper a genetic algorithm solution for determining the registration and point spread function (PSF) parameters for superresolution is proposed and implemented, and a superresolved image is generated using genetic algorithm optimization of an existing superresolution method. The solution space for superresolution problems is usually very large, even for relatively small images. The problems are highly computationally complex and have highly domain-dependent features. The dimensionality of the superresolution problem includes that of the unknown high-resolution image as well as that of a measurement vector, both of which may be in the hundreds of thousands of values. The goal of any superresolution method must be to implement a model that will simplify the problem.. Genetic algorithms combine the utility and complexity of problem-solving computer programs with the power and simplicity of natural selection. Genetic algorithms are stochastic search techniques based on the principles of evolution. Extensive research has been performed exploiting the robust properties of genetic algorithms and demonstrating their capabilities across a broad range of problems. These evolutionary methods have gained recognition as general problem solving techniques in many applications, including function optimization, image processing, classification and machine learning, training of neural networks, and system control.. Instead of attacking the superresolution problem directly with highly computationally complex algorithms, genetic algorithms can be applied to reduce the problem to the three genetic operators-mutation, recombination, and selection. Not only can genetic algorithms reduce the complexity of superresolution significantly, but genetic algorithms may also be easily mapped onto distributed computing systems, which are in essence powerful inexpensive supercomputers.. An overview of existing superresolution techniques is in section 2. Section 3 outlines the methodology of this genetic algorithm experiment to solve the superresolution problem, and section 4 presents the results. Conclusions are in section 5; future work is discussed in section 6. Superresolution is an inverse ill-posed problem, for which there has been limited success in solving in a timely and efficient fashion. To achieve superresolution restoration of image sequences containing anything more complex than global translational motion, a spatial-domain technique is required. Furthermore, spatial-domain techniques are able to utilize the apriori information necessary to constrain the search space in inverse ill-posed problems such as superresolution. In this experiment, a genetic algorithm method is implemented as a spatial-domain superresolution technique.. Many trials were performed on a single set of low-resolution images in order to find a set of parameters and constraints that would efficiently produce a superresolved image. Genetic algorithms have been successfully implemented for other inverse ill-posed problems, and they are especially useful in solving the superresolution problem. The ability to incorporate various constraints and the abstraction of the genetic operators from the complex details of superresolution provide a good framework for superresolution.. Several results are displayed to demonstrate the ability of the genetic algorithm programmer to add various constraints to handle different aspects of the superresolution problem. In order to best solve the superresolution problem, all available a priori knowledge should be applied to the algorithm. For example, one challenge was to balance a data fidelity constraint and a smoothing constraint. If a smoothing constraint is too strong, the high-frequency content will be limited (as seen in Figure 5 .1). Likewise, as noted by Baker and Kanade [1] , if the resolution enhancement factor is too large, the smoothness prior will produce a restored image with very little high-frequency content.. Some of the limitations in superresolution in general concern the existence and uniqueness of the solution. There are instances in which no solution exists, as when observation noise leads to conflicting constraints. Likewise there are instances in which the solution is not unique. It is necessary to determine conditions under which the iteration converges and whether or not that solution is unique. A powerful method for addressing the problems of nonexistence, nonuniqueness and continuity failings of ill-posed problems is through the use of a-priori knowledge which constrains the solution space. To elegantly and effectively tackle ill-posed problems, consider methods which utilize a-priori constraints to obtain regularized solutions to ill-posed inverse problems. This inclusion of a-priori constraints makes highquality solutions to even ill-posed inverse problems possible.. The genetic algorithm experiment is compared with the spatial domain method POCS. The advantages of the POCS restoration framework include simplistic framework. Some of the disadvantages of POCS include the possibility of the nonuniqueness of the solution, when the intersection of the convex constraint sets is not a point. Other disadvantages include the dependence of the solution on the initial estimate, and the expensive computational cost, with slow convergence.. Like POCS, the genetic algorithm method has a simplistic framework. The operations of the genetic algorithm are domainindependent. Unlike the POCS method, the genetic algorithm method is not as susceptible to nonuniqueness of the solution. Like POCS, the performance of the GA method is dependent on a good initial estimate. Also like POCS, the computational cost can be rather high, and convergence can be rather slow for such a complex problem.. The genetic algorithm solution for superresolution shares the advantages of other spatial-domain methods. The method allows flexible observation models, including degradation models resulting from problems such as missing pixels. The method also allows for inclusion of a-priori information for regularization, which is necessary for solving inverse ill-posed problems.. The genetic algorithm solution also shares several disadvantages of other spatial-domain methods. The method is more complex and computationally expensive than frequency-domain methods. Genetic algorithms and artificial intelligence have each been employed to solve many complex problems. In future work it would be ideal to combine the strengths of both genetic algorithms and artificial intelligence. In an evolutionary system, successful genes may propagate faster if the individuals are capable of learning. This principle, known as the Baldwin Effect, has been demonstrated in an artificial evolutionary system by [2] . The evolutionary system would benefit from having genomes with the ability to learn.. The superresolution system described in this experiment will be made more sophisticated by adding parameters to take into account the noise in the images. Also, the system can be expanded to test a more diverse collection of images, such as those in which the appearance of the target object begins to change more significantly as the camera orientation changes, as from a moving aircraft. More varied techniques will also be explored for producing the final superresolved image using registration and point spread function parameters. There were over 250 trials in this experiment, with significant differences in genetic algorithm parameters, regularization parameters, and general methodology. The genetic algorithm parameters include the size of the population, selection method used, probability of crossover, and probability of mutation. The regularization parameters include all types of constraints placed on the genomes and calculated values. For instance, the "salt and pepper" image was created when no constraints were placed on the mutation values-with a high rate of mutation, this resulted in many pixels out of normal range. The general methodology includes the method used to calculate the superresolved image. A thresholding operation was implemented to produce the best result, in order to produce more refined edges. The size of the superresolved image varied greatly in different trials. The best image had a size of 44x44 pixels, improved from a low-resolution size of 27x27 pixels.. The image registration and PSF were determined simultaneously. Then the superresolved image was produced. The optimization was toward a minimum error of the actual superresolved pixel values compared to pixel values of the registered low-resolution images. The convergence for best sample trial is also displayed. Early trials demonstrated the need for extensive experimentation with the smoothing prior. A lax smoothing prior allowed mutation to produce a "salt-and-pepper" effect (see Figure 2) . This was the result of a relatively high mutation rate, coupled with the failure to constrain new mutation values to normal pixel values. Pixel values for the grayscale image range from 0 to 255. However, a mutated superresolved pixel value must conform to the lowresolution values in the near neighborhood.. On the other hand, a smoothing prior that was too strong produced an image lacking the high-frequency details. For instance, if a pixel value is constrained to be exactly the same value as that of its neighbors, or a value that is very close to its neighbors, then the high definition values will be lost. The pixel values must be allowed to vary significantly in the range defined by the values of the low-resolution pixels. A sample result of a trial in which the smoothing prior was too constrained is displayed in Figure 5 .3. The convergence of this trial was fast compared with other trials (see Figure 5 .4).

O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O B O B I I O O O B O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B I O O O O O O O O O O O O O O O O O O B I O O O O B O O B I O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


MIGA, A Software Tool for Nonlinear System Modelling with Modular Neural Networks Abstract. This paper presents a software tool suitable for dynamic system modelling. The models generated by this tool are modular neural networks, see [1] . Each module behaves like a functional block and is connected to the other modules like in classical block diagrams. This tool allows the inclusion of a priori knowledge and, furthermore, to extract physical information from the models, once the system has learned. The modelling tool is capable of automatic model generation, parameter estimation and model validation. The identification of linear systems has a wellestablished theory and a wide range of efficient mathematical tools, including a variety of parameter estimation techniques, which provide a good basis for system analysis and controller design. This is not the case, however, for nonlinear systems. Since no generally applicable techniques exist for the analysis of these systems, the identification and controller design are usually performed on a case-by-case basis. The main available techniques for nonlinear modelling can be classified [2, 3] into three categories: functional se-ries methods, block oriented methods and black box methods.. Artificial Neural Networks (ANN) modelling techniques belong to the class of black box methods. Neural networks are structures used for knowledge synthesis by applying the learning-by-example paradigm. Different ANN structures have been used for system identification, e.g. [4] . The most commonly used neural architectures are feedforward networks with the backpropagation learning rule and recurrent networks with learning algorithms such as the real-time recurrent learning or the backpropagation-through-time algorithms.. One of the most frequent problems in using neural networks as black boxes is that the number of neurones increases rapidly with the order of the system and the learning process needs more time for each training pattern, and more training patterns, to be able to learn. When such problems arise in applied and theoretical sciences, the classical solution is to partition the problem: the divide and conquer approach. However, the ANN research community has carried out very little remarkable studies in this direction. Modular Artificial Neural Networks (MANN) are popular, in spite of this, in applications in which task subdivision can be made beforehand. A network is trained for each task and then the resulting nets are all combined to give the desired result, e.g. [5, 6] .. Two important studies on the design and training of modular neural networks are those by Jacobs and Jordan [7] and Happel and Murre [9] . The former describes an ensemble architecture composed of expert and gating networks. Expert networks compete to learn a certain aspect of the behaviour of a dynamic system while the gating network (or networks) decides which of the experts will contribute in the output. The work by Happel and Murre is based on a model of the cerebellar microcolumn. They call it CALM (Categorising Adaptive Learning Module) and it gives successful results in classification tasks when interconnected in a MANN fashion. This paper presents a software tool, called miga, useful for nonlinear system modelling with modular neural networks.. This tool does not rely entirely on neural learning to find a suitable model. It allows the inclusion of a priori structural knowledge about the relationship between the physical system and the model components. Furthermore, as far as highly structured networks are being used it is easier to extract information from the model, once the system has learned. It is naturally amenable to the representation and adaptive identification of control systems because of the adaptativity properties of ANN. Additionally, the resulting system's properties are easy to study because the neural network parameters are meaningful. Furthermore, the MANN approach overcomes, to a great extent, the problem of "explosion" in the number of neurones and, hence, in the learning time.. This architecture attempts to combine the ability of ANN to approximate any nonlinear function, with the clarity of information in block-oriented methodologies for system identification. She is a researcher at the Institute of Robotics and Industrial Computing, a joint research center of the Spanish National Research Council (CSIC) and UPC. Her research area is Control Engineering, more specifically Identification and Control of complex systems, with applications in industry. She takes part in several long-term cooperative projects with industry in this area.

O O O O O O O O O B I I O O O O O O O O O B I I O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Speeded-Up Robust Features (SURF) This article presents a novel scale-and rotation-invariant detector and descriptor, coined SURF (Speeded-Up Robust Features). SURF approximates or even outperforms previously proposed schemes with respect to repeatability, distinctiveness, and robustness, yet can be computed and compared much faster. This is achieved by relying on integral images for image convolutions; by building on the strengths of the leading existing detectors and descriptors (specifically, using a Hessian matrix-based measure for the detector, and a distribution-based descriptor); and by simplifying these methods to the essential. This leads to a combination of novel detection, description, and matching steps.The paper encompasses a detailed description of the detector and descriptor and then explores the effect of the most important parameters. We conclude the article with SURF's application to two challenging, yet converse goals: camera calibration as a special case of image registration, and object recognition. Our experiments underline SURF's usefulness in a broad range of topics in computer vision. The task of finding point correspondences between two images of the same scene or object is part of many computer vision applications. Image registration, camera calibration, object recognition, and image retrieval are just a few.. The search for discrete image point correspondences can be divided into three main steps. First, 'interest points' are selected at distinctive locations in the image, such as corners, blobs, and T-junctions. The most valuable property of an interest point detector is its repeatability. The repeatability expresses the reliability of a detector for finding the same physical interest points under different viewing conditions. Next, the neighbourhood of every interest point is represented by a feature vector. This descriptor has to be distinctive and at the same time robust to noise, detection displacements and geometric and photometric deformations. Finally, the descriptor vectors are matched between different images. The matching is based on a distance between the vectors, e.g. the Mahalanobis or Euclidean distance. The dimension of the descriptor has a direct impact on the time this takes, and less dimensions are desirable for fast interest point matching. However, lower dimensional feature vectors are in general less distinctive than their high-dimensional counterparts.. It has been our goal to develop both a detector and descriptor that, in comparison to the state-of-the-art, are fast to compute while not sacrificing performance. In order to succeed, one has to strike a balance between the above requirements like simplifying the detection scheme while keeping it accurate, and reducing the descriptor's size while keeping it sufficiently distinctive.. A wide variety of detectors and descriptors have already been proposed in the literature (e.g. [21, 24, 27, 37, 39, 25] ). Also, detailed comparisons and evaluations on benchmarking datasets have been performed [28, 30, 31] . Our fast detector and descriptor, called SURF (Speeded-Up Robust Features), was introduced in [4] . It is built on the insights gained from this previous work. In our experiments on these benchmarking datasets, SURF's detector and descriptor are not only faster, but the former is also more repeatable and the latter more distinctive.. We focus on scale and in-plane rotation invariant detectors and descriptors. These seem to offer a good compromise between feature complexity and robustness to commonly occurring deformations. Skew, anisotropic scaling, and perspective effects are assumed to be second-order effects, that are covered to some degree by the overall robustness of the descriptor. Note that the descriptor can be extended towards affine invariant regions using affine normalisation of the ellipse (cf. [31] ), although this will have an impact on the computation time. Extending the detector, on the other hand, is less straight-forward. Concerning the photometric deformations, we assume a simple linear model with a bias (offset) and contrast change (scale factor). Neither detector nor descriptor use colour information.. In section 3, we describe the strategy applied for fast and robust interest point detection. The input image is analysed at different scales in order to guarantee invariance to scale changes. The detected interest points are provided with a rotation and scale invariant descriptor in section 4. Furthermore, a simple and efficient first-line indexing technique, based on the contrast of the interest point with its surrounding, is proposed.. In section 5, some of the available parameters and their effects are discussed, including the benefits of an upright version (not invariant to image rotation). We also investigate SURF's performance in two important application scenarios. First, we consider a special case of image registration, namely the problem of camera calibration for 3D reconstruction. Second, we will explore SURF's application to an object recognition experiment. Both applications highlight SURF's benefits in terms of speed and robustness as opposed to other strategies. The article is concluded in section 6. The following presents both simulated as well as realworld results. First, we evaluate the effect of some parameter settings and show the overall performance of our detector and descriptor based on a standard evaluation set. Then, we describe two possible applications. For a detailed comparative study with other detectors and descriptors, we refer to [4] . SURF has already been tested in a few realworld applications. For object detection, its performance has been illustrated in [3] . Cattin et al. [7] use SURF for mosaicing human retina images-a task that no other detector/descriptor scheme was able to cope with. Taking this application to image registration a bit further, we focus in this article on the more difficult problem of camera calibration and 3D reconstruction, also in wide-baseline cases. SURF manages to calibrate the cameras even in challenging cases reliably and accurately. Lastly, we investigate the application of SURF to the task of object recognition.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I


Copula density estimation by total variation penalized likelihood with linear equality constraints A copula density is the joint probability density function (PDF) of a random vector with uniform marginals. An approach to bivariate copula density estimation is introduced that is based on a maximum penalized likelihood estimation (MPLE) with a total variation (TV) penalty term. The marginal unity and symmetry constraints for copula density are enforced by linear equality constraints. The TV-MPLE subject to linear equality constraints is solved by an augmented Lagrangian and operator-splitting algorithm. It offers an order of magnitude improvement in computational efficiency over another TV-MPLE method without constraints solved by log-barrier method for second order cone program. A data-driven selection of the regularization parameter is through K-fold cross-validation (CV). Simulation and real data application show the effectiveness of the proposed approach. The MATLAB code implementing the methodology is available online. A bivariate copula density c (u, v) , [u, v] Note that (P2) and (P4) implies (P3), so (P3) is redundant.. A bivariate copula C(u, v) defined on the unit square [0, 1] 2 is a bivariate cumulative distribution function (CDF) with univariate standard uniform margins:. Sklar's Theorem (Sklar 1959) states that the joint CDF F (x, y) of a bivariate random variable (X, Y ) with marginal CDF F X (x) and F Y (y) can be written as F (x, y) = C(F X (x), F Y (y)), where copula C is the joint CDF of (U, V ) = (F X (X), F Y (Y )). This indicates a copula connects the marginal distributions to the joint distribution and justifies the use of copulas for building bivariate distributions.. Let (X 1 , Y 1 ), . . . , (X n , Y n ) be a random sample from the unknown distribution F of (X, Y ). We wish to estimate aspects of the joint distribution of X and Y , in particular, the copula density function c (u, v) .. The copula density estimation has been mostly studied in a parametric framework, whereby c(u, v) is assumed to be a member of a copula family determined by a few parameters (for example, Shih and Louis(1995) ). The parametric copula density estimation problem is then essentially reduced to estimate the few parameters that determine the copula. We propose here to estimate the bivariate copula density non-parametrically. For practitioners, nonparametric estimates could be used as the first step toward selecting the right parametric family.. Nonparametric estimation of copula and its density does not assume a specific parametric form for the copula and the marginals and thus provides great flexibility and generality. Nonparametric estimators of a bivariate copula density using kernels have been suggested by Gijbels and Mielniczuk (1990) and by Fermanian and Scaillet (2003) . Kernel estimators have a severe drawback as they require a very large amount of data (page 195, Malevergne and Sornette, 2006) . Sancetta and Satchell (2004) employed techniques based on Bernstein polynomials. Hall and Neumeyer (2006) used a wavelet estimator to approximate the copula density. Autin et.al. (2009) dealt with the copula density estimation using wavelet methods by adaptive shrinkage procedures based on thresholding rules.. What does a copula density c(u, v) look like? In one extreme, when U and V are independent of each other, c(u, v) = 1. When U and V are dependent, c(u, v) can be smooth, have sharp boundaries, or even be unbounded. It is reasonable to assume that the total variation (TV) of c (u, v) , or at least its discrete version, is bounded. In practice, we often estimate and display the density in a finite grid. We propose a maximum penalized likelihood estimation (MPLE) with TV penalty method. This method is capable of capturing sharp changes in the target copula density, suffering less from edge effects when the copula density can be unbounded at boundaries in some statistically important cases.. The TV penalty based MPLE for copula density was proposed in Qu et al.(2009) , where the penalty term is the TV of the log density, and the unity requirement for a density function is imposed. However, the properties (P2) (P3) and (P4) are not enforced. In fact, we are not aware of any method that explicitly imposes the properties (P1-P4). The main reason behind this is probably related to the difficulty of the induced estimation or optimization procedure. In this paper, we enforce the properties (P2), (P3), and (P4) as linear equality constraints for the discretized copula density. We solve the problem of minimizing penalized negative log likelihood with TV penalty subject to linear equality constraints by an augmented Lagrangian and operatorsplitting algorithm. The effectiveness of our method is illustrated through numerical experiments.. Density estimation by TV penalized likelihood has been proposed by several groups of researchers. Koenker and Mizera (2007) used the TV of the derivative of the log density as the penalty in the univariate case and TV of the log density defined in a triogram in the bivariate case. Sardy and Tseng (2009), Mohler et al.(2009) We presented a TV penalized maximum likelihood copula density estimate subject to the the constraints that the marginal distributions are standard uniforms. The linear equality constrained (ĉ λ , c) ) over 100 replications for n = 500, m = 32. Gaussian 0.00 TV-best-BLF 0.000 (0.000) 0.000 (0.000) 0.000 (0.000) 0.000 (0.000) The theoretical questions such as the consistency and convergence rate of the estimator wait to be investigated. Few work exists regarding the asymptotic properties of TV regularized estimators, The MATLAB code implementing the method is available on the authors's website. The second author's work was supported in part by NSF Grant DMS-07-48839, ONR Grant N00014-08-1-1101, and an Alfred P. Sloan Research Fellowship.

B I I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O B I I I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


SURE-based optimization for adaptive sampling and reconstruction : Comparisons between greedy error minimization (GEM) [Rousselle et al. 2011 ] and our SURE-based filtering. With SURE, we are able to use kernels (cross bilateral filters in this case) that are more effective than GEM's isotropic Gassians. Thus, our approach better adapts to anisotropic features (such as the motion blur pattern due to the motion of the airplane) and preserves scene details (such as the textures on the floor and curtains). The kernels of both methods are visualized for comparison.We apply Stein's Unbiased Risk Estimator (SURE) to adaptive sampling and reconstruction to reduce noise in Monte Carlo rendering. SURE is a general unbiased estimator for mean squared error (MSE) in statistics. With SURE, we are able to estimate error for an arbitrary reconstruction kernel, enabling us to use more effective kernels rather than being restricted to the symmetric ones used in previous work. It also allows us to allocate more samples to areas with higher estimated MSE. Adaptive sampling and reconstruction can therefore be processed within an optimization framework. We also propose an efficient and memory-friendly approach to reduce the impact of noisy geometry features where there is depth of field or motion blur. Experiments show that our method produces images with less noise and crisper details than previous methods. Monte Carlo (MC) integration is a common technique for rendering images with distributed effects such as antialiasing, depth of field, motion blur, and global illumination. It simulates a variety of sophisticated light transport paths in a unified manner; it estimates pixel values by using stochastic point samples in the integral domain. Despite its generality and simplicity, however, the MC approach converges slowly. A complex scene with multiple distributed effects usually requires several thousand expensive samples per pixel to produce a noise-free image.. Adaptive sampling and reconstruction (or filtering, used interchangeably in the paper) are two effective techniques for reducing noise. Given a fixed budget of samples, adaptive sampling determines the optimal sample distribution by concentrating more samples on difficult regions. To decide which pixels are worth more effort, we require a robust criterion for measuring errors. Accurate estimation of errors is challenging in our application because the ground truth is not available. Reconstruction algorithms, in contrast, properly construct smooth results from the discrete samples at hand. One key issue that reconstruction must resolve is how to select the filters for each pixel, as the optimal reconstruction kernels are usually spatially-varying and anisotropic. Recently, approaches have been developed to address the challenge of spatially-varying filters Rousselle et al. 2011] , producing better results than those that use a single filter across the whole image. However, these methods are limited to symmetric filters and do not work well for scenes with anisotropic features such as highfrequency textures on the floor and curtains in Figure 1 .. We here propose an adaptive sampling and reconstruction algorithm to improve the efficiency of Monte Carlo ray tracing. The core idea is to adopt Stein's Unbiased Risk Estimator (SURE) [Stein 1981 ], a general unbiased estimator for mean squared error (MSE) in statistics, to determine the optimal sample density and per-pixel reconstruction kernels. The advantages of using SURE are twofold. For one, it provides a means by which to measure the quality of arbitrary reconstruction kernels -not just those that are symmetric (e.g. isotropic Gaussians used in greedy error minimization (GEM) [Rousselle et al. 2011] ). As such, it allows for the use of more effective filters such as cross bilateral filters and cross nonlocal means filters. Another advantage is that the per-pixel errors estimated by SURE can be used to guide further sample distribution. Thus more samples can be allocated to difficult regions. In addition to applying SURE, we propose an efficient and memoryfriendly approach to maintain the quality of cross bilateral filtering in the presence of noisy geometric features when rendering depthof-field or motion blur effects. We propose a normalized distance to alleviate the impact of noisy features, making the proposed method more robust to all types of distributed effects. Experiments show the proposed method provides significant improvements over previous techniques for adaptive sampling and reconstruction. We have presented an efficient adaptive sampling and reconstruction algorithm for reducing noise in Monte Carlo rendering by using Stein's Unbiased Risk Estimator (SURE) in the error estimation framework. For reconstruction, the use of SURE enables us to measure the reconstruction quality for arbitrary filter kernels. It does away with the limitation of using only symmetric kernels imposed by previous work. This freedom to use non-symmetric kernels significantly improves the effectiveness of the framework. When performing adaptive sampling, SURE can be used to determine the sampling density. Another contribution of this paper is an efficient and memory-friendly approach to detect noisy geometric features when rendering depth of field and motion blur. As a result, the proposed adaptive sampling and reconstruction method efficiently eliminates MC noise while preserving the vivid details of a scene. Experiments show the proposed method offers significant improvement over state-of-the-art approaches.. Global cross non-local means SURE cross non-local means MSE:0.085032 MSE:0.012010 Global SURE Reference Global SURE Reference Figure 11 : Comparison of cross non-local means filters without and with SURE-based framework. Compared to the global cross non-local means filter, our SURE-based optimization largely alleviates the oversmoothing problem. The sampling rate of the noisy input image is about 41 samples per pixel.. One possible future direction is to implement the proposed algorithms on GPUs for interactive applications. We also would like to extend the SURE-based framework to animation rendering. In the current algorithm, as there is no built-in mechanism specifically designed for temporal data, temporal coherence cannot be guaranteed. In practice, we have experimented with a naive approach that renders each frame independently. The results look good enough with only very subtle temporal flicking. However, a better way to handle animation would be to consider temporal samples and perform filtering in the spatial-temporal domain. Finally, it would also be interesting to adapt SURE to other rendering applications that require error estimation. Adaptive sampling and reconstruction. The seminal work of Mitchell [1987; 1991] over twenty years ago laid the foundation for methods for adaptive sampling and reconstruction. We categorize these techniques as image space methods, multidimensional methods, and adaptive filtering.. Image space methods estimate per-pixel errors with various criteria and allocate additional samples to difficult regions. These approaches usually are more general and can be used for various types of effects. Bala et al. [2003] combine edge information and sparse point samples to perform edge-aware interpolation. The quality of their results is highly dependent on the accuracy of edge detection. Overbeck et al. [2009] proposed adaptive wavelet rendering (AWR), a general wavelet-based adaptive sampling and reconstruction framework. They distinguish the sources of variances as the coarse level for distributed effects and the fine level for edges. These two types of noise are addressed separately by sampling the hierarchical wavelet coefficients adaptively; smooth images are reconstructed by thresholding wavelet coefficients. Recently, several approaches Rousselle et al. 2011] have been proposed to smooth out noise using multi-scale filters. Chen et al. [2011] focus on depth of field and describe a criterion to select spatially-varying Gaussian filters from a predefined filterbank based on the depth map. Rousselle et al. [2011] also use Gaussian filters to form a filterbank. Although their method uses an error minimization framework for adaptive sampling and reconstruction, and is more general, the framework can be used only for symmetric filters. We apply SURE to estimate MSE for more general filters and allow the use of more effective filters, which yield significant improvements.. Other approaches perform adaptive sampling and reconstruction in a multidimensional space. Hachisuka et al. [2008] distribute more samples to discontinuities in the high-dimensional space and reconstruct them anisotropically using structure tensors. Their approach achieves good quality but becomes less efficient as the number of dimensions increases. Approaches have also been developed that are based on transform domain analysis, focusing on specific distributed effects such as depth of field [Soler et al. 2009 ], motion blur [Egan et al. 2009] , and soft shadows [Egan et al. 2011] . Recently Lehitinen et al. proposed a novel method for reconstructing temporal light-field samples [2011] , and later extended it to handle the indirect light field for global illumination [2012] . By reprojecting samples along the sample trajectory in the multidimensional space, expensive samples can be reused. In general, to avoid the curse of dimensionality, multidimensional approaches usually focus on only one or two specific effects. These methods are able to generate better results for the effects they focus on because the anisotropy of the integrand is taken into account. Our method, however, is more general, efficient, and memory-friendly.. Some methods focus on reconstruction only. Xu et al. [2005] proposed smoothing out Monte Carlo noise with a modified bilateral filter with smoothed range values. Segovia et al. [2006] , Dammertz et al. [2010] , and Bauszat et al. [2011] focus on interactive global illumination. They exploit geometric properties such as depths and surface normals to identify edges. Shirley et al. [2011] use the depth buffer to help filtering, but target defocus and motion blur. Sen and Darabi [2012] proposed a general adaptive filtering approach based on information theory. By identifying the dependencies between random parameters on sample colors and scene features, they reduce the importance of sample values influenced by MC noise.. Denoising using SURE. Stein [1981] proposed an MSE estimator called Stein's Unbiased Risk Estimator (SURE) for estimators on samples with normal distributions. Donoho and Johnstone [1995] incorporate the estimator into a Wavelet shrinkage algorithm to reconstruct functions from noisy inputs. Recently, SURE has received much acclaim from the image denoising community and has been widely used to optimize denoising parameters [Blu and Luisier 2007; Van De Ville and Kocher 2009] .

O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I I O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O B I O O O O O O B I I I I O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O B O B O O O O O O O O O O B I I I I O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O