Using Shared Risk Link Groups to enhance backup path computation To cope quickly with all types of failure risks (link, node and Shared Risk Link Group (SRLG)), each router detecting a failure on an outgoing interface activates locally all the backup paths protecting the primary paths which traverse the failed interface. With the observation that upon a SRLG failure, some active backup paths are inoperative and do not really participate to the recovery (since they do not receive any traffic flow), we propose a new algorithm (SRLG structure exploitation algorithm or SSEA) exploiting the SRLG structures to enhance the admission control and improve the protection rate.With our algorithm, more flexibility is provided for the backup path selection since a backup path which protects against the failure of a link belonging to a SRLG does not systematically bypass all the links of that SRLG. Moreover, our algorithm permits to save more bandwidth because it does not allocate the bandwidth for the inoperative backup paths even if they are activated.Simulations show that our algorithm SSEA decreases the ratio of rejected backup paths and, it reduces in distributed environments the average number of messages sent to manage the bandwidth information necessary for the backup path computation. With the intense deployment of network real-time applications (voice over IP, TV, network games, etc.) in the last decade, fast recovery from network failures becomes desirable to ensure the communication service continuity. Hence, to cope quickly with network failures, local (proactive) protection pre-computing and often pre-configuring backup paths is preferred and adopted [1, 2] .. With the advent of MPLS (MultiProtocol Label Switching) [3] in the last decade, local protection is provided in efficient manner. In fact, MPLS offers a great flexibility for path (Label switched Path or LSP) selection and provides mechanisms allowing resource 1 reservations 2 and backup path preconfigurations. 3 Moreover and contrarily to the local protection in low layers (e.g. p_cycles [4] ), MPLS permits the separation of the traffic in several classes and to choose the classes of traffic to be protected. In order to cope with any physical failure 4 in a logical (MPLS/IP) level, three types of failure risks are defined: link, node and Shared Link Risk Group (SRLG). The first type of failure risk corresponds to the risk of a logical link failure due to the breakdown of an exclusive physical component of the logical link. The second type of failure risk corresponds to the risk of a logical node failure due to the breakdown of an exclusive physical component of the logical node. Finally, the third type of risk corresponds to a set of logical links that share a common physical component (optical fiber, crossconnect, etc.) whose failure may impact all links in the set [5] [6] [7] . Two types of backup LSP are defined for MPLS local protection [8] : Next HOP (NHOP) LSP and Next Next HOP (NNHOP) LSP. A NHOP LSP (resp. NNHOP LSP) is a backup path protecting against link failure (resp. node failure); it is setup between a primary node called Point of Local Repair (PLR) and one primary node downstream to the PLR (resp. to the PLR next hop) called Merge Point (MP). Such backup LSP bypasses the link (resp. the node) downstream to the PLR on the primary LSP. When a link failure (resp. node failure) is detected by a node, this later activates locally all its NHOP and NNHOP (resp. its NNHOP) backup LSPs by switching traffic from the affected primary LSPs to their backup LSPs.. In order to ensure that there is enough bandwidth after a failure (i.e. to guarantee the communication repair success), the backup paths should reserve the bandwidth they need beforehand. Besides, to decrease the bandwidth allocations and accept much more connection establishments, the practical hypothesis of single failure is often adopted [9, 6, 10, 11, 7, 12, 13] . With such hypothesis, all the backup paths protecting against failures of different components can share their bandwidth allocations (on their common links) since they cannot be active at the same time.. Several classical approaches [9, 6, 10, 11, 7, 12, 13] are developed to optimize the bandwidth allocated to the backup paths (called also protection bandwidth). In such approaches, the authors suggest to determine the cumulative bandwidth of the backup paths which would be activated on each link, after each possible failure. We note that a backup path is activated if its head-end router detects a failure on the protected link or node. As only the activate backup paths can really use their resources, the classical approaches propose to allocate the maximum of cumulative bandwidths of backup paths which could be active at the same time on each link.. Contrarily to the protection against link and node failure risks which uses only one backup path for each primary path, the protection against a SRLG risk employs several backup paths, one for each link which belongs to the primary protected path and to the SRLG. Moreover, for fast recovery from a SRLG failure, all the backup paths which protect against the failure of links belonging to the failed SRLG will be activated simultaneously. With the observation that some activated backup paths don't really use their resources (bandwidth) after a SRLG failure (because the traffic of the primary paths they protect was switched towards other backup paths which bypass their head-end routers), we propose in this article to enhance the protection quality and increase the bandwidth sharing by extending its application to some activated backup paths. In our approach, we explore the SRLG structures to determine the active backup paths which do not really use their resources after certain SRLG failures. Such active backup paths are in reality inoperative after such failures since they do not consume the bandwidth. In order to decrease the protection bandwidth that is allocated on each link, we propose to limit the concurrence for protection bandwidth to the backup paths which can be operative at the same time. In our proposition, more flexibility is provided for backup path selection since a backup path does not systematically bypass all the links sharing a SRLG with the protected link.. The rest of this article is organized as follows: In Section 2, we review some works related to the bandwidth sharing. In Section 3, we give a SRLG structure based classification of the backup paths that permits to improve the backup path computation. In our classification, the backup paths are grouped into two sets: the operative backup paths which receive the rerouted traffic after a failure, and the inoperative backup paths which do not receive any traffic after a failure, although they are active. In Section 4, we propose and describe a new algorithm (SRLG Structure Exploitation Algorithm or SSEA) which decreases the protection bandwidth allocations and provides more flexibility for the backup path selection. In Section 5, we give some ideas and propositions for the implementation of the SRLG structure exploitation algorithm in both centralized and distributed environments. In Section 6, we present and analyze some simulation results and we give, in Section 7, some conclusions. In this paper, we proved that it is possible to ensure the recovery from any single failure without forcing the (new) backup paths to bypass all the SRLGs containing the links to be protected. In fact, it is possible that a first active backup path does not receive traffic upon a SRLG failure since the traffic was already rerouted onto a second active backup path bypassing the head-end router of the first backup path. In such a case, the first backup path does not require any resource (bandwidth) and acts as an inoperative backup path upon that SRLG failure. However, the second backup path acts as an operative backup path that requires the bandwidth to reroute the traffic of the affected primary path. Obviously, only the operative paths (instead of all the activated backup paths) upon a failure of a SRLG should protect against the failure of that SRLG and can be in concurrence for a resource.. As the operative state of a backup path can be determined beforehand by taking the SRLG structures into account, we proposed a new and efficient approach to compute the backup paths. Our approach permits to increase the bandwidth availability (it decreases the protection bandwidth allocations) and provides more flexibility for the backup path selection (i.e. it improves the protection quality). It can be applied in both centralized and distributed environments. It also allows efficient design of networks since an effective combination of SRLGs can permit a significant reduction of the deployment cost without a decrease (or with a slight decrease) of the protection quality.. Simulations results show that the adoption of our approach decreases the number of rejected backup paths, increases significantly the relative gain in backup path rejection and saves the protection bandwidth by comparison with classical backup path computation algorithms. Moreover, when the backup path computations are distributed on the network routers, our approach (SSEA algorithm) reduces the average number of messages sent in the network to maintain the protection bandwidth information. With the increasing interest for local proactive protection in the last decade, several works [1, 2, 9, 6, 10, 11, 7, 12, 13] have been devoted to the determination of algorithms computing the backup paths. To minimize the quantity of bandwidth allocated on links while avoiding the bandwidth constraint violation (bandwidth insufficiency), the Backup Path Computation (BPC) algorithms require the knowledge of some information like the primary and backup paths, bandwidth allocations and protected risks.. Depending on the number of simultaneous failures that we would tolerate, the quantity of bandwidth reserved on each link for protection can be high (large number of simultaneous failures) or low (small number of simultaneous failures). Indeed, the number of simultaneous failures that can be processed successfully determine all the failure scenarios, which in turn control the number and structures of the backup paths which provide the protection. Due to the rarety of multiple failures 5 and the complexity to protect (in local and proactive manner) against this type of failure, and in order to increase the bandwidth availability (increase the bandwidth sharing), most of works in the literature consider only single failures [9, 6, 10, 11, 7, 12, 13] . With such type of failure (i.e. a single failure), the quantity of bandwidth that should be reserved on each link for protection, depends on the cumulative bandwidth of the paths which could be active at the same time after any single failure occurrence. Two strategies of bandwidth sharing are defined to reduce the protection bandwidth allocations: backup-backup bandwidth sharing and backupprimary bandwidth sharing. In the first strategy (backup-backup bandwidth sharing), the quantities of protection bandwidth allocated on links are decreased significantly with the application of 5 The most frequent multiple dependant failures are SRLG failures.. the bandwidth sharing between the backup paths [9, 6, 10, 11, 7, 12, 14] . This type of bandwidth sharing is made possible thanks to the hypothesis of single failures which ensures that some backup paths cannot be active (they do not use their bandwidth) at the same time. Thus, only the backup paths protecting against a same risk can be in concurrence for bandwidth allocation.. When a new backup path is being computed, control admission is applied on all its links to verify the bandwidth constraints. Two concepts are defined in [6] to ensure the respect of the protection bandwidth constraints: protection failure risk group and protection cost.. The protection failure risk group of a backup path b, denoted PFRG (b), is a set composed of all the risks whose failure activates the backup path b. With the definition of the function Act as follows (BPaths is the set of all the backup paths and Risks is the set of all the network failure risks):. Act : BPaths Â Risks ! f0; 1g ðb; rÞ#y ¼ 1 if b is active upon the failure of r; 0 otherwise:. We determine the protection failure risk group of a backup path b as follows:. PFRGðbÞ ¼ fr n r 2 Risks and Actðb; rÞ ¼ 1g:. The protection cost of a risk r on a link k, denoted d . For a SRLG risk srlg composed of link risks ðl 1 ; l 2 ; . . . ; l n Þ, the protection cost on a link k verifies always the following equality:. . To compute a new backup path b, only the unidirectional links k verifying the following inequality can be used:. where Pr k is the cumulated bandwidth of the backup paths traversing the arc k and C k is the capacity of the arc k.. To cope successfully with any single failure, the amount of protection bandwidth Bk k that should be reserved on each link k is determined as follows:. The backup-backup bandwidth sharing strategy improves substantially the bandwidth use and decreases the blocking probability. It is easy to be deployed in centralized environments where the unique BPCE (Backup Path Computation Element) knows all the bandwidth information (like protection costs, link capacities, cumulative primary bandwidths, etc.) that is required for the backup path computation. Indeed, the centralized BPCE can deduce the values of all the bandwidth parameters used in (3) from the structures and properties of the paths which it has previously computed. In distributed environments however, the advertisement of the bandwidth information that is required for the backup path computation is costly and could overload the network. Thus, using heuristics aggregating and/or reducing this bandwidth information before its advertisement in the network could give some interesting and practical solutions [9, 11, 7, 12, 14] . For instance, to decrease the size and frequency of the advertisement messages, the Kini's heuristic [9] suggests to approximate all the protection costs on a given unidirectional link by the highest protection cost on that link (i.e. 8ðk; rÞ : d k r is approximated by Max r ðd k r Þ). In this way, a given unidirectional link k can be used to establish a new backup path b if it verifies the following inequality: Pr k þ Max r ðd k r Þþ bwðbÞ 6 C k .. In the second strategy (backup-primary bandwidth sharing), another style of bandwidth sharing (bandwidth sharing between the primary and backup paths) is applied to decrease the protection bandwidth allocated on links. This type of sharing was proposed for the first time in [13] . It suggests to (pre)allocate the bandwidth freed by the deactivated (or bypassed) primary path segments upon a failure of a risk r to the backup paths which will be activated to recover from that failure. For instance, when a protected link (resp. an unprotected link) u-v traversed by a primary path p fails, a quantity of bandwidth equal to the bandwidth of p is freed on all the links located between the end nodes of the backup path repairing the primary path p (resp. on all the links located between the failed link and the destination node of the primary path p). Such freed bandwidth is then assigned to the backup paths which will be activated to recover from the failure of link u-v.. To avoid the violation of the bandwidth constraints with this second strategy, only the unidirectional links k verifying the following inequality can be selected to be in a new backup path b:. To cope successfully with any single failure, the amount of protection bandwidth Bk k that should be reserved on each link k is determined as follows:. where F k r is the total primary bandwidth freed on the link k after a failure of the risk r.. Compared to the first strategy of bandwidth sharing, this second strategy can enhance the bandwidth availability but it introduces several new drawbacks. Firstly, it complicates the resource preemption since the elimination of a primary path does not systematically free the associated bandwidth (because the bandwidth could be shared between a primary path and some backup paths). Secondly, it increases the amount of information that should be advertised in the network to enable the BPC to be performed in distributed environments. Concretely, in addition to the information required to perform the control admission with the first strategy, the application of the second strategy of bandwidth sharing requires the knowledge of the quantities of primary bandwidth freed on the links for all single failures.. Although there are some activated backup paths which do no receive any traffic after a SRLG failure, both the bandwidth sharing methods of the first and the second strategies allocate them bandwidth. This wastes bandwidth and blocks uselessly some protection requests.

O O O O O O O O B I O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I O O B I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B I O O O O O O O O O O O O O O O O O O O O


Unsupervised Learning of Image Manifolds by Semidefinite Programming Abstract. Can we detect low dimensional structure in high dimensional data sets of images? In this paper, we propose an algorithm for unsupervised learning of image manifolds by semidefinite programming. Given a data set of images, our algorithm computes a low dimensional representation of each image with the property that distances between nearby images are preserved. More generally, it can be used to analyze high dimensional data that lies on or near a low dimensional manifold. We illustrate the algorithm on easily visualized examples of curves and surfaces, as well as on actual images of faces, handwritten digits, and solid objects. Many data sets of images and video are characterized by far fewer degrees of freedom than the actual number of pixels per image. The problem of dimensionality reduction (Burges, 2005) is to understand and analyze these images in terms of their basic modes of variability-for example, the pose and expression of a human face, or the rotation and scaling of a solid object. The problem arises often in computer vision and pattern recognition Pless, 2004; Elgammal and Lee, 2004; Souvenir and Pless, 2005) . It is also of great interest to researchers in biological vision and computational neuroscience (Seung and Lee, 2000) .. Mathematically, we can view an image as a point in a high dimensional vector space whose dimensionality is equal to the number of pixels in the image (Turk and Pentland, 1991; Beymer and Poggio, 1996) . If the images in a data set are effectively parameterized by a small number of continuous variables, then they will lie on or near a low dimensional manifold in this high dimensional space (Lu et al., 1998) . Though one can imagine other types of hidden structure in ensembles of images, such as clusters (Gordon et al., 2003) or parts (Lee and Seung, 1999) , in this paper, we shall focus solely on the continuous structure that arises from image manifolds.. Beyond its applications in computer vision, manifold learning is best described as a problem at the intersection of statistics, geometry, and computation. The problem is simply stated: given high dimensional data sampled from a low dimensional manifold, what can we infer about the structure of the manifold? In the last few years, researchers have uncovered a large family of graph-based algorithms for computing faithful low dimensional representations of high dimensional data. These so-called spectral methods compute low dimensional embeddings from the top or bottom eigenvectors of an appropriately constructed matrix. Algorithms in this family-including Isomap (Tenenbaum et al., 2000) , locally linear (Roweis and Saul, 2000) , Laplacian eigenmaps (Belkin and Niyogi, 2003) , and others (Brand, 2003; Donoho and Grimes, 2003; Zhang and Zha, 2004 )-can reveal low dimensional manifolds that are not detected by classical linear methods, such as principal component analysis (Jolliffe, 1986) .. Our main contribution in this paper is a new algorithm for manifold learning based on semidefinite programming. Like other spectral methods for dimensionality reduction, it relies on efficient and tractable optimizations that are not plagued by spurious local minima. Interestingly, though, our algorithm is based on a completely different geometric intuition (and optimization) , and it overcomes certain well-known limitations of previous work. Our algorithm also reveals an interesting and unexpected connection to recent work on kernel methods in pattern recognition (Schölkopf and Smola, 2002) .. The organization of this paper is as follows. In Section 2, we review classical methods for linear dimensionality reduction, then introduce the nonlinear transformations that we consider for unsupervised learning of image manifolds. In Section 3, we show how to formulate manifold learning as a highly tractable problem in semidef-inite programming; this leads to a simple algorithm for computing low dimensional mappings that preserve the distances between nearby data points. In Section 4, we present experimental results on several data sets, including easily visualized examples of curves and surfaces, as well as images of faces, handwritten digits, and solid objects. In Section 5, we show how to relax the distance-preserving constraints in the original formulation of the algorithm; these relaxations can lead to improved solutions for sparsely sampled or noisy data. Finally, in Section 6, we compare our algorithm to previous approaches in manifold learning and conclude by describing several directions for future work. Our initial results for unsupervised learning of image manifolds seem promising. The algorithm in this paper has different properties than previous algorithms for manifold learning, and many of these properties can be construed as advantages. Like Isomap (and unlike LLE), its eigenvalue spectrum provides an estimate of the underlying dimensionality of sampled manifolds; unlike Isomap, however, it does not assume that the inputs are isometric to a convex subset of Euclidean space. The distance-preserving constraints in maximum variance unfolding can also be relaxed to encourage more aggressive solutions for dimensionality reduction.. The use of semidefinite programming has both advantages and disadvantages. Its main advantage is the simple manner in which we can express and enforce distance-preserving constraints. Such constraints can be tailored to particular applications of nonlinear dimensionality reduction. For example, a recent extension of maximum variance unfolding (Bowling et al., 2005) focused on images collected by a robot moving through a virtual environment. In this application, the robot moved only according to a finite set of actions, and it was assumed that transitions caused by the same actions should be mapped to jumps of equal distance in the embedding space. These additional constraints, naturally accommodated by the use of semidefinite programming, lead to Action Respecting Embeddings (ARE) that more accurately reflect the robot's motion and provide better support for path planning and localization.. The main disadvantage of semidefinite programming is the required amount of computation, which scales as O(n 3 + c 3 ), where n is the matrix size and c is the number of constraints (Borchers, 1999) . While our current implementation can handle data sets with up to n ≈ 2500 inputs, many applications in computer vision and pattern recognition involve much larger data sets. Thus our main task for future research is to develop shortcuts and approximations for large-scale implementations.. We are currently investigating a framework that has allowed us to reproduce many of the results in this paper in a small fraction of the original time . The framework is based on the observation that for well-sampled manifolds, the entire output inner product matrix K i j = y i · y j can be very accurately reconstructed from a much smaller submatrix of inner products between randomly chosen landmarks. In particular, we can write:. where L is the m × m submatrix of inner products between landmarks (with m n) and Q is an n × m linear transformation derived from solving a sparse set of linear equations. The factorization in (16) enables us to reformulate the semidefinite program in terms of the much smaller matrix L, yielding orderof-magnitude reductions in computation time. We are also exploring other ideas based on low-rank matrix factorizations (Burer and Monteiro, 2003) , distributed methods (Biswas and Ye, 2003) , the extrapolation of kernel matrices to out-of-sample inputs (Bengio et al., 2004) , and the post-processing of results from faster but less robust methods for manifold learning (Sha and Saul, 2005) .. In addition to the challenge of large-scale implementations, there are many other directions for future work. It would be interesting to investigate image manifolds which have spherical or toroidal geometries (Pless and Simon, 2002) . It might also be fruitful to study problems where more sophisticated similarity metrics (Simard et al., 1993; Belongie et al., 2002) have been developed using prior knowledge, rather than relying on naive nearest-neighbor computations using Euclidean distance. Finally, to the extent the our approach provides a new connection between work in manifold learning and kernel methods, we hope that it will lead to further advances in both areas. In this paper we have proposed an algorithm for unsuper-vised learning of image manifolds by semidefinite programming. Our work can be viewed as bridging two ongoing lines of research in machine learning, one on spectral methods for dimensionality reduction, the other on kernel methods for pattern recognition. After placing our work in the context of these two lines of research, we conclude by describing some open questions and plans for future work.

O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Design and evaluation of a compiler for embedded stream programs Applications that combine live data streams with embedded, parallel, and distributed processing are becoming more commonplace. WaveScript is a domain-specific language that brings high-level, type-safe, garbage-collected programming to these domains. This is made possible by three primary implementation techniques, each of which leverages characteristics of the streaming domain. First, we employ a novel evaluation strategy that uses a combination of interpretation and reification to partially evaluate programs into stream dataflow graphs. Second, we use profile-driven compilation to enable many optimizations that are normally only available in the synchronous (rather than asynchronous) dataflow domain. Finally, we incorporate an extensible system for rewrite rules to capture algebraic properties in specific domains (such as signal processing).We have used our language to build and deploy a sensornetwork for the acoustic localization of wild animals, in particular, the Yellow-Bellied marmot. We evaluate WaveScript's performance on this application, showing that it yields good performance on both embedded and desktop-class machines, including distributed execution and substantial parallel speedups. Our language allowed us to implement the application rapidly, while outperforming a previous C implementation by over 35%, using fewer than half the lines of code. We evaluate the contribution of our optimizations to this success. This paper presents the design and implementation of the WaveScript programming language. WaveScript supports efficient processing of high-volume, asynchronous data streams. Data stream processing applications-which typically process data in real time as it arrives-arise in a number of domains, from filtering and mining feeds of financial data to extracting relevant features from continuous signals streaming from microphones and video cameras.. We have used WaveScript in several applications requiring highvolume stream-processing, including water pipeline leak detection and road surface anomaly detection using on-board vehicular acPermission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. celerometers. In this paper, however, we focus on our most mature application: a distributed, embedded application for acoustic localization of wild Yellow-Bellied marmots which was deployed at the Rocky Mountain Biological Laboratory in Colorado during August, 2007. We described WaveScript, a type-safe, garbage collected, asynchronous stream processing language. We deployed WaveScript in an embedded acoustic wildlife tracking application, and evaluated its performance relative to a hand-coded C implementation of the same application. We observed a 1.38× speedup-which enabled a substantial increase in in-the-field functionality by allowing more complex programs to run on our embedded nodes-using half as much code. We also used this application to study the effectiveness of our optimizations, showing that the throughput of our program is substantially improved through domain-specific transformations and that our parallelizing compiler can yield near-linear speedups.. In conclusion, we believe that WaveScript is well suited for both server-side and embedded applications, offering good performance and simple programming in both cases. For the embedded case, its potential to bring high-level programming to low-level domains is particularly exciting. Stream processing (SP) has been studied for decades. In a typical SP model, a graph of operators communicate by passing data messages. Operators "fire" under certain conditions to process inputs, perform computation, and produce output messages. Many SP models have been proposed. For example, operators may execute asynchronously or synchronously, deterministically or nondeterministically, and produce one output or many.. In this section, we briefly review these major tradeoffs, and then discuss how existing stream processing systems are inappropriate Figure 1 . Diagram illustrating the major components of the marmot-detection application.. for our application domain. We also discuss related systems which inspired our approach.

B O B O O O O O O O B O O B O B O O B O B O O O O O O O O B I O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O B O O O O O O B O O O O O O O O B O O O B O B O O O O O O O O O O O O O O O B B O O O O O O O B B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O B O O O B O O B O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O B O B O O O O B O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O B O O O O


Pattern Codification Strategies in Structured Light Systems Coded structured light is considered one of the most reliable techniques for recovering the surface of objects. This technique is based on projecting a light pattern and imaging the illuminated scene from one or more points of view. Since the pattern is coded, correspondences between image points and points of the projected pattern can be easily found. The decoded points can be triangulated and 3D information is recovered. We present an overview of the existing techniques, as well as a new and definitive classification of patterns for structured light sensors. We have implemented a set of representative techniques in this field and present some comparative results. The advantages and constraints of the different patterns are also discussed. Surface reconstruction is one of the most important topics in computer vision due to its wide field of application. Some examples of applications are range sensoring, industrial inspection of manufactured parts, reverse engineering, object recognition and 3D map building. Among all the ranging techniques [1] , stereovision is based on imaging the scene from two or more points of view and then finding correspondences between the different images in order to triangulate the 3D position. Triangulation is possible if cameras are previously calibrated [2, 3] . However, difficulties in finding the correspondences arise, even when taking into account epipolar constraints. Coded structured light consists of replacing one of the two cameras by a device that projects a light pattern onto the measuring surface. The most commonly used devices are LCD video projectors, although previously the most typical were slide projectors. Such devices project an image with a certain structure so that a set of pixels are easily distinguishable by means of a local coding strategy. Thus, when locating such coded points in the image grabbed by the remaining camera, their correspondence problem is solved with no need for geometrical constraints. The projecting images are called patterns, since they are globally structured. This paper presents a comprehensive survey on coded structured light techniques, which updates the review presented in [4] and proposes a new, consistent and definitive classification. The paper focuses on the different coding strategies used in the bibliography and reproduces the experimental results of several techniques in order to evaluate and compare their accuracy and analyze their applicability. This article is structured as follows: firstly, the classification is presented in section 2. Secondly, in section 3 techniques based on projecting multiple patterns are explained. In section 4, techniques exploiting the spatial neighborhood paradigm are presented. Next, in section 5, coding strategies using direct codification are also explained. In section 6, the experimental results obtained with a set of implemented techniques are presented. Concluding, in section 7, a discussion about the advantages and drawbacks of every subgroup of techniques is included. In addition, general guidelines for choosing the most suitable technique (given the specifications of an application) are proposed. We have presented a comprehensive survey of coded structured light techniques. A new classification of the surveyed techniques has been proposed from the point of view of the coding strategies used to generate the projected patterns. Time-multiplexing was the first paradigm of coded structured light used to obtain 3D data from an unknown surface. The advantages of these techniques are the easy implementation, the high spatial resolution and the accurate 3D measurements that can be achieved. The main drawback is their inapplicability to moving surfaces since multiple patterns must be projected. Techniques based on projecting stripe patterns encoded with Gray code can obtain very good accuracy, but the maximum resolution cannot be achieved. In order to obtain maximum resolution, a technique based on the combination of Gray code and Phase shifting must be used. In this subgroup, the technique proposed by Gühring [19] must be highlighted. Its drawback, however, is the large number of projecting patterns (32 patterns when using maximum resolution). If maximum spatial resolution is not the principal aim of the application, but rather the minimization of the number of projecting patterns, a technique based on n-ary codes is appropriate. Such methods obtain an accuracy equal to or even better than a Gray code approach, reducing exponentially the number of projecting patterns. For example, a Gray code technique based on the projection of £ patterns can encode ¦ b© stripes, while an n-ary technique only requires patterns and ¢ grey levels or colors to obtain such resolution, for ¢ $ . However, the system using n-ary codes must be calibrated in order to correctly differentiate among the set of grey levels or colors used. If a good calibration cannot be achieved, then it is recommended to decrease the number of grey levels or colors by projecting more patterns.. Spatial neighborhood coding is the second big group of coded structured light techniques. The advantage compared with time-multiplexing is that such strategy permits, in most cases, moving surfaces to be measured. However, since the codification must be condensed in a unique pattern, the spatial resolution is lower. Moreover, local smoothness of the measuring surface is assumed in order to correctly decode the pixel neighborhoods. Since this local smoothness is not always accomplished, errors in the decoding stage can arise producing errors in the 3D measurements. In order to minimize such errors, the algorithms of the decoding stage must be more robust, so that the whole complexity of the technique increases. Techniques which define the neighborhoods empirically usually present pattern periodicity or repetition of neighborhoods, which is not recommended. Such problems have been eliminated by strategies based on De Bruijn sequences and M-arrays.. Techniques based on a unique pattern coded using a De Bruijn sequence have a trade off between the length of the sequence, i.e. the resolution of the system, the number of colors involved and the size of the window property. Most of these methods use either horizontal or vertical windows with a limited size in order to preserve the local smoothness assumption of the measuring surface. If the window size is not too big (in our opinion a good limit is about 10% of the sequence length), more than two colors must be used with the aim of preserving a good resolution. The number of colors used increases the noise sensitivity when measuring colorful scenes. Using up to 6 colors is not very problematic. With 5 colors and a window size of 3, a resolution of ¢ ¦ c b slits per pattern (similar to a Gray code system based on 7 patterns) can easily be achieved with a robust decoding stage. The most complete technique that can be found in the bibliography is the one proposed by Zhang et al. [13] . This technique takes into account that disorders between elements of the sequence can occur when projecting the pattern. The solution proposed is based on multi-pass dynamic programming, which seems the most robust way to recover the original sequence. Besides, techniques based on M-arrays are more difficult to generate. However, since every coded point has both row and column codewords, a higher degree of redundancy is included. In order to take advantage of this redundancy, an additional step must be programmed in the decoding stage for validating the codeword of every coded point. Similar trade offs to the ones involved when using De Bruijn sequences also appear with M-arrays-based patterns. The segmentation complexity of the observed patterns in such techniques must also be addressed. The most typical representations of an Marray in a pattern are the grid representation and the array of dots. In our opinion, the grid representation can be segmented more easily by edge detection. The encoded points are the intersection of edges, so they can be found very accurately. In addition, when projecting dots, their mass centers must be located. So it is important to detect when a dot appears only partially in the image, since its mass center will be incorrect. Moreover, the grid techniques allow adjacent cross-points to be located by tracking the edges, while with the dot representation, some sort of euclidian distance must be used to locate the neighbors of a given dot.. Notice that a technique based on spatial neighborhood can always be translated to a time-multiplexed technique by expressing the colors in binary intensity levels distributed over a sequence of patterns.. Direct coding techniques are useful for achieving large spatial resolution and few projecting patterns. However, these techniques present a lot of drawbacks. Firstly, the limited bandwidth of LCD projectors provokes integration of intensities over adjacent pixels. Secondly, variations of light intensities due to the different colors and depths of the measuring surface. Finally, the error quantization introduced by the camera, which is very sensitive to noise. Therefore, the correct identification of every projected intensity or color is not easy to achieve. In most cases, the use of such techniques requires a device that projects a unique wavelength for every grey level or color. Therefore, LCD projectors are not suitable for such a purpose. Some authors use nonstandard optical devices to polarize white light, producing monochromatic light planes. Furthermore, since a large spectrum of wavelengths are used, cameras with large depth-per-pixel must be considered (about 11 bits per pixel) for accurate quantization. Notice that most of these techniques cannot measure moving scenes because they need additional patterns to normalize intensity or color. In addition, these techniques are usually limited to color-neutral surfaces. Nevertheless, some techniques that can be implemented with an LCD projector and a standard CCD camera were proposed by Wust and Capson [53] and Sato [52] . Moreover, both techniques are theoretically capable of reconstructing colorful surfaces, and the technique by Wust and Capson can also measure moving surfaces. Accordingly, experimental results given by the technique by Sato showed that, in the synthesized image, most part of the surface colors are eliminated.. Finally, the depth per pixel used by a coded structured light technique is also an important parameter. The noisier the application environment is where the technique will be applied, the smaller the number of grey levels or colors used should be. Therefore, time-multiplexing techniques based on binary patterns are the most robust against noise. However, when increasing the number of grey levels or colors, differentiating the slits becomes more difficult. The non-linearities of the light spectrum of the projector and the spectral response of the cameras, and the non-uniform albedo of the measuring surface mean that the read colors hardly match with the projected ones. In order to overcome this problem, a full colorimetric calibration procedure should be considered. The illumination model proposed by Caspi et al. [12] or even a simple linear normalization may be a good solution.

O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A control-theoretic approach to distributed discrete-valued decision-making in networks of sensing agents Abstract-We address the problem of global sensor fusion for the purpose of distributed decision-making, from a control-theoretic perspective. In particular, we introduce a quasi-linear stochastic distributed protocol, using which a network of sensing agents can reach agreement in order to take a collective action. Using control-theoretic methods, we design the parameters of our protocol-which include weights in the local update rules used by the agents and a finite stopping time-to achieve agreement in a fair and rapid manner. We show analytically that the developed protocol achieves fair agreement with certainty in the noise-free case and achieves fair agreement with high probability even in the presence of communication noise and assuming very little information storage capability for the agents. Our development is illustrated throughout with a canonical example motivated by autonomous vehicle control. I N many application areas, networks of agents with sensing capabilities are required to integrate their individual observations and to make decisions or take actions collectively based on this fusion of information (e.g., [1] ). The task of information fusion and decision-making in these networks is often complicated by the essential decentralization of the network dynamics and control: Power/cost constraints and security/reliability concerns dictate that communication is short-range and, in some cases, that decision-making is done by the individual agents. For instance, swarms of military vehicles, such as unmanned aerial vehicles (UAVs), may need to collectively decide on the presence or absence of a target and make a decision on whether to destroy the target, in a totally distributed manner.. In this paper, we put forth the viewpoint that it is useful to consider the information-fusion and decision-making tasks of networks with sensing agents 1 jointly, as a decentralized stabilization or agreement problem. In pursuing this control-theoretic viewpoint, we propose a stochastic protocol for decision-making or agreement that is particularly suitable given typical characteristics of networks with sensing agents-i.e., networks with agents that operate at low power and with little memory, are subject to communication failures, and are seeking to make discrete-valued decisions, such as whether or not to attack a target. By formulating the developed protocol in terms of the influence model (see [2] , [3] ), we are able to analytically determine the performance characteristics of our protocol and to relate the protocol's performance with the network topology and fault characteristics.. Our work draws on and aims to contribute to both the literature on sensor networking and the control-theoretic study of agreement-protocol design. Qi et al. [1] review recent developments and challenges in distributed sensor networking. Several recent works have recognized the energy savings and scalability engendered by use of distributed algorithms (e.g., [4] , [5] , [6] ) in sensor networks, providing broad motivation for our study. Of particular interest to us, Krisnamachari and lyengar [4] develop a distributed algorithm that uses information fusion for a specific application-event region detection-in a manner that suppresses faults and, hence, allows commitment to a decision and ensuing action. We also seek to achieve agreement among agents for the purpose of decisionmaking but take the perspective that these agents begin with heterogeneous opinions about the issue to be decided on, not only because of faults, but because of heterogeneity in their observation capabilities and, perhaps, intrinsic differences in their motives. Thus, in contrast to [4] , we seek distributed algorithms for which the decided-on opinion of each agent takes into account the initial opinions of agents throughout the network, rather than of only a set of nearby neighbors.. Agreement protocols are iterative algorithms, using which a network of agents with initially different opinions can reach a collective decision (e.g., [7] , [8] ) and, hence, take an action in a distributed manner. Agreement has been studied in the computer science literature for several years-see [7] for a thorough introduction. Recently, a control-theoretic viewpoint on agreement has been developed (e.g., [8] , [9] , [10] ); this control-theoretic viewpoint is also deeply connected with the more general graph-theoretic study of For information on obtaining reprints of this article, please send e-mail to: tmc@computer.org, and reference IEEECS Log Number 0323-1204.. 1. We use the terminology "networks with sensing agents" rather than "sensor networks" because we envision our work as applicable to not only distributed and wireless sensor networks but also to networks that combine sensing with other dynamics (such as networks of UAVs). We also use the terminology to clarify that the networks that we consider do not necessarily contain a large number of agents. stabilization in distributed dynamic systems (e.g., [11] , [12] ). The control-theoretic viewpoint has the advantages of facilitating development of graphical conditions for agreement and of allowing characterization of protocol performance using linear system analysis techniques. Our previous work [10] is concerned with designing the dependence of the agreed-upon value on the initial opinions of the agents. This idea of agreement-law design in [10] plays a significant role in our current study, since we are interested in protocols that fairly 2 weight the initial opinions of the agents in making a collective decision.. We believe that several extensions of the controltheoretic studies on agreement and stabilization ( [8] , [9] , [11] , [10] , [12] ) are required for broad application to various networks with sensing capabilities:. . Agreement among discrete-valued opinions should be considered, since many networks with sensing capabilities perform (discrete-valued) detection or classification tasks as well as binary decisionmaking. With the exception of our previous work [10] , we do not know of any control-theoretic study of agreement that considers discrete-valued opinions. We focus on agreement upon discrete-valued opinions in this paper. . A stochastic protocol is needed in order to achieve agreement in a manner that equitably weights the initial opinions of the agents. As will be discussed further, a stochastic protocol can also be simpler to implement than some of the protocols described in the literature. To this end, we pursue a stochastic protocol in this work. . In many networks with sensing capabilities, faults in the communication of information may occur (often because of severe limitations in transmitter power and communication bandwidth) and, hence, communication faults should be considered explicitly in our protocol design. Our analysis of the developed agreement protocol explicitly considers stochastic faults in communication. . When the occurrence of faults is combined with limitations in the storage capabilities of agents and/ or a requirement of fast decision-making, loss of the information contained in the initial opinions of the agents may result if too many iterations of the agreement protocol are used. Thus, a metric for stopping the agreement algorithm should be developed. In contrast to the previous control-theoretic studies of agreement (which define agreement in an asymptotic manner), we define a stopping time for our algorithm. The stochastic protocol for agreement/decision-making that we propose here has a special quasi-linear structure that permits significant characterization of its performance. In particular, the closed-loop dynamics of our network (i.e., the evolution of the opinions of the agents upon application of the agreement protocol) can be represented using a specially structured stochastic automaton known as the influence model ( [2] , [3] ). The special tractability of the influence model then allows us to check the fairness of our protocol (and, in turn, to design the parameters of the protocol for fairness) using our results from [10] and to choose an appropriate stopping time for the protocol. Further, the influence model representation permits justification of the algorithm's success in a limiting case and makes clear the connection between the the network topology and the performance of the agreement protocol.. The remainder of the paper is organized as follows: We conclude this section with a summary of the notation and terminology for graphs used in the paper. In Section 2, we formulate our model for a distributed network of sensing agents and introduce our stochastic strategy (protocol) for agreement or decision-making in this network. In Section 3, we use the influence model representation of the closedloop model to evaluate the convergence properties and initial-condition dependence (and, hence, fairness) of our agreement protocol and, in the process, identify a stopping time for our protocol. A discussion of the protocol's complexity is also included. Finally, Section 4 contains discussion and evaluation of our protocol and suggests directions of future work. Examples are given throughout, for the purpose of illustration. We have developed a protocol for fair agreement or decision-making in a network of sensing agents that may be subject to faults in communication. As is made clear by the autonomous vehicle control example, our protocol holds promise as a tool for distributed decision-making. In particular, it builds on the current control-theoretic studies of agreement protocol design in several respects, including permitting agreement among discrete-valued opinions, explicitly modeling faults in observation, and introducing the notion of a stopping time.. From a broader perspective, we view our work as a first step toward developing a control theory for decisionmaking in networks of communicating or sensing agents. Our control-theoretic viewpoint allows for such advances as the design of the agreement law by bringing to bear linear systems and controls notions. We believe that our work (along with other recent control-theoretic studies of agreement, e.g., [8] ) also has the complementary benefit of introducing the problem of fair, distributed decisionmaking to the control community. While this article puts forth some advantages of a control-theoretic viewpoint on agreement protocol design, we believe much remains to be done in evaluating the developed protocol and in tying our results to those given in the computer science community (e.g., [7] ). 6 . For an analogous asynchronous algorithm, only unidirectional communication is needed: An agent can simply pass its opinion on to the other agents at randomly selected times. The following are a few specific directions that we believe should be pursued to better evaluate the application of our protocol and to further improve the protocol.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Analog placement with symmetry and other placement constraints In order to handle device matching in analog circuits, some pairs of modules are required to be placed symmetrically. This paper addresses this device-level placement problem for analog circuits and our approach can handle symmetry constraint and other placement constraints simultaneously. The problem of placing devices with symmetry constraint has been extensively studied but none of the previous works has considered symmetry constraint with other placement constraints simultaneously. Instead of handling the constraints by having a penalty term in the cost function to penalize violations, a unified method is proposed that, by adjusting the edge weights in a pair of constraint graphs, can try to satisfy all the placement and symmetry constraints simultaneously in a candidate floorplan solution. The maximum distance of the modules in a symmetry group from the corresponding symmetry axis will be minimized in this weight adjusting step, in order to minimize the total packing area. We have compared our method with the most updated results on this problem [2] when there are only symmetry constraints and results show that our approach can give solutions of better quality, in an acceptable amount of run time. We will also demonstrate the effectiveness of our approach in handling different types of constraints simultaneously by testing on data sets with both symmetry and other placement constraints, and the results are very promising. In the design of analog circuits, it is often required to have some devices (modules) to be placed symmetrically with respect to one or several common axes. The main reason is to match the layout-induced parasitics. Failure to do so may lead to higher offset voltages and degraded power-supply rejection ratio [5] . Placing devices symmetrically can also reduce the circuit sensitivity to thermal gradients.. The problem of placing devices with symmetry constraint has been extensively studied [6, 8, 10, 1, 12, 2] . Most of them used simulated annealing as an optimization engine based on a packing representation. We can classify these representations into two categories: (1) absolute representation, and (2) topological representations. In absolute representation [6, 8, 10] , modules are represented by their absolute coordinates on the chip plane. Since no restrictions is made to the relative positions between modules, illegal overlaps will occur. A penalty term in the cost function will be associated with those infeasible overlaps, which will be driven to zero in the optimization process. However the size of the solution space is huge in this absolute representation, which will affect the solution quality given a limited amount of search time. In topological representation, the relative positions between the modules are encoded. The solution space is much smaller in comparison with that of absolute representation, but complicated computations are needed for checking symmetry feasibility and adjusting the module positions to satisfy the constraints. Topological representations like sequencepairs [11] , O-tree [4] , B*-trees [3] have been applied to handle symmetry constraints in [1, 12, 2] . Comparisons in [2] have shown that the segment tree approach [2] has out-performed others in both solution quality and run time. TCG-S [9] has also been applied to handle symmetry constraints in [13] and the comparisons in [13] have shown that this TCG-S approach is comparable with the segment tree approach in solution quality but the run time is longer especially when the size of the data set increases.. None of the previous works has considered symmetry constraints and other general placement constraints simultaneously. In placement of analog circuits, there are also other placement constraints like device separation constraint (an upper bound on the separation distance between pairs of critically matched devices), alignment constraint, abutment constraint, boundary constraint, preplaced constraints and range constraint, etc. In this paper, we try to address this analog placement problem with both symmetry and other general placement constraints simultaneously. Sequence pair is used as the representation in the simulated annealing engine, but the approach can be applied to any other representations as long as they pack by constructing constraint graphs. Instead of handling the constraints by having a penalty term in the cost function to penalize violations, we will try to adjust the edge weights in the constraint graphs in a unified way so that all the placement and symmetry constraints can be satisfied simultaneously During this process, we will try to minimize the maximum distance of the modules in a symmetry group from the corresponding symmetry axis, in order to minimize the total packing area. We have compared our method with the segment tree approach [2] when there are only symmetry constraints and results show that our approach can give solutions of better quality. We have also demonstrated the effectiveness of our approach in handling different types of constraints by testing on some data sets with both symmetry and other placement constraints, and the results are all very promising.. In the following, we will first define the problem (section 2), then we will discuss the methodology used in our system (section 3). In section 4, we will describe the simulated annealing process employed, and the experimental results will be reported in section 5. The pseudocode below shows a summary of the steps to handle symmetry constraint.. Pseudocode Symmetry() // Given a pair of acyclic constraint graphs (H h , H v ) which are // already augmented with edges to handle other types of place-// ment constraints, this procedure either announces that the // symmetry constraints cannot be satisfied simultaneously or // further augments them to take into account the symmetry // constraints. In our approach, sequence pair [11] is used to represent a placement. In sequence pair, a packing is represented by a pair of permutations of the module names (s1, s2). If the relative positions of two modules A and B in s1 and s2 are . . . A . . . B . . . and . . . A . . . B . . . (. . . A . . . B . . . and . . . B . . . A . . .) respectively, A is on the left of B (A is below B). In each step of the annealing process, a candidate solution x = (s1, s2) will be generated. We will first have an initial scan to check if x will be a feasible solution satisfying all the constraints. Details of this scanning process to check the feasibility of a sequence pair will be given in a later section. After this initial scan, a pair of constraint graphs (H h , Hv) will be built according to the sequence pair to represent the relative positions between the modules. In the horizontal (vertical) constraint graph H h (Hv), the vertices represent the modules and the edges represent the relationship between the modules in the horizontal (vertical) direction, e.g., if A is on the left of B (A is below B), there will be an edge from A to B in H h (Hv) with weight w(A) (h(A)) where w(A) (h(A)) is the width (height) of module A. Additional nodes and constraint edges will be inserted into H h and Hv to enforce the required symmetry and placement constraints in some later stages. Some of those newly inserted edges have variable weights and we need to determine their weights to minimize the packing area and to ensure that no positive cycles will be created. Finally, if no positive cycles exist in the graphs, all constraints can be satisfied simultaneously and we will pack accordingly to obtain one feasible candidate solution. fig. 1 and details of each step will be given in the following sections.

O B O O O O B O O O O O O O O B I O O O O O O O O O O O O O O O B O O B I O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B O O O B O B I O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Caching on the World Wide Web With the recent explosion in usage of the world wide web, the problem of caching web objects has gained considerable importance. Caching on the web di ers from traditional caching in several ways. The non-homogeneity of the object sizes is probably the most important such di erence. In this paper we g i v e a n o verview of caching policies designed speci cally for web objects and provide a new algorithm of our own. This new algorithm can be regarded as a generalization of the standard LRU algorithm. We examine the performance of this and other web caching algorithms via event and trace driven simulation. The recent increase in popularity of the world wide web has led to a considerable increase in the amount of tra c over the internet. As a result, the web has now become one of the primary bottlenecks to network performance. When objects are requested by a user who is connected to a server on a slow network link, there is generally considerable latency noticeable at the client end. Further, transferring the object over the network leads to an increase in the level of tra c. This has the e ect of reducing the bandwidth available for competing requests, and thus increasing latencies for other users. In order to reduce access latencies, it is desirable to store copies of popular objects closer to the user. Consequently, w eb caching has become an increasingly important t o p i c 1 , 8 , 1 4 , 1 8 , 1 9 , 22] . Caching can be implemented at various points in the network. On one end of the spectrum, there is typically a cache in the web server itself. Further, it is increasingly common for a university or corporation to implement specialized servers in the network called caching proxies. Such proxies act as agents on behalf of the client in order to locate a cached copy of a object if possible. More information on caching proxies may be found in 17] . Usually caching proxies and web servers behave as secondary or higher level caches, because they are concerned only with misses left over from client caches. Such client c a c hes are built into the web browsers themselves. They may store only those accesses from the current i n vocation (so-called non-persistent cache), or they may retain objects between invocations. Mosaic, for example, uses a non-persistent c a c he. In this paper, we shall discuss general main memory cache replacement policies designed speci cally for use by w eb caches. The results are applicable to web server, proxy and client c a c hes. One of the key complications in implementing cache replacement policies for web objects is that the objects to be cached are not necessarily of homogeneous size. For example, if two objects are accessed with equal frequency, the hit ratio is maximized when the replacement policy is biased towards the smaller object. This is because it is possible to store a larger number of objects of smaller size. In the standard least recently used ( L R U) caching algorithm for equal sized objects we maintain a list of the objects in the cache which is ordered based on the time of last access. In particular, the most recently accessed object is at the top of the list, while the least recently accessed object is at the bottom. When a new object comes in and the cache is full, one object in the cache must be pruned in order to make room for the newly accessed object. The object chosen is the one which w as least recently used. Clearly the LRU policy needs to be extended to handle objects of varying sizes. In addition to non-homogeneous object sizes, there are several other special features of the web which need to be considered. First, the hit ratio may not be the best possible measure for evaluating the quality o f a w eb caching algorithm. For example, the transfer time cost for transfering a large object is more than that for a small object, though the relationship is typically not straightforward. It will depend, for instance, on the distance of the object from the web server. Furthermore, web objects will typically have expiration times. So, when considering which objects to replace when a new object enters a web cache, we m ust consider not only the relative frequency, but also factors such as object sizes, transfer time savings and expiration times. A related issue to that of replacement i s admission control. In other words, when should we allow an object to enter the cache at all? It may not always be favorable to insert an object into the cache, because it may l o wer the probability of a hit to the cache. We list below some cache replacement policies evaluated in 1, 26] . Each of them can be combined with an admission policy.. (1) LRU: In the most straightforward extension of LRU for handling non-homogeneous sized objects, one would prune o as many of the least recently used objects as is necessary to have su cient space for the newly accessed object. This may i n volve zero, one or many replacements. Thus, this extension of LRU t a k es size into account only peripherally while performing the cache replacement decisions. As we shall see, such a replacement policy turns out to be naive in practice.. (2) LRUMIN: This policy is biased in favor of smaller sized objects so as to minimize the number of objects replaced. Let the size of the incoming object be S. Suppose that this object will not t in the cache. If there are any objects in the cache which h a ve size at least S, w e remove the least recently used such object from the cache. If there are no objects with size at least S, then we start removing objects in LRU order of size at least S=2, then objects of size at least S=4, and so on until enough free cache space has been created.. (3) SIZE policy: In this policy, the objects are removed in order of size, with the largest object removed rst. Ties based on size are somewhat rare, but when they occur they are broken by considering the time since last access. Speci cally, objects with higher time since last access are removed rst.. Note that all of these policies take i n to account either the size or the time since last access or both. It was concluded in 1, 26] that policies which take i n to account the size tend to perform better than those which do not. This is because removing larger objects makes room for multiple smaller ones. In this paper we devise a web cache replacement policy which appears to achieve performance better than any of the above s c hemes. We describe a corresponding admission control policy as well. The scheme we propose is quite general purpose, is easy to implement, and works well on many di erent kinds of workloads. We brie y survey and categorize some additional cache replacement s c hemes for the web. The list below is certainly not exhaustive, though many replacement algorithms can be classi ed into one or more of the following categories.. (1) Direct extensions of traditional policies: Besides LRU, traditional policies such a s Least Frequently Used (LFU) and First In First Out (FIFO) are well-known cache replacement strategies for paging scenarios 24] . Just as with LRU, it is possible to extend these policies to handle objects of non-homogeneous size. The policy in 22] can be regarded as an LRU extension, though time since last access is rounded to the nearest day. The di culty w i t h such policies in general is that they fail to pay su cient attention to object sizes.. (2) Key-based policies: The idea in key-based policies is to sort objects based upon a primary key, break ties based on a secondary key, break remaining ties based on a tertiary key, a n d to employ a potentially general function of the di erent factors such as time since last access, entry time of the object in the cache, transfer time cost, object expiration time and so on. For example, the algorithm described in 7] employs a weighted rational function of the transfer time cost, the size and the time since last access. The algorithm described in 27] employs a weighted exponential function of the access frequency, the size, the latency to the server and the bandwidth to the server. The Least Normalized C o s t R eplacement (LNC-R) algorithm described in 23] employs a rational function of the access frequency, the transfer time cost and the size. This algorithm is certainly the most similar to our own scheme, which is also function-based.. The paper is organized as follows. In Section 2 we c o n c e n trate on formulating a theoretical optimization model for web caching which generalizes LRU. We devise an optimization model and show h o w this can be approximately solved by a simple heuristic. We call the policy derived here Size-adjusted L R U, o r SLRU. In Section 3 we show h o w t o m a k e this heuristic more easily implementable in practice. We call the resulting policy the Pyramidal Selection Scheme, o r PSS. Section 4 describes web-speci c extensions to the above t wo algorithms in order to handle general access costs and expiration times. In Section 5 we discuss an admission control policy. Results of event and trace driven simulations are presented in Section 6. In particular, we compare PSS with the LRU, LRUMIN and SIZE schemes. Finally in Section 7, we present a summary and conclusion.

B O O B I I O O O O O O O O B I I O O O B O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O B I I O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O