Image Denoising and Decomposition with Total Variation Minimization and Oscillatory Functions Abstract. In this paper, we propose a new variational model for image denoising and decomposition, witch combines the total variation minimization model of Rudin, Osher and Fatemi from image restoration, with spaces of oscillatory functions, following recent ideas introduced by Meyer. The spaces introduced here are appropriate to model oscillatory patterns of zero mean, such as noise or texture. Numerical results of image denoising, image decomposition and texture discrimination are presented, showing that the new models decompose better a given image, possible noisy, into cartoon and oscillatory pattern of zero mean, than the standard ones. The present paper develops further the models previously introduced by the authors in Vese and Osher (Modeling textures with total variation minimization and oscillating patterns in image processing, UCLA CAM Report 02-19, May 2002, to appear in Journal of Scientific Computing, 2003. Other recent and related image decomposition models are also discussed. Recently, a new concept and a new task have been introduced in image analysis: decompose a given image f : → R into two components u + v, such that u is a cartoon representation of f , a simplified piecewisesmooth approximation, while v is the oscillatory component, consisting of texture and noise. This new image analysis task has been first formulated in theory by Meyer [27] , and the authors of the present paper have proposed a first practical new model to achieve this task in [40] . In this present paper, we develop further the model introduced in [40] , discuss some of its variants, and illustrate its range of applications to image denoising, image decomposition into cartoon plus texture, as well as the capability of texture discrimination and texture segmentation. We also discuss other new related models from the literature, all inspired by [27] and [40] .. The canonical, standard, previous models in image analysis start from the assumption that an image f is the sum of two components, f = u + v, such that only u contains the important main features of f . These models represent u as a function of bounded variation (in BV ( ) or in a subset, SBV( )), allowing for discontinuities along curves, therefore sharp edges and contours in the image u. The residual v := f − u is modeled in the standard models by a function in L 2 ( ). The main canonical examples are the total variation minimization of Rudin-Osher-Fatemi [35] , Rudin-Osher [34] , and the Mumford-Shah segmentation model [29] . The first model decomposes f into u + v, such that u ∈ BV( ) and v := f − u ∈ L 2 ( ), while the second model gives a decomposition f = u + v with u ∈ SBV( ) (special functions of bounded variation), and v := f − u ∈ L 2 ( ), as in the ROF model. These and other related models retain only the piecewisesmooth component u from f , and remove away the texture + noise component v := f − u. Other models having related properties are [3, 6, 12, 33] , among other. Instead, we use here a more refined model for describing the textured component v. It means using a norm which is truly sensitive to oscillating patterns, and our choice is motivated by some unexpected properties of the Rudin-Osher-Fatemi model [35] , as it was remarked by Meyer. Also, some drawbacks of the standard models are reduced or eliminated in the new models. Also, more recent analysis favor the idea that the space BV( ) is not sufficient to model images with textures and small details properly, such as natural images. We refer the reader to Gousseau and Morel [19] , where an experimental study of the distribution of bilevels of natural images is presented, showing that their total variation blows up to infinity with the increasing of resolution. A related work by the authors of [19] together with L. Alvarez is [2] . Also, Mumford and Gidas [28] explain that natural images are better modeled by Schwartz distributions which are not measurable functions.. More precisely, we assume that we have a given image f ∈ L 2 ( ), where is an open and bounded domain on R 2 , with Lipschitz boundary. In order to recover u from f , in [34, 35] it has been proposed to solve the following convex minimization problem:. where BV( ) is the space of functions of bounded variation, λ > 0 is a positive tuning parameter and |∇u| denotes the total variation of u. This minimization problem has minimizers in the space BV( ) (see [1, 12] ; also [41] for a more general case, [4, 5] for other properties). New formulations using duality of more general cases are presented in [41] , as well as in [13, 14] for the TV minimization.. A unique minimizer u formally satisfies the EulerLagrange equation associated with the above minimization problem:. (for a more precise and rigorous calculation of the subdifferential of the functional, we refer the reader to [12, 41] ).. As Meyer pointed out [27] , the residual v = f − u in ROF can be expressed as v = div g, where g is a vector field given, in the sense of distributions, by. < ∞. Therefore, the residual v = f − u in the RudinOsher-Fatemi model is expressed, in the sense of distributions, as the divergence of a vector field g ∈ L ∞ ( ), i.e. as a generalized function, and this also belongs to L 2 ( ). It turns out that the space of generalized functions. is the space W −1,∞ ( ), the dual space to W 1,1 0 ( ). On the other side, for each u ∈ BV( ), its total variation |∇u| is finite and can be expressed as:. Meyer [27] suggests that, in order to better extract from f = u + v the cartoon part u ∈ BV( ), and to leave in v the noise or the textured components only, other weaker norms should be used for the residual f − u in (1) 2 and where the infimum is computed over all decompositions v = div g of v.. We have the inclusions. * , therefore the norm in G( ) is a weaker norm than the L 2 ( ) norm. In [40] and in this present paper, we propose a new model for image decomposition into cartoon plus texture. To model the cartoon component u, we use, as in [34, 35] The outline of the paper is as follows: in the next section we give the description of the proposed new models, and discuss some of the properties. In Section 3 we discuss other related recent models for image decomposition, inspired from [27] and [40] . Then in Section 4 we present numerical results and comparisons, illustrating the motivations for the new models. We end the paper with a short concluding section.. For related results on oscillatory patterns in nonlinear problems, we refer to [30] . For related image restoration models based on the total variation minimization model in a wavelet framework, we refer the reader to [25] and [10, 26] . There is a very large literature on texture modeling and analysis in the computer vision and image analysis community, so it is impossible to mention all related references. We only mention in addition [8, 11, 20-24, 32, 36-38, 42, 43] , among many other work. In this paper, we have presented a new, general and efficient class of models for image denoising and image restoration, which combine the total variation minimization model of Rudin-Osher-Fatemi for image restoration, with dual Sobolev spaces appropriate for oscillatory functions, following ideas introduced by Meyer. Numerical results for image denoising, image decomposition and texture segmentation have been presented. This paper is also an extension of the models introduced by the authors in [40] . His research contributions are in algorithm development and applications in high resolution shock capturing methods, level set methods, and PDE based methods in computer vision and image processing. A few examples are: the Engquist-Osher scheme, TVD schemes, entropy conditions, ENO and WENO schemes, numerical schemes for Hamilton-Jacobi type equations, total variation model for image denoising, images on implicit surfaces, the level set method in image processing and computer vision. He is co-author with R. Fedkiw of the book The Level Set Method and Dynamic Implicit Surfaces.

B I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Smoke Detection in Video: An Image Separation Approach Abstract Existing video-based smoke detection methods often rely on the visual features extracted directly from the original frames. In the case of light smoke, the background is still visible and it deteriorates the quality of the features. This paper presents an approach to separating the smoke component from the background such that visual features can be extracted from the smoke component for reliable smoke detection. Specifically, an image is assumed to be a linear blending of a smoke component and a background image. Given a video frame and its background, the estimation of the blending parameter and the actual smoke component can be formulated as an optimization problem. Three methods based on different models for the smoke component are proposed to solve the optimization problem. Experimental results on synthesized and real video data have shown that the proposed approach can effectively separate the smoke component and the smoke detection performance is significantly improved by using the visual features extracted from the smoke component. The presence of smoke often signals the onset or possibility of a fire event. Thus, early detection of smoke can serve as a warning for incidence of fire. Conventional point smoke detectors, which include photoelectric and ionization detectors, mainly detect the presence of certain particles generated by smoke and fire. Photoelectric detectors make use of photometry to detect the presence of these particles. Ionization detectors achieve this by means of monitoring the reduced quantities of ionized air molecules. Both methods depend on the transportation of the smoke towards the detector and sufficient concentration of the molecules or particles being present. There is a delay inherent in the transportation and this is exacerbated in outdoor scenario where there could be draught or wind. Smoke detectors used in open areas require to be in close proximity of the source in order to be effective. Apart from the limitation of proximity these detectors do not provide information about the location of the fire, its burning rate or other key indicators.. Compared to the methods described above, visual inspection and detection do not suffer similar drawbacks. Recent advances in real-time video-based surveillance techniques have made vision-based smoke detection a promising approach to early detection of fire. Vision-based smoke detection is suitable in both enclosed and open spaces and there is an additional benefit of being able to specify the location of the fire, its scale and intensity.. Most vision-based smoke detection techniques adopt a pattern recognition paradigm in which the input image or video is preprocessed and divided into blocks. For each block, salient features are extracted and employed to classify the block into smoke and non-smoke. The success of these techniques depends on identifying robust visual features that can characterize smoke and to this end, there are reported studies (Toreyin et al. 2005; Yuan 2008; Calderara et al. 2011) . Despite these efforts, vision-based smoke detection is still a challenge because of the difficulty in quantifying the visual characteristics of smoke. Some of the visual characteristics of smoke include irregular and deformable shape, irregular motion and varying degree of transparency. Unlike rigid objects, the irregular and deformable nature of smoke makes it difficult to extract geometrical or edge-based features. The motion of smoke, especially in the presence of draught or wind, can be irregular and somewhat random. Thus, well-known motion descriptors such as optical flow have not been very successful as features. The possibility of using texture as a feature descriptor for smoke is promising because of its dispersive distribution. However, one needs to account for the varying degree of transparency as the background covered by smoke may be included in the feature being extracted.. The potential offered by texture as feature to characterize smoke serves as a key motivation to pursue methods of dealing with the varying degree of transparency associated with smoke.Opaque objects, such as human faces and vehicles, act as solid objects and will obscure other objects lying directly behind them. However, smoke can be transparent and the level of transparency changes with the density of smoke. When smoke is heavy enough, it behaves as an opaque object and no visual information of the covered scene (background) is available. Light smoke, on the other hand, has a degree of transparency and the background is partially visible regardless of the color of the smoke. Texture features extracted from an image with light smoke will capture the visual information of both smoke and background, which renders such features ineffective in describing smoke. The images shown in Fig. 1 illustrate the possible varied effects different background images can have on the features extracted from light smoke.. We therefore seek to remove the effect of background image and extract textural features from the smoke component. A straightforward background subtraction method is ineffective in this case because of the gradual and slow temporal change in the density of smoke. When we consider the state-of-the-art background subtraction methods, most of the efforts are directed at background modeling. Given a video frame, each pixel is classified as either background or foreground pixel based on the background model. However, foreground pixels with light smoke certainly include background information. Figure 2 illustrates the presence of residual background information when a simple background subtraction method is used. This paper presents a novel approach to separating the smoke component from the background such that visual features can be extracted from the smoke component for reliable smoke detection. We proceed by assuming that the image under consideration is a linear blending of a smoke component and a background component, also referred to as pure smoke and background respectively hereafter. We capture the characteristics of the smoke component using three variants of the model. In the first model we take advantage of the structure of inter-pixel correlation within a smoke image and impose a "local smoothness" constraint. Algebraically, the smoke component may be a manifold embedded in a higher dimensional space. Hence in the second model the smoke component is restricted to be a linear combination of the bases obtained by principal component analysis. Using similar algebraic argument, we construct the third model such that the smoke component is sparsely represented using an overcomplete dictionary that can be learned from pure smoke samples. Given a video frame and its background, the estimation of the blending parameter and the actual smoke component is formulated and solved as an optimization problem.. Following this introduction, we present a review of existing smoke detection and image separation methods in Sect. 2. The image formation model and the formulation of the smoke component separation models are presented in Sect. 3. The solutions of the three models are also provided. A smoke detection framework based on the estimated blending parameter of the image formation model and the smoke component is described in Sect. 4. We generated synthesized smoke images and also used real smoke video to validate the proposed models. Experimental results based on both real and synthetic data are presented in Sect. 5 along with discussions. Some conclusions and perspectives on vision-based smoke detection research are presented in Sect. 6. In our evaluation we set the methods proposed by Toreyin et al. (2005) and Tian et al. (2011) as two baselines and compared the results of the proposed methods to the results they published. The two baseline methods extracted wavelet and LBP-based features from the original image f to perform smoke detection respectively. Further, as part of our evaluation, we compared our approach to the direct background subtraction in a bid to dispel the notion that it is a simple and appropriate method for smoke separation. The difference image between f and b was computed and LBP features were extracted for the experiments. Figure 18 shows the ROC curves of different methods for detecting different types of smoke. As can be seen, irrespective of whether local smoothness or sparse representation is used as the smoke component model, the proposed smoke detection methods based on image separation outperform the baseline methods. Furthermore regardless of whether smoke is heavy or light, fully or partly covering, significant improvements were achieved by using the features extracted from smoke component s compared to the case when features were extracted directly from the image f. Compared with the case when features were extracted from the difference images, it is noteworthy that the detection performance obtained was much worse than those achieved by the proposed approach; a result that justifies the unsuitability of background subtraction for smoke component separation.. Moreover, in many cases, the difference image could introduce or increase the background information and lead to poor detection performance when compared with the case when features were extracted directly from the image f as shown in Fig. 18a-c . In our experiments one exception was observed for the image category SI04 (Fig. 18d) when only less than half of image area was covered by smoke. The reason adduced for this observation is that in this case background information dominates in both f and b. Thus the resulting difference image becomes a homogeneous patch.. In all, it can be concluded that the comparative results presented here verify the effectiveness of the proposed approach. First, we review existing works on vision-based smoke detection and highlight the novelty of the proposed approach. In order to provide further context for the approach we have taken in this paper we also review some representative image separation techniques. Existing smoke detection methods mainly follow the traditional pattern recognition paradigm and characterize smoke by using visual signatures of smoke regions such as motion, color, edge and texture. For instance, an accumulative motion model has been proposed to capture the motion characteristics of smoke (Yuan 2008) . However, the assumption that smoke usually drifts upwards makes this model ineffective in a scenario where strong wind is present. Other research efforts have extracted motion features of smoke using optical flow (Kolesov et al. 2010; Yu et al. 2010) ; the results have been mixed. Recognizing the fact that the color of smoke is usually grayish, Chen et al (2006) extracted chromatic features of smoke according to a set of decision rules. In a similar vein Calderara et al. (2011) selected a reference color model in the RGB color space to represent smoke. Given the image of a scene, blurred edges could be observed in smokecovered areas and the consequent decrease in high frequency has been used as cue to perform smoke detection (Toreyin et al. 2005) . However, this decrease in high frequency is not unique to smoke covering as occlusion from other opaque objects could also lead to a decrease in high frequency. Local binary pattern (LBP) has been used successfully to capture texture features and was applied to smoke detection (Tian et al. 2011) . Recent progress in smoke detection research points at two emerging trends. First, increased attention is now being directed at characterizing the nature of smoke. For example, the fractal (Maruta et al. 2010 ) and transmission (Long et al. 2010) property of smoke have been employed to detect smoke. Second, techniques based on feature fusion are being reported. In a representative work (Tung and Kim 2011) , motion, surface roughness and area randomness information of smoke were all included in the feature vector.. It is important to note that the different types of features investigated in the literature are usually extracted directly from the original images captured by a stationary camera. A distinguishing point about our proposed approach is that the textural features of smoke are extracted from pure smoke separated from the background in an image sequence.

B I O O O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Local procrustes for manifold embedding: a measure of embedding quality and embedding algorithms We present the Procrustes measure, a novel measure based on Procrustes rotation that enables quantitative comparison of the output of manifold-based embedding algorithms such as LLE (Roweis and Saul, Science 290(5500), [2323][2324][2325][2326] 2000) and Isomap (Tenenbaum et al., Science 290 (5500), [2319][2320][2321][2322][2323] 2000). The measure also serves as a natural tool when choosing dimension-reduction parameters. We also present two novel dimensionreduction techniques that attempt to minimize the suggested measure, and compare the results of these techniques to the results of existing algorithms. Finally, we suggest a simple iterative method that can be used to improve the output of existing algorithms. In this section, we emphasize the main results of this work and indicate possible directions for future research.. We demonstrated that overall, the measure R(X, Y ) provides a good estimation of the quality of the embedding. It allows a quantitative comparison of the outputs of various embedding algorithms. Moreover, it is quickly and easily computed. However, two points should be noted.. First, R(X, Y ) measures only the local quality of the embedding. As emphasized in Fig. 3 , even outputs that do not preserve the global structure of the input may yield relatively low R-values if the local neighborhood structure is generally preserved. This problem is shared by all manifold-embedding techniques that try to minimize only local attributes of the data. One way to solve this problem is to have an additional measure that tests if far away input points where mapped close by. Such a measure was suggested by Venna and Kaski (2006) . Another research direction is to circumvent this problem by adding a penalty for outputs that embed distant geodesic points close to each other. Distant geodesic points Fig. 3 The input, an 800-point sample taken from a cylinder is presented in (A). (B)-(H) show the output of GP, PSA, LLE, Isomap, LTSA, SDE and MVE, respectively, for k = 12. Note that the cylinder has no embedding in R 2 and it is not clear what is the best embedding in this case. While PSA, Isomap, LLE, SDE and MVE succeeded in finding the topological ring structure of the cylinder, only PSA and LLE succeed in preserving the width of the cylinder. GP and LTSA collapse the cylinder and therefore fail to find the global structure, though they preform well for most of the neighborhoods (see Table 2 ) Fig. 4 The projection of the three-dimensional output, as computed by PSA, on the first two coordinates (small points). The input used was a 1965-point sample of grayscale images of faces (see Table 1 ). The boxes connected by lines are nearby points in the output set. The images are the corresponding face images from the input, in the same order. We see that nearby images in the input space correspond to nearby points in the output space can be defined, for example, as points at least s-distant on the neighborhood graph, with s > 1. The main problem with this idea is the difficulty in tuning the penalizing constant in a meaningful way (see discussion in Chen 2006, Sect. 2) .. Second, R(X, Y ) is not an ideal measure of the quality of embedding for algorithms that normalize their output, such as LLE (Roweis and Saul 2000) , Laplacian Eigenmap (Belkin and Niyogi 2003) , and LTSA (Zhang and Zha 2004) . This is because normalization of the output distorts the structure of the local neighborhoods and therefore yields high R-values even if the output seems to find the underlying structure of the input. This point (see also discussion in Sha and Saul 2005, Sect. 2) raises the questions: which qualities are preserved by these algorithms and how can one quantify these qualities? However, it is clear that these algorithms do not perform faithful embedding in the sense defined in Sect. 3. The measure R C (X, Y ) addresses this problem to some degree, by allowing separate scaling of each neighborhood (see Table 2 ). One could consider an even more relaxed measure which allows not only rotation, translations and scaling but a general linear transformation of each neighborhood. However, it is not clear what exactly such a measure would quantify. Fig. 5 The input consists of 300 images of the globe, each of 100 × 100 pixels. The images were obtained by changing the globe's azimuthal and elevation angles. The parameters of the variations are given by a 30 × 10 array that contains −45 to 45 degrees of azimuth and 0 to 30 degrees of elevation (see Hamm et al. 2005) . The upper left corner of the array is presented in (A). (B)-(H) show the output of GP, PSA, LLE, Isomap, LTSA, SDE, and MVE, respectively, for k = 12. Note that regardless of the input dimension, GP, PSA, Isomap, and SDE succeed to find the underlying structure of the data. For explanation regarding the behavior of LLE and LTSA in this example, see Goldberg et al. (2008) . The reason for the failure of MVE is not clear, however MVE succeeds for k = 6. The measure R C (X, Y ) also confirms this behavior, with values 0. 33, 0.78, 0.82, 0.84, 0.86 for k = 6, 9, 12, 15, 18, respectively Two new embedding algorithms were introduced. We discuss some aspects of these algorithms below.. PSA, in the form we suggested in this work, uses SA to align the tangent subspaces at all points. While PSA works reasonably well for small input sets and low output dimension spaces, it is not suitable for large data sets. However, the algorithm should not be rejected as a whole. Rather, a different or modified technique for subspaces alignment, for example the use of landmarks (de Silva and Tenenbaum 2003) , is required in order to make this algorithm truly useful.. GP is very fast (O(n) where n is the number of sample points), can work on very large input sets (even 100,000 in less than an hour), and obtains good results as shown both in Figs. 1-4 and in Table 2 . This algorithm is therefore an efficient alternative to the existing Fig. 7 The input, a noisy version of an 1600-point sample taken from a swissroll, is presented in (A). (B) shows the values of the Procrustes measure R(X, Y ) on the output of GP, for k = 12 and output dimensions d = 1, . . . , 10 (solid blue line). The lower bound is indicated in red dashed line. The elbow at d = 2 indicates that d = 2 is correct output dimension Fig. 8 The profile of noisy input of a 2500-point sample taken from a swissroll is presented in (A). (B)-(F) show the output of GP, PSA, LLE, Isomap and LTSA respectively. Note that only GP and PSA succeed to find the structure of the swissroll algorithms. It may also be used to choose optimal parameters, such as neighborhood size and output dimension, before other algorithms are applied. R(X, Y ) can be used for the comparison of GP outputs for varied parameters.. An important issue that was not considered in depth in this paper is that of noisy input data. The main problem with noisy data is that, locally, the data seems q-dimensional, even if the manifold is d-dimensional (d < q). To overcome this problem, one should choose neighborhoods that are large relative to the magnitude of the noise, but not too large with respect to the curvature of the manifold. If the neighborhood size is chosen wisely, both PSA and GP should overcome the noise and perform the embedding well (see Fig. 8 ). This is due to the fact that both algorithms are based on Procrustes analysis and PCA, which are relatively robust against noise. Further study is required to define a method for choosing the optimal neighborhood size.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O


Efficient Detection Algorithms for MIMO Communication Systems Abstract In this paper, two new efficient detection algorithms, Type 1 (T1) with better complexity-performance tradeoff and Type 2 (T2) with lower complexity, are derived from one generalized framework for multiple-input multipleoutput (MIMO) communication systems. The proposed generalized detection framework constructed by parallel interference cancellation (PIC), group, and iteration techniques provides three parameters and three sub-algorithms to generate two efficient detection algorithms and conventional BLAST-ordered decision feedback (BODF), grouped, iterative, and B-Chase detection algorithms. Since the group interference suppression (GIS) technique is applied to the proposed detection algorithms, the complexities of the preprocessing (PP) and tree search (TS) can be reduced. In (8,8) system with uncoded 16-QAM inputs, one example of the T1 algorithm can save complexity by 21.2% at the penalty of 0.6 dB loss compared with the B-Chase detector. The T2 algorithm not only reduces complexity by 21.9% but also outperforms the BODF algorithm by 3.1 dB. Multiple-input multiple-output (MIMO) technology can significantly improve data transmission rate in bandwidthlimited wireless communications without increasing the transmission power. Much research [1, 2] has shown that the channel capacity increases while the number of antennas is raised. Because of the above advantage, the MIMO technique has been considered in modern highspeed wireless communication standard including wireless LAN [3] and mobile wireless MAN. The Bell Laboratories layered space-time (BLAST) wireless communication system [1] uses multi-element antenna arrays at both the transmitter and receiver to achieve high spectral efficiency. This technology is referred to as the diagonal BLAST (D-BLAST). The D-BLAST theoretically approaches the Shannon capacity for multiple transmitters and receivers, but the D-BLAST is complex and impractical in hardware design. The vertical BLAST (V-BLAST) system [4, 5] is a simplified architecture of the D-BLAST, where the BLASTordered decision feedback (BODF) detection algorithm named in [6] (also called successive interference cancellation (SIC) detection algorithm named in [7] ) is applied. Although the BODF algorithm [8, 9] has low computational complexity, the bit-error rate (BER) performance is not satisfactory. In terms of BER performance in the MIMO detection system, the maximum likelihood (ML) detection scheme is an optimum solution to the receiver. However, it is manifest that the detection complexity significantly raises as the number of antennas and the constellation size increase. Thus, the ML scheme is not suitable for high-speed hardware implementation. The sphere decoding (SD) scheme [10, 11] searching for the closest lattice point inside the bounded radius of sphere achieves the same ML detection performance with efficient computational complexity. An efficient SD algorithm [12] improved the computational complexity by the projection scheme with sacrificing BER performance. However, the complexity of the SD algorithm is unstable owing to the variation of the iteration number which is sensitive to the signal-to-noise ratio (SNR). Especially, while the SD algorithm requires larger iteration number, the higher computational complexity at low SNR environment is incurred. Thus, the variable throughput of the SD algorithm affects the system performance.. Currently, there have been many studies [6, [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] on developing detection algorithms in both complexity and performance between the ML and BODF detection algorithms. The research work [13, 14] divides symbols into two groups. After group partition, the first-group symbols are detected by the ML detection and the secondgroup symbols are detected by a suboptimal algorithm after cancelling the interference from the first-group symbols. Although the previously published schemes [13, 14] using the ML and suboptimal detection algorithms can achieve better performance, the high computational complexity is incurred. The B-Chase detection algorithm [6] that lists more candidates for the parallel detection shows good performance and complexity trade-off for different demands. However, while increasing the number of antennas, the preprocessing (PP) complexity of the B-Chase algorithm largely increases because a full size channel matrix is processed without matrix decomposition. On the other hand, under the limited complexity, the B-Chase algorithm, in most cases, cannot choose better candidates from adjacent points for detection. Due to above reasons, we are motivated to propose two efficient detection algorithms with the group interference suppression (GIS) technique via one generalized detection framework constructed by the parallel interference cancellation (PIC), group, and iteration schemes. Compared with the B-Chase algorithm, the proposed Type 1 (T1) algorithm features the low computational complexity and satisfactory performance for large number of antennas. The proposed Type 2 (T2) algorithm shows lower complexity compared with the BODF and T1 algorithms.. This paper is organized as follows. Brief review of the MIMO detection algorithms is described in Section 2. In Section 3, one generalized detection framework has been presented. In the same section, how to generate existing detection algorithms through this framework will be discussed. In Section 4, we propose two new efficient detection algorithms via the generalized framework. The parallel characteristic comparison, complexity analysis and performance simulation results are presented in Section 5. Finally, the conclusions are remarked in the last section. Several acronyms used in this paper are listed in Appendix. In this paper, the proposed generalized framework is capable of generating six conventional and two new efficient MIMO detection algorithms. Due to the GIS technique, the T1 algorithm can choose better or fewer candidates with higher probability than other parallel detection algorithms under the limited complexity. On the other hand, the PP complexity of the T1 and T2 algorithms can be reduced since the lower dimensional sub-matrix is processed after applying the GIS technique. The T1 detection algorithm attains better complexity-performance tradeoff than the B-Chase detection algorithm with slightly sacrificing BER performance. For example, in (8, 8) system with 16-QAM inputs, at high performance end, T1(4,4,1) and T1(4,2,1) can reduce multiplication complexity by 21.2% and 26.6% at the penalty of 0.6 dB and 1 dB loss compared with the B-Chase(16) detection, respectively. Furthermore, the proposed T2 algorithm can attain lower complexity compared with the T1, S-GD, BODF, BChase algorithms. For example, in (8, 8) system with 16-QAM inputs, the T2 algorithm not only reduces complexity by 21.9% but also outperforms 3.1 dB compared with the BODF detection algorithm. . Pro. T2 Figure 14 Complexity-performance tradeoff of the proposed T1, proposed T2, S-GD, B-Chase and GPIC(1,0) algorithms with QPSK inputs and T=8. . T1(4,1,1)/B-Chase(12) Pro. T1(4,16,1)/GPIC(1,0) Pro. T1(4,4,1)/GPIC(1,0) Pro. T2/ Pro. T1(4,1,1) Pro. T2/S-GD Figure 16 Complexity ratio of the proposed T1, proposed T2, S-GD, B-Chase, and GPIC(1,0) algorithms in (8, 8) MIMO system with 16-QAM inputs.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Modeling user context with applications to media retrieval In this paper, we develop a theoretical understanding of multi-sensory knowledge and user context and their inter-relationships. This is used to develop a generic representation framework for multi-sensory knowledge and context. A representation framework for context can have a significant impact on media applications that dynamically adapt to user needs. There are three key contributions of this work: (a) theoretical analysis, (b) representation framework and (c) experimental validation. Knowledge is understood to be a dynamic set of multi-sensory facts with three key propertiesmulti-sensory, emergent and dynamic. Context is the dynamic subset of knowledge that affects the communication between entities. We develop a graph based, multi-relational representation framework for knowledge, and model its temporal dynamics using a linear dynamical system. Our approach results in a stable and convergent system. We applied our representation framework to a image retrieval system with a large collection of photographs from everyday events. Our experimental validation with the retrieval evaluated against two reference algorithms indicates that our context based approach provides significant gains in real-world usage scenarios. In this paper we develop a representation framework for multi-sensory knowledge and user context, and apply this framework in a media retrieval application. The development of a generic representation for context is an important problem in multimedia. Intuitively, context is critical in explaining the "why" in communication -i.e., it represents the set of concepts that fully explain the concept that is contained in the message. Thus context play an important role in the exchange of messages in any communication or interaction. In faceto-face human communication, contextual cues play a key role [13, 31] , allowing us to act and participate in conversations in a rich manner. Similarly, a flexible representation framework that additionally addresses temporal evolution of context, can lead to powerful media applications that gracefully adapt to user interaction.. In this paper, we focus on developing a theoretical understanding of context, and its relationship to multisensory knowledge. Then we develop a graph based representation of knowledge, and show how we can model context as a dynamic subset of knowledge. We apply this framework on a photo browsing application with a large database of photographs from ordinary everyday events. We now summarize the unique and novel contributions of this paper are as follows.. • Theoretical analysis: We present a detailed understanding of multi-sensory knowledge and context in the communication between two entities. We define knowledge as a dynamic set of multi-sensory facts. There are three key aspects of knowledge -(a) multi-sensory, (b) emergent and (c) dynamic. We define context as "finite and dynamic set of multi-sensory and inter-related conditions that influences the exchange of messages between two entities in communication." Context forms a dynamic subset of multi-sensory knowledge that is central to the semantics of the communication.. • Representation framework: We develop a graph based representation of multi-sensory knowledge. Each concept is a node in this graph, and a pair of nodes can have multiple relationships (feature based as well as semantic and each relationship represented as a weighted edge). Context is represented as the subset of nodes in attention that affect the communication. A key innovation is the development of a linear dynamical system to model the evolution of knowledge. We show that the system is stable and convergent.. • Experimental validation: We have applied our representational framework to a practical image retrieval application. The dataset comprised a large collection (∼4,000) of personal photographs of ordinary everyday events. This dataset is challenging as it is not organized into discrete categories, but is a better indicator of real-world results. We tested our algorithm of context aware retrieval against relevance feedback [27] and random browsing (baseline). Our user studies with graduate students indicate that our approach has significant gains.. It is important to note that the specific attributes of context will always be application dependent, driven by the real-world application needs. Our research instead focuses on a generic representation framework that allows for a systematic mechanism to model knowledge with arbitrary attributes and their temporal evolution. This allows our contributions to context and knowledge be applied to other applications.. The rest of this paper is organized as follows. In the next section we begin by developing our notion of context -we provide real-world examples and then define context, multi-sensory knowledge and their inter-relationships. In Sect. 3, we discuss related work. In Sect. 4, we develop the key contribution of this paper in terms of a context model for media retrieval. In Sect. 5, we present the architecture of our photo retrieval application. In Sect. 6, we present our experiments on a large real-world data set and in Sect. 7, we discuss the limitations of this work. Finally in Sect. 8, we present our conclusions and future work. In this paper, we presented a novel framework for modeling user context. We have developed a theoretical understanding of multi-sensory knowledge and user context and their inter-relationships. We presented our definition of context as "the finite and dynamic set of multi-sensory and inter-related conditions that influences the exchange of messages between two entities in communication." We stated properties of context and its relationship to knowledge and the application. We then presented a dynamic, emergent and multi-sensory knowledge representation framework in which context can be estimated and represented as a subset that is in focus. We modeled the dynamics of the context and knowledge as linear dynamical system. We provided proofs for the convergence, stability and controllability of the user context model under all circumstances. We tested the validity of our proposed user context model by applying it to a photo retrieval application. Our experiments demonstrate that the model of user context helps retrieve more relevant images for the user as compared to relevance feedback. The experiments also demonstrate the improvement in the relevance of the retrieved images with increasing user interaction.. We have identified that for a robust context model, we need a comprehensive framework for knowledge acquisition and representation. Hence, we plan to focus our future work mainly on improving the process of user knowledge acquisition, and multi-scale and multi-modal knowledge representation. We are also working on more exhaustive experiments with more users and a larger dataset and improved features to provide statistically more significant results. Early work on context [10] [11] [12] 28] treat context as pieces of information like location, identity, activity and time and has been successfully used for the purposes of configuration and adaptation in the areas of context aware ubiquitous computing. An attempt to integrate all such information was made in [9] . There the context was modeled for ubiquitous computing environments and hence was limited to the anticipated needs of the relevant ubiquitous computing applications. The context was defined as "any information that can be used to characterize the situation of an entity, where an entity is a person, place or object that is considered relevant to the interaction between a user and an application, including the user and application themselves." This definition is very broad as pointed out in [34] . In [34] , the author bounded the definition of context as the set of set of information that is relevant for the current communication. This definition is still broad and does not focus on two important properties of construct -(a) Context is a dynamic construct [14] and (b) Context is related to knowledge and cannot be discussed independent of it [26] . Our definition of context recognizes these important properties of context.. The generic model for context is not a well posed problem [17] . In [17] the authors pointed out the tradeoff between the desire for abstraction (to make modular and tractable systems) and the desire for contextsensitivity. This also points towards an irony in making a generic context model (making a generic context model is actually motivated by this desire for abstraction) which is actually a drift away from the context sensitivity. Thus the attributes of context can only be decided on a per application basis. In this paper we focus on models of query context, with a model easily adapted to other applications. The needs of context in a media retrieval application are concerned with semantic interrelationships between concepts, which can be arbitrary. Also the relationships that we wish to explore in multimedia are linguistic, statistical as well as common sense rules such as from the CyC database [2] . We address these issues in our work on context models.. The context model would thus consist of two structures:. • A knowledge base (personal or shared ontology) consisting of the concepts and the relationships between them • A temporally evolving context representation that is in relation to this ontology.. Most recent attempts on knowledge representation and context modeling fall into one of the two categories namely the logical representation [7, 21] and statistical representation [24, 33] . The logical representations are mostly driven by linguistic interpretations and the construction of meaning in language. On the other hand, the statistical knowledge representations are driven by pattern classification applications. For multimedia applications, we need a multi-modal knowledge representation that integrates different approaches in single modalities. In [5] , the authors proposed a multimedia knowledge framework involving arbitrary relationships between concepts. They suggested a learning framework in which the topology of the knowledge framework and the relationships between the concepts are statistically learned into a Bayesian network. This gives an ontology depicting purely arbitrary statistical relationships between the concepts but does not provide a meaningful interpretation and evolution of the knowledge framework. The other attempts to represent semantic knowledge include Wordnet [23] , Cyc [16] and OpenMind Commonsense [20] . These knowledge representations focus on one or more relationships such as lexical relationships and commonsensical relationships.. Based upon such knowledge bases, several attempts have been made to represent and use context. This has been used in web-base learning environments, e-commerce applications, media recommendation applications, intelligent agents and others. However, the knowledge frameworks do not evolve dynamically with the user interactions in the system, nor do these models consider the personalized and shared nature of the multi-modal knowledge of each individual.. A significant survey on the content-based image retrieval has been given in [30] . Prior works on adaptive media retrieval have used relevance feedback [8, 27] for a better query estimate and refinement of distance metrics in the low-level feature space. Recent work on retrieval has focused on context-aware similarities [35] , however the work focuses on learning the similarity metric using a kernel trick and the model of context is not proposed. These works on media retrieval consider that image relationships exist in low-level feature space. However, most of the times people associate images semantically and not using low-level features. Media retrieval that uses semantics for retrieval [3, 19] uses general and static knowledge bases like ConceptNet [20] . The problem with this approach is that it does not model the differences in the similarities between the media elements seen by different people in different contexts. We build a media retrieval system based upon the dynamic multimodal model of the user context as a subset of the personalized user knowledge. Our experiments show that media retrieval using the models of user context as in our approach produces better results as compared to relevance feedback as in [27] .. Some recent media retrieval approaches [15, 25] use a significant amount of prior knowledge for image retrieval. In [15] , the authors use a pure metadata approach for image retrieval. The description of the images is in from the metadata of the images. Hierarchically, structured ontologies, obtained from the domain and from the media annotation define a set of properties of the images such as time, place and others. The query is also represented in a similar structure. Creation of such ontologies and in particular the annotation of all the images in the database is a significant problem. Often the image database is sparsely annotated. Hence, such approaches should be merged with content-based approaches (). The search and recommendations are then performed by a set of logical rules (recommendation rules, hierarchy rules and mapping rules). In [25] the authors present an interesting task based approach to define the context of the user query and the context of the images and their annotations. The authors also have a dynamic notion of task context that captures the user's task history. However, the approach is driven by annotation and relies on continuous annotation by the users and hence several tools are provided for the purpose of annotation in the application. Both the approaches [15, 25] need significant amount of information from the user continuously. On the other hand, our approach makes limited demands from the user and grows the knowledge automatically by analyzing the user interactions. This is very crucial for keeping the users interested and at the same time provides opportunities for integration with incentive based strategies [29] for gathering information, for knowledge growth, from the user.

O B I O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I O O O O O O O O O O O O B I O O O O B I O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


On Counting Independent Sets in Sparse Graphs Abstract. We prove two results concerning approximate counting of independent sets in graphs with constant maximum degree ∆. The first implies that the Markov chain Monte Carlo technique is likely to fail if ∆ ≥ 6. The second shows that no fully polynomial randomized approximation scheme can exist for ∆ ≥ 25, unless RP = NP.Key words. Primary, 68Q17; Secondary, 05C69, 60J10, 68E10, 68Q25, 68W40 AMS subject classifications. approximation algorithms, computational complexity, independent sets, mixing times of Markov chains, randomized algorithms PII. S0097539701383844 1. Introduction. Counting independent sets in graphs is one of several combinatorial counting problems which have received recent attention. The problem is known to be #P-complete, even for low-degree graphs [5] . On the other hand, it has been shown that, for graphs of maximum degree ∆ = 4, randomized approximate counting is possible [9, 5] . This success has been achieved using the Markov chain Monte Carlo method [8] to construct a fully polynomial randomized approximation scheme (fpras). This has led to a natural question of how far this success might extend.Here we consider in more detail this question of counting independent sets in graphs with constant maximum degree. We prove two results. The first, in section 2, shows that the Monte Carlo Markov chain method is likely to fail for graphs with ∆ = 6. This leaves open only the case ∆ = 5.Our second result gives an explicit value of ∆ above which approximate counting, using any kind of polynomial-time algorithm, is impossible unless RP = NP. The bound we obtain is ∆ = 25, though we suspect that the true value is in single figures, probably 6.We note that Berman and Karpinski [2] have recently given new explicit bounds for the approximation ratio for the maximum independent set and other problems in low-degree graphs. These directly imply an inapproximability result for counting. (See Luby and Vigoda [9], specifically the proof of their Theorem 4.) However, the bound on ∆ obtained this way is larger than ours by at least two orders of magnitude.The questions we address in this article could also be studied in a more general setting in which vertices included in an independent set have weights or "fugacities" other than 1. In this setting, the weight of an independent set of size k is deemed to be λ k for some constant k, and the goal is to compute the sum of the weights of all independent sets. One could then ask, for each ∆, at what exact λ an fpras ceases to

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B B B B B B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Practical simulation application: Evaluation of process control parameters in Twisted-Pair Cables manufacturing system Intensive competition and rapid technology development in an increasingly global marketplace has left no room for competing manufacturers to harbour system inefficiencies. Moreover, customers are becoming more demanding and meeting their expectations represents an increasing challenge. TPC are used in various communication and networks hardware applications; their manufacturing facilities face many challenges through the many phases of their product life cycle including various product configurations with different equipment settings, different product flows and work in process (WIP) space limitations. The quest for internal efficiency and external effectiveness mandates that companies have to align their internal settings and resources with external requirements/orders, or in other words, significant factors must be identified prior to manufacturing process. Simulation, as one of the most flexible and powerful tools which provides a comprehensive understanding of manufacturing process variations, is increasingly aiding management and production team's decisions. An IDEF0 model in conjunction with a simulation model and a design of experiments (DOE) have been developed to characterize the Twisted-Pair Cable (TPC) production system and examine various production setting scenarios aiming to get the best product flow time. Never have the pressures on TPC manufacturers been more severe to deliver on time in the context of volatile demand from telecommunication suppliers. Moreover, global competition and technology advancement impose further complexity into the manufacturing processes of these products. TPCs have different characteristics including cable types, diameters, number of pairs and length. Each change in these characteristics influences product flow, equipment settings and product dispatching; hence creates complex interdependencies between manufacturing parameters. The manufacturing is performed in an environment of significant uncertainty both in dispatching rules used and process parameters.. Traditional analytical techniques and simple mathematical models are currently inadequate to analyze these complex manufacturing environments. Therefore, powerful modeling & simulation technique integrated with system analysis approaches (IDEF0, DOE) are needed to properly model the dynamics as well as variability of the system and then optimize the production variables.. Simulation has proved to be a useful tool in various manufacturing applications (A. Arisha et al. 2004 , Duilio Curcio et al. 2007 , Darrell W.Starks et al. 2006 , J.T. Lin et al. 2008 and W.Rocky Newman et al. 1999) , as it is used to analyze the alternative system configurations. Many articles have been published about simulation approach in production systems, however few if any addressed TPC manufacturing issues. Therefore, this study is directed to analyze the influence of the changes in dispatching rules and process parameters on system performance by conducting a simulation-based full factorial design. The large range of products in various configurations along with the objective of ontime delivery creates a challenging manufacturing environment for TPC. As process analysis is considered a critical task in such complex systems, it becomes necessary to choose effective approaches to model them.. IDEF0 has been used to develop the conceptual model applying standard formats to define the sequences, relationships and interdependences between TPC activities. It proves to be an efficient system description tool which offers a structured modeling approach. IDEF0 has enabled the system analysis phase by breaking the processes into stages which describe each manufacturing activity as well as envisage the overall system view.. Simulation modeling often provides a robust tool to evaluate the impact of different manufacturing policies and strategies with a minimum cost and risk. Hence, it has been employed to imitate the shop floor activities of TPC production. The significance of process parameters on system performance were analyzed using factorial design of experiments. Results show that in contrast to preventive maintenance policy, machine speed levels, machine operation rules and their interactions have significant effects on AFT regardless of dispatching rules applied.. This paper presents an integrated model of IDEF0, simulation modeling and simulation optimization (DOE) that supports decision making in TPC industry. The model has been verified and validated against real data and showed satisfactory results. The best combination of process control parameters (e.g. machine speed level, machine operation rules, and preventive maintenance policies) and the significance of their impact on system performance can easily be obtained as model outputs.

O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Dynamic memory balancing for virtual machines Virtualization essentially enables multiple operating systems and applications to run on one physical computer by multiplexing hardware resources. A key motivation for applying virtualization is to improve hardware resource utilization while maintaining reasonable quality of service. However, such a goal cannot be achieved without efficient resource management. Though most physical resources, such as processor cores and I/O devices, are shared among virtual machines using time slicing and can be scheduled flexibly based on priority, allocating an appropriate amount of main memory to virtual machines is more challenging. Different applications have different memory requirements. Even a single application shows varied working set sizes during its execution. An optimal memory management strategy under a virtualized environment thus needs to dynamically adjust memory allocation for each virtual machine, which further requires a prediction model that forecasts its host physical memory needs on the fly. This paper introduces MEmory Balancer (MEB) which dynamically monitors the memory usage of each virtual machine, accurately predicts its memory needs, and periodically reallocates host memory. MEB uses two effective memory predictors which, respectively, estimate the amount of memory available for reclaiming without a notable performance drop, and additional memory required for reducing the virtual machine paging penalty. Our experimental results show that our prediction schemes yield high accuracy and low overhead. Furthermore, the overall system throughput can be significantly improved with MEB. Recently, virtualization technologies, whose roots can be traced back to the mainframe days, are drawing renewed attention from a variety of application domains such as data centers, web hosting, or even desktop computing, by offering the benefits on security, power-saving, and resource-efficiency. Virtual machines (VMs) on top of the hypervisor or the virtual machine monitor (VMM) 2 , which multiplexes the hardware resources. The VMM has the ultimate control on all hardware resources while offering each guest OS an illusion of a raw machine by virtualizing the hardware. No matter if we use a virtualized system for security or hardware multiplexing, we usually boot several virtual machines on a single computer. Those machines eventually share or compete for the hardware resources.. In a typical virtualized system, resources, like processors and network interfaces, can be assigned to a virtual machine when needed and given up when there is no demand. The host memory allocation is mostly static-each virtual machine is assigned a fixed amount of host memory in the beginning. Although Xen and VMware provide a ballooning driver to dynamically adjust host memory allocation, existing studies are insufficient to tell when to reallocate and how much memory a virtual machine needs or is willing to give up to maintain the performance of the applications running on it (Barham et al. 2003; Waldspurger 2002) . This paper proposes MEmory Balancer (MEB) which dynamically monitors the memory usage of each virtual machine, accurately predicts its memory needs, and periodically reallocates host memory to achieve high performance.. The VMware ESX server (Waldspurger 2002 ) uses a sharebased allocation scheme. The share of a virtual machine is determined by its memory utilization and a maximum and minimum memory quota predefined by the system administrator. VMware uses a sampling method to determine memory utilization. The system periodically samples a random set of pages and the utilization is the ratio of the pages actively in use. A virtual machine with low memory utilization has a lower share and thus is more likely to get its memory reclaimed. One drawback of the sampling approach is that it can only predict the memory requirement of a virtual machine when there is free memory. Furthermore, it cannot tell the potential performance loss if more memory than what is currently idle is reclaimed.. Recent studies have used Mattson et al.'s LRU stack distancebased profiling (Mattson et al. 1970) to calculate page miss ratio of a process with respect to memory allocation (Zhou et al. 2004; Yang et al. 2006) . These studies either require tracking virtual addresses or need interaction between processes and the OS, and thus cannot be directly applied to virtual machines where we need to estimate memory needs of each virtual machine. In a virtualized environment, execution of a memory access instruction is usually handled by the hardware directly, bypassing the hypervisor unless it results in a page fault. On the other hand, page tables are considered as privileged data and all updates to page tables will be captured by the VMM, which provides us a chance to manipulate the page table entry for each page table update request. Inspired by previous process-level memory predictor, our predictor, which is located in VMM, tracks normal memory accesses by revoking user access permission and trapping them into VMM as minor page faults (Zhou et al. 2004) . Note that it is prohibitive to track virtual addresses for each process in a VM. We instead build an LRU histogram based on host physical addresses (or machine addresses). As a consequence, this estimator cannot provide the relationship between page miss and memory size beyond the size of machine memory allocation. When a VM's memory is under-allocated, the miss ratio curve is unable to predict how much extra memory is needed. MEB uses a simple predictor to monitor the swap space usage and uses it as a guide to increase the memory allocation for a virtual machine.. We have implemented MEB in Xen (Barham et al. 2003) . Experimental results show that it is able to automatically balance the memory load and significantly increase the performance of the VMs which suffers from insufficient memory allocation with a slight overhead to the initially over-allocated VMs.. The remainder of this paper is organized as follows. Section 2 presents the background and related work. Section 3 describes the dynamic balancing system in detail. Section 4 discusses experimental results, comparing with previous approaches, and Section 5 concludes. As virtualization technology gains more ground in both academics and industry, resource management in a virtualized system remains a key problem. Memory contention among virtual machines can significantly drop the overall system throughput. In this paper, we present MEB, a memory balancer that estimates the working set size of a VM and automatically adjusts the memory allocation to improve memory resource utilization. An evaluation of our Xenbased implementation shows that MEB is capable of boosting the overall performance through automatic memory balancing. In future work, we plan to extend MEB to support adaptive parameter selection to further reduce overhead while improving prediction accuracy. As discussed in Section 1, in (Waldspurger 2002) , a page sampling strategy is presented to infer the memory utilization and thus the memory needs of a VM. During a sampling interval, accesses to a set of random pages are monitored. By the end of the sampling period, the page utilization of the set is used as an approximation of global memory utilization. Reclaimed memory is reallocated to the VMs based on their utilization. Our experiments in Section 4 show that the accuracy of this sampling-based scheme is lower than our LRU histogram based estimation. And since the performance of a virtual machine is not usually proportional to the size of allocated memory, it is hard to evaluate the performance impact of various amount of memory allocation to a VM. In particular, it cannot give the performance impact if some actively used memory is reclaimed for usage in another virtual machine for a more important application. Therefore it may miss an opportunity to minimize systemwide page misses. Magenheimer (Magenheimer 2008) uses the operating system's own performance statistics to guide the memory resource management. However, the memory usage reported by most modern operating systems includes the infrequently used areas, which can be reclaimed without a notable performance penalty.. Zhou et. al (Zhou et al. 2004) propose the page miss ratio curve to dynamically track working set size. They suggest two approaches: hardware based and OS based. The former one tracks the miss ratio curve for the entire system by monitoring physical memory accesses. The latter one maintains a per-application miss ratio curve based on applications' virtual addresses. Unfortunately, neither of them fits in the domain of virtualization well:. 1. The hardware solution requires an extra circuit, which is unavailable on commodity processors.. histogram for every process, resulting in prohibitive space and management cost.. Nonetheless, the page miss ratio curve and memory access intercepting are extensively used in recent studies with extensions. CRAMM (Yang et al. 2006 ) is an LRU histogram based virtual memory manager, which maintains an LRU histogram for each process. A modified JVM communicates with CRAMM to acquire intelligence about its own working set size (WSS) and the OS's available memory, and adjusts the heap size to minimize the occurrence of garbage collection without incurring an unacceptable number of major page faults. CRAMM builds a model that correlates the WSS of a Java application and its heap size. Hence, by detecting WSS changes, the heap size can be adjusted accordingly. However, there is no such correlation between a virtual machine's WSS and its allocated memory size. In addition, CRAMM requires modifications to the OS, which we manage to avoid.. Geiger (Jones et al. 2006) , on the other hand, detects memory pressure and calculates the amount of extra memory needed by monitoring disk I/O and inferring major page faults. However, when a guest is over-allocated, it is unable to tell how to shrink the memory allocation, which prevents the resource manager from reclaiming the idle memory for use in other guests. The Black-box and Gray-box strategies (Wood et al.) migrate hot-spot guests to the physical machines with enough resources. Its memory monitoring function is a simplified version of Geiger and therefore inherits its drawbacks.. Hypervisor exclusive cache (Lu and Shen 2007) is most relevant to our research. The VMM uses an LRU based miss ratio curve to estimate WSS for each virtual machine. In this design, each VM gets a small amount of machine memory, called direct memory, and the rest of the memory is managed by the VMM in the form of exclusive cache. By allocating a different amount of cache to a VM, it achieves the same effect as changing its memory allocation. Since this approach can track all memory accesses above the minimum memory allocation, both WSS increases and decreases can be deduced, although the maximum increment is still bounded by the cache size. This design introduces an additional layer of memory management. Moreover, since the cache is exclusive, system states spread across a VM's direct memory and the VMM, which breaks the semantics of the VMM. It is therefore hard to migrate VMs. Our solution uses ballooning which keeps the VMs migrate-able. The hypervisor exclusive cache requires modification to the guest OS to notify the VMM of page eviction or release. Our approach only needs to trace page table changes and the whole balancing is transparent to guest OSes.

O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O