Bayesian characterization of uncertainty in intra-subject non-rigid registration In settings where high-level inferences are made based on registered image data, the registration uncertainty can contain important information. In this article, we propose a Bayesian non-rigid registration framework where conventional dissimilarity and regularization energies can be included in the likelihood and the prior distribution on deformations respectively through the use of Boltzmann's distribution. The posterior distribution is characterized using Markov Chain Monte Carlo (MCMC) methods with the effect of the Boltzmann temperature hyper-parameters marginalized under broad uninformative hyper-prior distributions. The MCMC chain permits estimation of the most likely deformation as well as the associated uncertainty. On synthetic examples, we demonstrate the ability of the method to identify the maximum a posteriori estimate and the associated posterior uncertainty, and demonstrate that the posterior distribution can be non-Gaussian. Additionally, results from registering clinical data acquired during neurosurgery for resection of brain tumor are provided; we compare the method to single transformation results from a deterministic optimizer and introduce methods that summarize the high-dimensional uncertainty. At the site of resection, the registration uncertainty increases and the marginal distribution on deformations is shown to be multi-modal. The vast majority of non-rigid registration methods only report a single transformation (i.e. a point-estimate) result without any estimate of the registration uncertainty, which could provide valuable information whenever important clinical decisions, or high-level analysis, are based on registered data. Various factors such as the ill-posed nature of non-rigid registration, the stochastic nature of the images, the large variability of anatomy, the presence of homogeneous intensity regions, and imaging artifacts like distortion and biasfield, may negatively affect the registration uncertainty. Despite this, state-of-the-art registration methods are typically evaluated on a limited set of representative images through estimation of the alignment error of sparse homologous landmarks, or regions, based a single registration result. Suppose a new dataset, with new pathology or image artifacts that are not represented in the evaluation dataset, was to be registered -then it cannot be assumed that the resulting deformation point-estimate will carry the same accuracy as the previous evaluation results would indicate. If the registration result was associated with some uncertainty measure, or confidence limits, it could be possible to detect that the method had difficulty in finding a proper alignment of the images in certain locations, and the surgical risk involved with making decisions based on the registered data could be evaluated. It should be noted that a highly uncertain (low precision) registration result does not necessarily mean that the result is also inaccurate and vice versa, so systematic errors (bias) of the registration model should be evaluated before placing trust in precise registration results.. In image-guided neurosurgery, where surgeons assess the operative risk based on registered image data, misplaced confidence in registration results may have severe consequences. Important functional areas of the brain are commonly defined from pre-operative studies. If these eloquent areas are located adjacent to a tumor, it is critical that their locations are mapped as precisely as possible to the intra-operative scans. Factors such as resection of tissue, degraded intra-operative image quality, or blood and edema around a tumor, may all contribute to an increase in registration uncertainty and/or decrease of registration accuracy, especially in the vicinity of the surgical field, precisely where accuracy is most essential. For a neurosurgeon, whose goal is to maximize the resection of tumor tissue while preserving eloquent areas, information of the uncertainty in the location of the functional areas in relation to the tumor is just as important as the optimal estimate. Current image guided navigation systems use rigid registration to map between the pre-and intra-operative space (Shamir et al., 2009; Gumprecht et al., 1999) , and researchers have focused on estimating the distribution of errors in rigid registration, especially in fiducialbased registration where a set of homologous points are used to drive the rigid transform that aligns the two spaces (Maurer et al., 1997) . It is generally known that the Fiducial Registration Error (FRE) is a poor predictor of real registration error (Danilchenko and Fitzpatrick, 2011) . Hence, recent works have focused on characterizing the distribution of the Target Registration Error (TRE), i.e. the distance between homologous points other than the fiducials used in the FRE, after registration (Danilchenko and Fitzpatrick, 2011; Seginer, 2011) . An extension to the standard rigid fiducial registration algorithm was presented by Pennec and Thirion (1997) where, given a set of matched geometric features in the form of points or frames, they estimate a rigid motion as well as the covariance matrix which is compatible with the matched points. Another well known method, the Iterative Closest Point (ICP) algorithm, which does not assume point homology, has been extended to include point uncertainties (Maier-Hein et al., 2012; Fieten et al., 2010) .. In rigid intensity-based registration, Kybic (2008) assumed normally distributed registration (translation) parameters and used the Hessian of the similarity criterion to compute confidence intervals of the registration parameters. Robinson and Milanfar (2004) used the Cramèr-Rao bounds to find a lower bound on the covariance of the parameters in a 2Dtranslational registration. Bansal et al. (2009) introduced a framework for quantifying errors in registration by computing the confidence intervals of the estimated parameters (translation, rotation and scale) for the similarity transform. They show that the parameters are multi-variate normally distributed and use the covariance matrix to compute the confidence interval of the transformation parameters.. Rigid registration cannot account for the general motion of brain tissue that occurs during neurosurgery which has spurred interest in non-rigid registration methods. However, few authors have attempted to quantify the full uncertainty of the estimates produced by nonrigid registration methods. Hub et al. (2009) developed a heuristic method for quantifying uncertainty in B-spline image registration by perturbing the B-spline control points and analyzing the effect on the similarity criterion. A statistical approach was presented by Kybic (2010) who considered images to be random processes and proposed a bootstrapping method to compute statistics on registration parameters. However, their method was only demonstrated on 2D datasets with low-dimensional transformation models. Gee et al. (1994 Gee et al. ( , 1995a described a Bayesian approach using a prior based on linear elasticity to regularize the problem. By using local quadratic approximations of the likelihood term, they constructed a Gibbs sampler to estimate the posterior mean (Gee et al., 1995a) , while a deterministic method was used to estimate the posterior mode (Gee et al., 1994) . They reported results of estimating transformation parameter variance for the case of 2D registrations. Another fast Bayesian approach for approximating the posterior uncertainty on registration parameters using variational Bayes was introduced by Simpson et al. (2012) where an approximate posterior distribution on the registration and hyper-parameters is estimated based on a set of simple parametric distributions and has shown to be useful in classification (Simpson et al., 2011a) and segmentation propagation tasks (Simpson et al., 2011b) .. In Bayesian inference, typical approaches to finding the maximum a posteriori (MAP) estimate of non-parametric distributions involve some form of local gradient based maximization of the log posterior. Unfortunately, there are usually no guarantees that the optimization method finds the global maximum, nor are "error-bars" or credible intervals of the MAP results usually provided. Because the posterior distribution is non-parametric, and assumed to have heavy tails and possibly be multi-modal, Markov Chain Monte Carlo (MCMC) is the most principled approach to infer posterior estimates, including the uncertainty (Gelman et al., 2003) . MCMC numerically approximates the posterior distribution by a set of posterior samples. While it asymptotically characterizes the entire posterior distribution, a large number of samples may be needed to characterize the posterior to the needed accuracy, and MCMC methods are consequently compromised by a high computational burden. As a result, few applications of MCMC have been reported in medical image analysis. Recently, Smal et al. (2012) applied MCMC for motion analysis of tagged magnetic resonance images. For segmentation purposes, Fan et al. (2007) used an MCMC method for sampling curves from the exponentiated negative segmentation energy and showed how to summarize posterior marginal distributions in terms of marginal probability maps. An extension of that work was presented in Chang and Fisher (2011) where they sample level sets instead of curves, which facilitates topology changes. MCMC has been widely used in functional Magnetic Resonance Imaging (fMRI) analysis (Woolrich et al., 2006; de Pasquale et al., 2008) and Diffusion Tensor Imaging (DTI) (Behrens et al., 2003; McNab et al., 2009 ).. The posterior distribution over deformations is a highly informative high-dimensional spatially varying object with local correlations. In this article, we introduce a Bayesian nonrigid registration framework where the posterior distribution on deformation parameters is characterized by MCMC. Although posterior approximations through, for instance, variational Bayes may provide fast estimates of the posterior, it is not known how well they approximate the unknown shape of the posterior distribution. Even though MCMC comes at a hefty computational price, and consequently may not be directly applicable in many clinical applications today, the insights gained into the complexities of the posterior distribution can be used to guide construction of new and fast posterior approximations and help point the way for designing future clinical systems that puts clinically relevant error-bars on registration results to reduce surgical risk. The framework allows for any traditional dissimilarity and regularization energy to be included in the likelihood and prior distribution respectively using Boltzmann's distribution. We note that other authors have derived prior probability distributions on deformations from regularization energies using Gibbs distribution (e.g. Gee and Bajcsy (1999) and Papademetris et al. (2001) ) which is closely related to Boltzmann's distribution.. We demonstrate the method with a Sum of Squared Differences (SSD) based likelihood, where the Boltzmann temperature models the image noise variance. The prior model over deformations is based on a linear elastic bio-mechanical model which, together with Boltzmann's distribution, implements a phenomenological approach to regularization where the Boltzmann temperature modulates the inverse stiffness of the tissue material. Higher temperatures (lower stiffness) facilitate more radical changes to the tissue configuration, while lower temperatures (higher stiffness) prevent large non-smooth deformations. Consequently, one major concern is the effect of model hyper-parameters, in our case the mechanical properties of the tissue and the variance of the image noise, on the registration results. The hyper-parameters are typically specified manually on an ad-hoc basis because it is difficult to find appropriate point-estimates for them. In the context of prostate registration , we have shown that the posterior distribution is highly sensitive to the particular value of these hyper-parameters. If training data is available, e.g. in the form of homologous structures or landmarks, the hyper-parameters can be estimated. Examples of this in registration are e.g. in a classic machine learning framework for model selection through optimization of the cross-validation error (Yeo et al., 2010) , or by maximumlikelihood estimation . However, if no such data is available, the standard way of treating the uncertainty in hyper-parameter values in a Bayesian framework is by equipping them with uninformative hyper-prior distributions and marginalizing them out from the final registration estimates. Another advantage of marginalization of broad hyper-parameters is that the likelihood and prior can, when appropriate, become heavy tailed. Assuming they were modeled with Gaussian distributions, then the marginalized distribution can be viewed as a mixture of Gaussian distributions with varying variance, and such distributions are well equipped for handling outliers. Working with the posterior distribution under marginalized hyper-parameters is complex and computationally expensive and may necessitate fast approximations such as variational Bayes (Simpson et al., 2012) . By using a system of sequential local Laplace approximations, we show how to marginalize model hyper-parameters representing image noise and tissue stiffness over broad uninformative log-normal hyper-priors .. While a lot of research has gone into finding ways of reducing high-dimensional uncertainty information into simple local low-dimensional objects that can be easily visualized (Spiegelhalter et al., 2011) , limited work has been reported on posterior uncertainty visualization in medical applications; including visualization of uncertainty in image guided needle insertions (Simpson et al., 2006) , uncertainty in radiation dose control in radiation oncology (McCormick et al., 2004) , determining uncertainty in white matter fiber orientation of diffusion tensors (Jones, 2003) and visualization of the local registration variance characterized by variational Bayes (Simpson et al., 2011a) or Gibbs sampling (Gee et al., 1995a) . Gee and Bajcsy (1999) also showed how to derive Bayesian credible intervals about deformed point estimates from the posterior variance. While the main focus of this paper is non-parametric estimation of the posterior distribution over deformations, we also show techniques to extract simple and robust low-dimensional posterior summaries, such as the joint mode and the registration uncertainty, and show the importance of such summaries in the realm of neurosurgery. We presented a Bayesian non-rigid registration framework where the posterior distribution is characterized through MCMC sampling. In contrast to conventional registration approaches that only report posterior modes, the posterior distribution, represented by a set of MCMC samples, contains other valuable information such as the registration uncertainty; this is important information when higher level inferences are drawn based on the data being registered. energy of a deformation, which we use as a regularizing term to penalize physically implausible solutions, takes the form E el (u) = ∫ x∈Ω f u(x) ⊤ K x u(x)dx, where K x is the stiffness matrix that is directly proportional to the Young's modulus. Under the linear element model, this is equivalent to (A.3) where K i,j is non-zero only if vertices v i and v j share a tetrahedral element.. Consequently, both the SSD and elastic energy contributions of one vertex v i under displacement u i depends only on the displacements of vertices v j that share tetrahedral elements with v i . Therefore, the energy term exhibits Markov structure and the Markov blanket for vertex v i is = ∪ e \v i such that v i ∈ .. Even though linear elasticity puts a penalty on non-smooth deformations, it does not prevent unwanted folding of tissue/elements. Folding of elements can be prevented by restricting the determinant of the Jacobian over the deformed element to be positive |J(u e (Ω e )| > 0. It has been our goal in this investigation to characterize the posterior distribution in some intra-subject non-rigid registration problems. Our longer range goal is to provide uncertainty information to clinical end-users, because we feel that it should be factored into decision making or treatment optimization. For example, communicating the distributional information in Fig. 13(c) gives the hypothetical end-user the option to discount the utility of the registration information if the uncertainty in the result is too large; this opportunity is not available if the MAP result, i.e. Fig. 13(b) , is the only information about the registration result that is provided.. A Bayesian non-rigid registration framework, that characterizes the posterior distribution on deformations by MCMC, was presented. The likelihood and prior distribution on deformations is constructed by converting dissimilarity and regularization energies into probability density functions by way of Boltzmann's distribution. Proper treatment of hyperparameters in registration, e.g. noise precision and the tissue parameters/regularization weight, is an important but often neglected problem. Non-rigid registration is a highly under-determined problem and therefore generally needs a prior (regularizer) to reduce the solution space. The prior introduces a bias, but with a large reduction in uncertainty. Decreasing the importance of the prior, e.g. by setting the prior temperature to a large value, effectively makes the regularization negligible which increases the posterior uncertainty. Even though registration results are sensitive to these parameters, they are in current practice often set on an ad-hoc basis. One approach to determine hyper-parameters that are highly informative for a specific clinical application is to learn them from a collection of representative clinical datasets where homologous landmarks or structures are identified . However, in cases where such data is not available, and little information is available regarding the optimal value of these parameters, the recommended Bayesian approach to handle them is to equip them with uninformative hyper-priors and marginalize them out. We formulated the posterior distribution over deformations, with the log-normal hyper-parameters marginalized over, in a variational free-energy framework where the variational density is the Laplace approximation of the hyper-parameter posterior. A log-normal hyper-distribution was chosen because it provides very broad hyper-priors (with variation over orders of magnitude), reflecting the uncertainty in the exact values of the hyper-parameters. The hyper-prior ensures that while tissue-stiffness and image-noise variance parameters are allowed to vary over a very wide range of values, they are kept physically sensible by disallowing fantastically huge or small values. However, ideally, they should be centered around some sensible values derived for instance through hyperparameter optimization using homologous landmarks/structures. The posterior estimation of the prior temperature can be seen as an estimate of the homogeneous inverse stiffness of the underlying tissue. In the future we plan to extend the current methodology to marginalize/ estimate compartment-or element-wise tissue stiffness.. Our experiments have shown that marginal posterior distributions on deformations may be multi-modal -which does not imply that the complete posterior distribution is necessarily multimodal. However, because we have shown that the marginal distributions are non-Gaussian, and possibly multi-modal, we can claim that the posterior distribution is non-Gaussian. While it may be surprising to some that the marginal distributions can have multiple modes, our experience in image registration has been that registration systems can become trapped in local extrema in the face of truncation or resection. Likelihood functions are complicated and highly non-convex functions of the transformation parameters; when there is structural disagreement among images, the likelihood functions frequently exhibit multiple modes that reflect spurious matches of image structures. Because of this, multiple modes in posterior marginals, or even the complete posterior distribution, are a possibility. Furthermore, increasing the degrees of freedom of the warp, which is similar to reducing the smoothing in the objective function, seems unlikely to reduce the occurrence of multiple modes in the posterior marginal; frequently it happens that increasing the degrees of freedom leads to more extrema in optimization problems.. While the proposed MCMC method is a principled approach to Bayesian inference, and provides a numerical approximation to the exact posterior distribution, it does come with high computational cost which is not unreasonable in an exploratory investigation. Although the computational cost may currently be too high for use in neurosurgery, it is straightforward to apply the method to new applications; e.g. to monitor delivered dose in adaptive radiotherapy where each fraction is delivered with approximately 24 hour intervals. For other applications, the proposed methodology can provide valuable insights into complex posterior distributions to help guide the design of faster methods that find simple robust and accurate approximations to the complex posterior distribution. One example is the variational Bayes approach taken by Simpson et al. (2012) which provides a fast exact analytical solution to a parametric approximation to the posterior distribution that facilitates high-dimensional registration. However, the effect of the approximation on the estimated posterior statistics is unknown and may be significant because of the possibly multi-modal and non-Gaussian posterior distribution. Parametric approximations of the posterior should therefore be compared against the asymptotically correct posterior distribution that can be generated by MCMC.. Another MCMC approach to registration, the Gibbs sampler proposed by (Gee et al., 1995b,a) , requires approximations of the likelihood function. By using MH, such likelihood approximations are not required and the proposed method is consequently more general in terms of the dissimilarity energies that can readily be used. On the other hand, Gibbs sampling should in theory converge quicker on the posterior distribution because every sample is accepted. The proposed marginalization over hyper-parameters using local Laplace approximations can readily be incorporated with the Gibbs sampler proposed by Gee et al. (1995b,a) .. The sampling approach we have used here is computationally intensive. Consequently, for the experiments in this article, we have used meshes with approximately 1000-2500 degrees of freedom, which corresponds to a vertex spacing of approximately 15mm in the clinical case; this is comparable to control point spacing typically used in intra-subject registration (Rueckert et al., 1999) . Our vertex spacing is larger than the 5mm B-spline spacing that is commonly used in inter-subject registration (Klein et al., 2009) , and it is possible that with higher resolution meshes and data, the estimated posterior uncertainty would decrease. With higher degrees of freedom comes not only higher computational cost, but also higher storage requirements because of the need to store enough samples to characterize the posterior. It is common practice to thin the MH sequence by a factor of at least ten to reduce intra-chain correlation and to discard the first half of the sample chain to remove burn-in effects. Consequently, if 10 7 samples are required to characterize a posterior distribution with 100k degrees of freedom, the storage requirements will be approximately 200Gb after thinning and burn-in. This does not pose a significant problem in practice, especially if advanced storage formats such as HDF5 (The HDF Group, 2000 -2010 , which can sequentially compress large files, are used.. Because of the high computational complexity of MCMC, some simplifying model assumptions were made to achieve tolerable running times. First, the likelihood assumes that the intensity difference between registered voxels differ by i.i.d. Gaussian noise. Theoretically, this is not a valid assumption, especially when Gaussian blurring is applied, but has been shown to work well in practice. In our work, where light blurring was used, we have expanded on the Gaussian noise model by marginalizing over the noise precision under a log-normal hyper-prior, which effectively can convert the Gaussian noise distribution into a distribution with heavy tails which is better suited for handling outliers, e.g. areas of resection. A well founded approach for handling the noise correlation caused by smoothing of the images is to include a constant Virtual Decimation (VD) factor in the likelihood term (Simpson et al., 2012; Groves et al., 2011) , which is similar to decimating the data. This is especially important if the main goal is to infer correct noise and regularization (stiffness) parameters. However, in our case, where the main goal is to characterize the posterior distribution on deformations, the constant VD factor will be absorbed into the marginalized noise temperature and will not have an affect in practice, aside from changing the scale of the temperature parameter. Although the SSD-based likelihood does not currently accommodate for functional or statistical intensity differences, such complex relationships can easily be included in the model, though likely with some computational cost.. Secondly, the FE-based transformation model was a natural choice given the use of the phenomenologically inspired linear elastic prior, but other popular transformation models such as B-splines (Rueckert et al., 1999) can also be incorporated.. Thirdly, linear elasticity has been widely used as a prior in non-rigid registration, but more complex prior models such as visco-and poro-elasticity (Kyriacou et al., 2002) may be more appropriate. Choosing a suitable prior that is a good model for the underlying tissue under deformation may improve the registration results and reduce registration uncertainty. If a segmentation of the underlying tissue is available, and relative stiffness between tissue types is known, a mechanical model that reflects this can easily be constructed and spatially varying (pseudo-)stiffness parameters can easily be encoded in the model without any modifications to the proposed posterior sampling scheme. It is also possible to use a random variable for each tissue type, and equip it with a hyper-prior and marginalize over it, but this requires changing the Laplace approximation accordingly, or using MH of the per tissue type hyper-parameters.. We have used common modeling assumptions, and some approximations, and our goal is to understand the implications of those assumptions on the uncertainty in the posterior distribution (or in deterministic terms, the landscape of the objective function). In the future, it is likely that developments in likelihood modeling, prior modeling, FEM technology, inference algorithms, and application specific development and engineering will lead to faster methods that could have tighter posterior distributions and associated marginals than those that are illustrated in Fig. 13(c) . Our experiments indicate that, depending on the application, it may be appropriate to consider the implications of non-Gaussian, and potentially multi-modal, posterior distributions in the design of the system. Approaches that should be investigated to speed up chain convergence, reduce inter-sample correlations and the random walk behavior of MH include Hybrid Monte Carlo (Duane et al., 1987) , which uses gradients of the log posterior distribution to take larger proposal steps while keeping the rejection rate low, and adaptive MCMC where the proposal distribution is dynamically fitted to match the posterior based on the accepted samples (Andrieu and Thoms, 2008) . One approach for learning the best compromise between likelihood model, prior model, mesh size, accuracy and uncertainty is through model selection using e.g. the Deviance Information Criterion (DIC) which is a suitable model selection criterion when the posterior is obtained by MCMC (Spiegelhalter et al., 2002) .. In future work we will adapt the framework to registration of MRI and ultrasound where we suspect the posterior distribution will be multi-modal because of the high noise level and lack of contrast in ultrasound images. We also suspect that exploiting the high-dimensional information available in the posterior distribution on deformations when doing high-level inferences based on the registered data will open up new and exciting research opportunities. Simpson et al. (2011a) showed that utilizing the registration uncertainty in classification of Alzheimer patients improved their classification results. In adaptive radiotherapy, monitoring the radiation dose delivered to critical tissue is important due to changes in anatomy over the treatment period. In Risholm et al. (2011) we showed that the uncertainty in registration can induce significant uncertainty in the estimated cumulative dose delivered to critical tissue during head and neck radiotherapy. The estimated delivered dose and the associated uncertainty was visualized in terms of dose volume histograms. Finding lowdimensional representations of the posterior distribution that are suitable for visualization is a difficult task and an active research topic (Spiegelhalter et al., 2011) . In this paper we used simple posterior summaries to convey uncertainty. However, one can envision that more complex approaches to interacting and extracting clinical relevant summaries from the posterior distribution can be designed in the future -for instance by exploiting the local correlations in the posterior. The proposed method for characterizing the posterior distribution over deformations was evaluated on both synthetic and clinical image data. In Section 3.1 the method is evaluated on a synthetic dataset with a non-convex likelihood and known ground-truth deformation, while Section 3.2 demonstrates the feasibility of registering clinical Magnetic Resonance (MR) images acquired during neurosurgery for tumor resection. All clinical data was used with informed consent under an institutional review board approved protocol.. To evaluate the MAP mode estimated with the mean-shift method described in Section 2.3.1, the posterior mode of Eqn.. (2) was also estimated with a global non-linear optimization procedure implemented in C++ and NLopt (Johnson, 2012) . The global stochastic optimizer, a Multi-Level Single-Linkage (MLSL) algorithm (Rinnooy Kan and Timmer, 1987) , was coupled with a local gradient-based Method of Moving Asymptotes (MMA) (Svanberg, 2002) , and non-linear constraints to prevent flipping of tetrahedra (positive deformation Jacobian over the element) were included through an augmented Lagrangian method which adds a penalty to the objective function for any violated constraints. After characterizing the posterior distribution with Algorithm 1 on a given dataset, the deterministic optimizer was run with the temperatures fixed to the mean of the temperatures marginalized through Laplace approximations.. All algorithms were implemented in C++ with extensive use of OpenMP ® for parallelization whenever feasible, and the experiments were run on Linux machines with 2 × 6-core Intel Xeon ™ 2.67GHz CPUs, 12GB DDR-3 RAM.. For all experiments, the material parameters of the linear elastic regularizer were initialized with a Young's modulus of E = 1.0 and Poisson's ratio of ν = 0.49. Because the elastic model is used as a regularizer, as opposed to a pure mechanical description of the deformation determined from boundary conditions and external forces, the Young's modulus is unitless and only relative stiffness can be prescribed. Every drawn proposal sample that caused folding of the FE-mesh was rejected. Consider two images m : Ω m → ℝ and f : Ω f → ℝ, where we want to align the moving image m(x), x ∈ Ω m , with a fixed image f (x), x ∈ Ω f , where Ω f , Ω m ⊂ ℝ d are the domains of the d-dimensional fixed and moving images respectively. The usual aim of non-rigid registration (Crum et al., 2004) is to find a high-dimensional displacement field u(x) : x ∈ Ω f ↦ u(x) + x ∈ Ω m that minimizes an energy function composed of an image dissimilarity measure E s (u; f, m) and a regularization term E r (u) on the displacements. Although this framework does not depend on the specific form of the dissimilarity energy, regularization energy or transformation model, for clarity and computational tractability, we restrict discussion in this paper to the Sum of Squared Differences (SSD) dissimilarity measure, a linear elastic regularization model and a Finite Element (FE) based transformation model. (Zienkiewicz and Taylor, 2000; Bro-nielsen, 1998 ) is a popular method for solving bio-mechanical problems and has shown great potential in modeling tissue deformations (Miller et al., 2010) . Here, the non-rigid transformation is effected through deformations of the FE mesh, which implements the bio-mechanical model based on tissue stiffness. Furthermore, it provides machinery to rapidly compute the energy functionals E s and E r as well as interpolating the displacement u(x) at inter-vertex positions. The FE method is described further in Appendix A.

B O O B O O B B O O O O O O O O O O O O O B B O O O O O O O O O O B B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B B O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O B O O O O O O B O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O B B B O O O O O O O O B O O O O O B O O O O O O O O O O O O O O B O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B B B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A cortical framework for invariant object categorization and recognition Abstract In this paper we present a new model for invariant object categorization and recognition. It is based on explicit multi-scale features: lines, edges and keypoints are extracted from responses of simple, complex and endstopped cells in cortical area V1, and keypoints are used to construct saliency maps for Focus-of-Attention. The model is a functional but dichotomous one, because keypoints are employed to model the ''where'' data stream, with dynamic routing of features from V1 to higher areas to obtain translation, rotation and size invariance, whereas lines and edges are employed in the ''what'' stream for object categorization and recognition. Furthermore, both the ''where'' and ''what'' pathways are dynamic in that information at coarse scales is employed first, after which information at progressively finer scales is added in order to refine the processes, i.e., both the dynamic feature routing and the categorization level. The construction of group and object templates, which are thought to be available in the prefrontal cortex with ''what'' and ''where'' components in PF46d and PF46v, is also illustrated. The model was tested in the framework of an integrated and biologically plausible architecture. Object detection, segregation, categorization and recognition are linked processes which cannot be completely sequential; they must be done in parallel, at least partially, and therefore they are overlapping; Rensink (2000) . These processes are achieved in the ventral ''what'' and dorsal ''where'' pathways, Deco and Rolls (2004) , with bottom-up feature extractions in areas V1, V2, V4, and IT 1 (what) in parallel with top-down attention from PP via MT to V2 and V1 (where). The latter is steered by possible object templates in memory, i.e., in prefrontal cortex with a ''what'' component in PF46v and a ''where'' component in PF46d. The Deco and Rolls model can explain invariance and attention besides the facts that cells at higher cortical areas have bigger receptive fields and that they are coding more complex patterns. However, their model is based on responses of simple cells in V1, whereas we are aiming at functional feature extractions in V1 and beyond. Although many image and object features are represented implicitly by simple cells, we apply explicit feature extractions: multi-scale line, edge and keypoint representations on the basis of cortical simple, complex and end-stopped cells; du Buf (2006, 2008) . The ultimate goal is to integrate feature extractions into a cortical architecture.. We are studying three related problems: when, where and how does categorization take place. The ''when'' problem allows for two hypotheses. The easy one is to assume that categorization occurs after recognition; Riesenhuber and Poggio (2000) : if specific neurons respond in the case of recognizing dog-1, dog-2, and dog-3, a grouping cell can combine all responses: a dog. This view is too simplistic, because the system must collect evidence for a specific object or object group in order to select possible templates in memory. For example, when we glance a portrait made by Arcimbaldo, the famous, sixteenth-century Italian painter, our first reaction is ''a face!'', but then follows ''fruits?'' and finally ''the cheek is an apple!'' When categorization occurs before recognition, GrillSpector and Kanwisher (2005) , the ''where'' problem is, at least partly, solved: it must take place at a very high level, with access to object templates in memory, and just before recognition. In fact, recognition can be seen as a last categorization step. Therefore, the ''how'' problem can be solved by taking into account feature extractions in V1 and beyond and the propagation of features to higher cortical areas. During the past years, we concentrated on the extraction of low-level primitives: lines, edges and keypoints, all multi-scale, see, e.g., Rodrigues and du Buf (2004 , 2006 , 2008 . We showed that keypoint scale space provides ideal information for constructing saliency maps for Focus-of-Attention (FoA), and that the grouping of keypoints at different scales is robust for face detection; Rodrigues and du Buf (2006) . Therefore, keypoints and FoA are thought to provide major cornerstones for the ''where'' system. In parallel, we showed that the multiscale line/edge representation provides ideal information for object and face recognition, Rodrigues and du Buf (2008) A less open question concerns the use of features detected at different scales: information at coarse scales propagates first to higher areas, after which information at progressively finer scales arrives there; Bar (2004) . This probably implies that coarse-scale information is used for a first, fast, but rough categorization, after which categorization is refined using information at progressively finer scales until an object is recognized. Bar (2003) proposed that a first categorization is based on a lowpass-filtered image of the object, but a smeared blob lacks structure. In our own experiments, Rodrigues and du Buf (2008) , we therefore applied a different approach: after segregation, the coarse-scale line/edge representation of the outline is used for pre-categorization, after which all information is used for final categorization and recognition.. Any 3D object can lead to an infinite number of different projected images on the retinae due to variations in position, distance, lighting, and other factors including rotation and deformation. The ability to identify objects despite all possible transformations is central to visual object recognition. However, this still is a poorly understood mechanism, Cox et al. (2005) , and transform-tolerant recognition remains a major problem in the development of artificial vision systems. In our brain, transform-invariant object recognition is automatic and robust, but it ultimately depends on experience; Tarr (2005) . Recent findings, e.g., Cox et al. (2005) , even support the idea that visual representations in the brain are plastic and largely a product of our visual environment and that invariant object representations are not rigid nor finalized-they are continually evolving entities, ready to adapt to changes in the environment. This idea complicates the classical idea of static representations in which only two but related problems need to be solved: (1) partial invariance to reasonable transformations like 2D rotation in the case of any canonical object view, which is addressed in this paper, and (2) the total number of (3D) canonical object views that must be stored in memory. However, also plasticity can be explored at the two levels, in this paper in the form of dynamic routing for obtaining partial invariance to reasonable transformations.. There are several approaches to biological object recognition. Here, we focus briefly on approaches which, to some degree, are related to our own approach and architecture. Olshausen et al. (1993) described a model that relies on a set of control neurons, which dynamically modify the synaptic strengths of intracortical connections such that information from a windowed region of the primary cortex is selectively routed to higher cortical areas. Local spatial relationships (i.e. topography) within the attentional window are preserved as information is routed through the cortex. This enables attended objects to be represented in higher areas within an object-centered reference frame that is position and size invariant. Olshausen et al. hypothesize that the pulvinar (at the posterior part of the thalamus) may provide the control signals for routing information through the cortex. In preattentive mode, the control neurons receive their input from a low-level ''saliency map'' representing potentially interesting regions of a scene. During the pattern-recognition phase, control neurons are driven by the interaction between top-down (memory) and bottom-up (retinal input) sources.. In Rensink's (2000) triadic architecture, early preattentive processes feed both an attentional system concerned with coherent objects, and a non-attentional system concerned with scene gist and spatial layout. Instead of operating sequentially, the latter two subsystems operate concurrently for providing a context that can guide the allocation of attention. In this view, attention is no longer a central gateway through which all information must pass, but just one system that operates concurrently with several other (sub)systems. Furthermore, a scene is experienced via a ''virtual representation'' in which object representations are formed in a ''just-in-time'' fashion, only existing as long as they are needed.. The laterally interconnected synergetically self-organizing map (LISSOM), Miikkulainen et al. (2005) , consists of a ''family'' of computational models which aim to replicate the detailed development of the visual cortex. The model can explain invariant (only size and viewpoint) detection of objects like faces. Hamker (2005) presented a feature-based computational model for invariant (but only translation) object detection in complex backgrounds (natural scenes) driven by attention in V4 and IT.. The collaboration called ''Detection and Recognition of Objects in the Visual Cortex'' integrates effort at several laboratories, aiming at a quantitative, hierarchical recognition model. The integrated architecture, like our own, reflects the general organization of the visual cortex in a stack of layers from V1 to IT to PF cortex; Serre and Riesenhuber (2004) . Walther et al. (2005) are extending the basic recognition model by integrating a saliency-based and essentially bottom-up attentional model. Deco and Rolls (2004) presented an invariant model that incorporates feedback-biasing effects of top-down attentional mechanisms in a hierarchically organized set of cortical areas with convergent feed-forward connectivity, reciprocal feedback connections and local area competition. The model displays space-based and object-based covert visual search by using attentional top-down feedback from either the PP or the IT cortical modules, with interactions between the ventral and dorsal data streams occurring in V1 and V2. Deco and Rolls (2005) described a computational framework and showed how an attentional state held in short-term memory in PF cortex can, by topdown processing, influence the ventral and dorsal data streams in different cortical areas. Stringer et al. (2006) showed that invariant object recognition can be based on spatio-temporal continuity (during object translation and rotation) with ''continuous transformation (CT) learning,'' which operates by mapping spatially similar input patterns to the same postsynaptic neurons in a competitive neural network system. The goal of this paper is to show that low-level processing in terms of multi-scale feature extractions (keypoints, lines and edges) can be extended to higher-level processing: invariance in object categorization and recognition. We present a new model for obtaining 2D translation, rotation and size invariance by dynamic mapping of saliency maps based on multi-scale keypoint information. In addition, we present an integrated architecture in which coarse-scale information is used for a first but rough categorization, after which additional information at finer scales is used to refine categorization until objects are identified. As a consequence, extended models can cover more cognitive aspects in the near future. For example, processes like the learning of new objects or new, unexpected views of known objects will become subject to explicit modeling.. The rest of this paper is organized as follows: the next section deals with multi-scale feature extraction: lines, edges and keypoints plus the construction of saliency maps. Invariant categorization and recognition by dynamic routing, the construction of group templates and experimental results are presented in Section ''Invariant object categorization and recognition''. Section ''The creation of group templates'' concerns an integrated cortical architecture for the invariant categorization and recognition model. In the ''Discussion'' Section we discuss our approach and lines for future research. Mathematical formulations of the methods are provided in Appendix. There are many properties of a real-world scene that can be defined independently of the objects. For instance, a forest scene with trees can be described in terms of the degree of roughness and homogeneity of its textural components. Oliva and Torralba (2006) conclude that there is converging evidence that natural scene recognition may not depend on recognizing objects, and that the gist does not need to be built on top of the processing of individual objects. Nevertheless, these processes are complementary. The initial gist can be the key for selecting the first group templates to start object recognition, but at some stage the objects should corroborate for the interpretation of the scene, and those objects must somehow be segregated. Any computational model of the cortical architecture should start with a model for getting the gist (forest scene), after which object recognition follows using segregated items, from generic information (trees) to more detailed information (tree type, leaf type). Only at the end of the entire process it may be possible to specify the gist; for example, a Mediterranean forest with tall pine trees.. Gist has not yet been implemented in our architecture, because we think that segregation of complex environments like natural scenes and gist are well interconnected processes. These processes may be based on complementary information channels which address motion and disparity, but also surface properties instead of structural object shape: (a) color processing in the cytochrome oxidase blobs, which are embedded in the cortical hypercolumns with simple, complex and end-stopped cells for line, edge and keypoint coding, must attribute colors to homogeneous (line/edge-free but also textured) object surfaces, and (b) texture coding based on specific groupings of outputs of grating cells in the case of rather periodic patterns, or other but similar processes in the case of more stochastic patterns. As shown by du Buf (2006) , groupings of outputs of grating cells is a straightforward, data-driven and, therefore, fast bottom-up process which provides a segmentation (segregation) of linear, rectangular, and hexagonal textures. Therefore, a gist model, when seeing an image with blue and some white above green with a rather irregular pattern, may classify the scene, after sufficient training of course, as Mediterranean outdoor, thereby pre-selecting tree templates with a bias toward different pine trees (tall and more round etc.).. Not yet having a gist model, we simply assumed in our experiments that all group templates are available at the first categorization level, and that input objects are always seen against a homogeneous background (i.e. already segregated). At an early stage, only very coarse scales with big intervals are available, then medium scales with smaller intervals appear and finally the fine scales. The appearance and therefore the use of scales is directly related to all steps of the recognition process. The initial segregation starts with coarse scales, which provide a very diffuse object representation. This first segregation triggers a first categorization. When medium scales appear, and then fine scales, the segregation is improved and so is the categorization. The same occurs with the construction of the saliency map, first using keypoints detected at coarse scales and improving the map by adding keypoints detected at increasingly finer scales.. Invariance by neural routing from V1 via V2 to V4 etc. is based on the recurrent network layers used in the Deco and Rolls (2004) model, however, with one big difference: instead of only using simple cells (Gabor model) we apply explicit feature extractions and can use specific features to guide the routing. As a matter of fact, the routing can be seen as two vessels (input object and template) throwing anchors toward each other: the first, big anchor is the central object keypoint at very coarse scales and this is used to ''position'' the normalized template above the (shifted) input object. The second anchor is the most significant peak of the saliency map, obtained by summing keypoints over many scales, and this is used to match rotation and size. Once ''anchored together,'' the ''ropes'' are used to steer many more ropes that connect specific structures of the vessels, like bow, rail and stern, in order to check whether the structures are similar and the vessels are of the same type.. Our ''anchoring'' method is similar to the theory developed by Olshausen et al. (1993) , suggesting that the position and size of the reference frame can be set by the position and size of the object in the scene, assuming that the scene is at least roughly segmented, and that the orientation of the reference frame can be estimated from relatively low-level cues. The computational advantage of such a system is obvious: only a few views of an object need to be stored for recognition under different viewing conditions. The disadvantage, of course, is that a scene containing multiple objects requires serial processing, the system only being able to attend one object at a time. The same happens in our model and that of Deco and Rolls: dynamic routing steers the information flow by adapting neural interconnections in V2, etc. for some time, until recognition has been achieved, after which the adapted steering can be released for the inspection of another object (or region around a fixation point). Psychophysical evidence suggests that the brain, indeed, employs such a sequential strategy; Rensink (2000) .. An interesting aspect of models is which features-and therefore which image representation-are being used. In our own model, explicit features are used: lines, edges, and keypoints are detected on the basis of responses of simple, complex, and end-stopped cells. The existence of other cells with very specific functions, like bar and grating cells, points at explicit feature extractions with increasing complexity at higher cortical areas; Rodrigues and du Buf (2006) ; du Buf (2006) . The same idea, extended with increasing receptive field sizes, is supported by Deco and Rolls (2004) , however, without explicit feature extractions. By only using simple cells (Gabor model), higher features are represented implicitly: complex cells group outputs of simple cells and end-stopped cells group outputs of complex cells. Nevertheless, in principle-they did not test this-their model should also be able to achieve invariant object recognition by combining feedback effects of top-down attentional mechanisms in a hierarchically organized set of cortical areas with convergent forward connectivity, reciprocal feedback connections, and local intra-area competition. As a consequence, we may say that these two models are converging, but eventually the same will happen with other computational models; Olshausen et al. (1993); Hamker (2005) .. The templates used to illustrate the architecture were built from a very small database of 80 different objects. In future work the system must be extended and tested against a huge number of objects with many more categorization levels, thereby simulating a real application which approaches the challenge that our visual system faces every day. In this case, instead of having only three levels to obtain object recognition (pre-categorization, categorization and recognition), we will have n levels as shown in Fig. 7 , each level only having N elements. An input object must not be compared with all the templates in the entire database, nor with a significant number of group templates. Part of the solution, not even mentioned until here, is to apply biasing of associative memory over time, as occurs in our brains: data streams in the case of frequently and recently seen objects are short and fast, whereas those in the case of occasionally and sparsely seen objects are longer and slower.. Every time that an object is recognized, its features could be added to all the matching templates. This way the system will be able to learn by updating the database using the most recent views of common objects. As discussed before, this can be done by biasing associative memory, but the memory itself must also be changed, for example by a weighted summation of new and old features, which can be fast in short-term memory but much slower in long-term memory. These ideas raise some problems which are not yet addressed by the present architecture: (a) for avoiding overgeneralization of the groups (classes), a threshold has to be implemented such that, before a class becomes too generic, it can be split (shown in Fig. 7 by ''\ \ \''), and yet the two classes which provoked that split must still be generic enough for either class. (b) Objects with noise or occasional variations, like changing illumination or the deformation of non-rigid objects, or which are at the ''edge'' between two classes, are the ones which pose most problems. Therefore, their features should have a much smaller weight when contributing to the templates. (c) In general, different classes can have different weight factors, for example as a function of the number of objects that has been recognized within the class (group), creating the idea of a priority-secondary (but in reality continuous) organization, especially if temporal modulation (frequently and recently vs. occasionally and sparsely seen objects) is also applied. In addition, temporal changes, not only a sudden change of context, can lead to an evolution of the hierarchy of templates: they can shift forward toward early (coarse) categorization or back toward late (fine) categorization. (d) Finally, the contributions of all features can be weighted, especially when additional features like disparity (3D shape), texture and color will be included in the system. Apples vary between green, yellow, orange, red, and brown, so only blue can be excluded, but all oranges are orange and tomatoes are either green or red. However, such rules are not fixed because yellow and orange bell peppers appeared next to red and green ones only a few years ago in the supermarkets.. Summarizing, we presented a new model for invariant object categorization and recognition based on realistic multi-scale features which are extracted in the primary visual cortex. The model employs dynamic routing of features through the different layers to obtain 2D translation, rotation, and size invariance. The model was tested in the framework of an integrated and biologically plausible architecture in which information at coarse scales is used first and information at progressively finer scales later. By employing feedback loops, which are known to exist in abundance in the visual cortex, attention information based on keypoints and saliency maps is used to control the process. The entire process is composed of different categorization levels, recognition being the last one, with sequentially (but overlapping) coarse-to-fine-scale processing.. Although not yet yielding perfect results, the architecture can deal with reasonable translations, rotations, and scalings. In a next step, the maximally allowable transformations must be determined, which depend on the number of neural layers used in the routing, and this will provide information on how many views of objects must be stored in memory.. withx ¼ x cos h þ y sin h andỹ ¼ y cos h À x sin h, the aspect ratio c = 0.5 and r determines the size of the RF. The spatial frequency is 1/k, k being the wavelength. For the bandwidth r/k we use 0.56, which yields a halfresponse width of one octave. The angle h determines the orientation (we use 8 orientations), and / the symmetry. We can apply a linear scaling between f min and f max with hundreds of contiguous scales. The scale of analysis is given in terms of k expressed in pixels (k = 1 corresponds to 1 pixel, and images presented have a size of 256 9 256 pixels).. Responses of even R s,i E (x, y) (with / = 0) and odd . There are two types of end-stopped cells (Heitger et al. (1992) ), single (S) and double (D). If [.] ? denotes the suppression of negative values, and usingĈ i ¼ cos h i and S i ¼ sin h i , then S s;i ðx; yÞ. and. The responses of the two cell types are obtained by S s x; y ð Þ ¼ P N h À1 i¼0 S s;i ðx; yÞ and D s x; y ð Þ ¼ P N h À1 i¼0 D s;i ðx; yÞ: The distance d is scaled linearly with the filter scale s (we use d = 0.6 s). To compute the keypoint maps, all responses of end-stopped cells along straight lines and edges are suppressed, for which tangential (T) and radial (R) inhibitions are used:. and. Then we apply I s = I s T ? I s R for obtaining the end-stop- i.e., the orientation with the maximum response of Ĉ s;i , where. denote lateral (L) and cross-orientation (C) inhibition, which are necessary because simple and complex cells respond beyond line and edge terminations, for example beyond the corners of a rectangle, see Rodrigues and du Buf (2008) , using b & 1. At each position (x, y) for which Ĉ s [ 0, with As explained in the Section ''Lines, edges, keypoints and saliency maps'', we can assume that the visual system extracts lines and edges for object recognition, and that responding ''line cells'' and ''edge cells'' are also interpreted symbolically for creating a brightness representation; see du Buf and Fisher (1995) . For each of the four event maps L s,± (x, y) and E s,± (x, y), the corresponding 1D profiles W s,± (x) and K s,± (x) are rotated to the dominant orientation i d , and multiplied by the amplitude of the complex cells C s;i d x; y ð Þ at the detected positions. For generating the 2D representation maps W s,± (x, y) and K s,± (x, y), it is necessary to interpolate values between two consecutive profiles (of neighboring cells) such that gaps are filled. In Fig. 1 with LP(x, y; r r ) a Gaussian-filtered lowpass image, c a coefficient which balances the lowpass component and the line/edge representations, and N s the number of scales used. The rightmost image on the middle row in Fig. 1 was obtained with c = 0.5, r r = 5 and N s = 8 (k = {4 ; 8; 12; 16; 20; 24; 28; 32}) . Processes like the 2D interpolation of 1D cross-profiles are speculative, but they are necessary for showing 2D images; for more details see Rodrigues and du Buf (2008) . The results obtained were quite good: from the 64 modified input images, pre-categorization (animal, fruit, car, and cup) failed in 12 cases. Of the remaining 52 images, categorization (animal: horse, cow, dog; fruit: tomato, pear, apple) failed in 8 cases. Recognition failed for 4 of the 44 remaining images. The final recognition rate is therefore 62.5%. However, the above numbers are not definitive because they concern a first test of the concept and many errors can be explained. For example, some image manipulations were too extreme and we could have selected less extreme manipulations.. As for our previous results obtained with only normalized objects, Rodrigues and du Buf (2008) , categorization errors occurred mainly for apples and tomatoes, which can be explained by the fact that the shapes are very similar and no color information has been used. In pre-categorization some fruits were categorized as cups. This mainly concerned pears and can be explained by the tapered-elliptical shape in combination with size variations, such that keypoints and line/edge events of input pears can coincide with those of the cups-group template (Fig. 5 top-right) . As expected, especially in the case of recognition, problems occurred with extreme size variations. The scales used (k = [4, 32]) are related to the size of the objects and the level of detail that can be represented. Figure 6 (middle three images in the fourth column) shows the smallest objects that could be dealt with by using these scales. The image at bottom-right proved too extreme (all modified objects shown on the bottom line were not correctly categorized or recognized).. It should be emphasized that the method can be applied to images which contain multiple objects. Although our visual system has a limited ''bandwidth'' and can test only one object at any time, Rensink (2000) , this problem can be solved by sequential processing of all detected and segregated objects. However, if object segregation and recognition are coupled processes, we are left with a typical chicken-or-egg problem, unless the process is controlled by, e.g., the gist system (see Discussion). Finally, it should be mentioned that dynamic routing of keypoints (significant peaks in saliency maps) and line/edge events in intermediate neural layers has consequences for the minimum number of canonical object views in memory, i.e., the number of templates. If a horse template has the head to the left and the legs down, but an input horse has been rotated (2D) by 180 degrees such that the head is to the right and the legs are up, dynamic routing will not be possible because there will be a crossing point in the routing at some neural layer. In this case a separate template is necessary. In addition, recognition in the case of 3D rotation may require more templates because of asymmetrical patterns of a horse's fell on its left and right flanks.

O O O O O O B O B O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O B I O O O O O O O O O O O O O O B O B O O O O O O O O O B I O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O


On the decomposition of stochastic cellular automata In this paper we present two interesting properties of stochastic cellular automata that can be helpful in analyzing the dynamical behavior of such automata. The first property allows calculating cell-wise probability distributions over the state set of a stochastic cellular automaton, i.e. images that show the average state of each cell during the stochastic cellular automaton evolution. The second property shows that stochastic cellular automata are equivalent to so-called stochastic mixtures of deterministic cellular automata. By means of this property, any stochastic cellular automaton can be decomposed into a set of deterministic systems, each of which contributes to the behavior of the stochastic system. Cellular Automata (CAs) are often used for constructing models in a variety of fields of application including chemistry, biology, medicine, physics, ecology and the study of socioeconomic interactions. In many of these settings, Stochastic CAs (SCAs) are considered due to the stochastic nature of the phenomenon being modeled or the lack of understanding of the exact rules driving the phenomenon [1, 2] . Therefore, a better understanding of the dynamics of SCAs is crucial. Only a few methods for dealing with the analysis of models based on SCAs have been developed. In the case of many practical applications, sampling methods, relying on repetitive computer simulations are sufficient [3, 4, 5] , especially in cases when the averaged behavior of the system is concerned. Techniques built on the mean-field theory can be used to study correlations and long-term behavior of SCAs [6] . The theory of Markov chains can be applied to provide analytical tools for analyzing the model's behavior [7] although in practice, due to the theoretical and computational complexity of such methods, the application scope is limited. This paper is devoted to providing effective analytical tools for the analysis of multi-state SCAs, which are based on deterministic CAs. Although the theoretical foundation of the presented results is already (at least partially) known in the literature [7, 8, 9] , so far the applications are limited. Therefore, the main aim of this paper is to provide a complete, formal description of the discussed properties in a form that is suitable for applications and does not require a strong mathematical background, as well as to present examples that can motivate further applications in the domain of systems modeling.. The methods presented here are developed in the context of 1D SCAs on finite lattices, but can be easily generalized to the case of higher dimensions and infinite spaces.. Two main results are presented in this paper. The first one involves constructing images that show the cell-wise probability distribution over the state set, at any time step. The method is based on associating an SCA with a deterministic, Continuous CA (CCA). The second result shows the equivalence of SCAs and stochastic mixtures of deterministic CAs. Based on this finding, any SCA can be decomposed into a finite set of deterministic CAs, each of them contributing to the behavior of the stochastic system. An effective method for finding a decomposition is presented. It allows to uncover the deterministic component in the mixture with the highest impact on the behavior of the SCA. This paper is organized as follows. We start with some preliminaries and definitions in Section 2. In Section 3, we introduce the concept of CCAs and the formalism enabling the analysis of multi-state CAs. Section 4 contains the definition of multi-state SCAs and holds the main results of this paper. The paper is concluded with Section 5, discussing the experimental results that illustrate both properties. A summary is presented in Section 6. In this paper, two basic, yet important properties of SCAs were discussed. We have shown, both theoretically and practically that SCAs can be effectively analyzed in the context of deterministic CAs and CCAs. The stochastic mixture representation of SCAs in some cases allows to understand the underlying dynamics, while the CCA representation allows to quickly uncover the averaged behavior of the system. Further research is undertaken to further expand the application scope of the presented methods. Figure 6 : Impact of the probabilities p 1 and p 2 on the distances of space-time diagrams ∆(p 1 , p 2 ) and the final configurations after T = 49 time steps ∆ T (p 1 , p 2 ) of the totalistic SCA, obtained from the same initial condition.

O O B O B I I O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O