Order Selection of the Linear Mixing Model for Complex-Valued FMRI Data Functional magnetic resonance imaging (fMRI) data are originally acquired as complex-valued images, which motivates the use of complex-valued data analysis methods. Due to the high dimension and high noise level of fMRI data, order selection and dimension reduction are important procedures for multivariate analysis methods such as independent component analysis (ICA). In this work, we develop a complex-valued order selection method to estimate the dimension of signal subspace using information-theoretic criteria. To correct the effect of sample dependence to information-theoretic criteria, we develop a general entropy rate measure for complex Gaussian random process to calibrate the independent and identically distributed (i.i.d.) sampling scheme in the complex domain. We show the effectiveness of the approach for order selection on both simulated and actual fMRI data. A comparison between the results of order selection and ICA on real-valued and complex-valued fMRI data demonstrates that a fully complex analysis extracts more information about brain activation. Functional magnetic resonance imaging (fMRI) is a non-invasive and powerful brain imaging technique that has been utilized since the early 1990s [1] to study human brain function. The MRI signal is intrinsically complex-valued due to the fact that the signal contrast is originally acquired in a spatial frequency space, i.e., the k-space, and image reconstruction is achieved through an inverse Fourier transform. Conventionally, only the magnitude of fMRI data is used in analysis and the phase information is discarded. Studies show that phase of fMRI signal contains useful information for inference on blood oxygenation during functional activation [2] and the orientation of large blood vessels [3] as well as different tissue types [4] . In a recent proposed fMRI technology [5] , the phase change is utilized to estimate the brain activation.. To utilize phase information from complex-valued fMRI data, the general linear model (GLM) based analysis has been extended to the complex domain [6] , [7] . Although widely used, the GLM is only able to infer brain activations from a priori response time sequences.. provide a more flexible alternative to estimate multiple brain activation sources without a specified response model [8] . A group of commonly used complex ICA approaches, such as maximum likelihood, maximization of non-Gaussianity and nonlinear decorrelations [9] , [10] , have been effectively applied in the analysis of complex-valued fMRI data [11] , [12] , and shown advantages in estimation of brain activation [13] , [14] .. Given the abundance of information in complex-valued fMRI data, several challenges exist for applying the complex ICA approach. First, fMRI data has low signal to contrast ratio, typically about 0 dB for a robust task paradigm, which indicates that a noise model has to be incorporated into the analysis. Second, fMRI data has high spatial and temporal dimension, e.g., 10,000-100,000 spatial voxels by 100-1000 time points. Direct application of a multivariate analysis method, such as ICA, on high dimensional datasets is liable to overfitting. Therefore, order selection and dimension reduction are typically incorporated as steps before ICA on fMRI data. Information-theoretic criteria, such as Akaike's information criterion (AIC) [15] , Draper's information criterion (DIC) [16] and the minimum description length (MDL) criterion [17] are reasonable candidates of the order selection criteria, for that the optimal model orders in these criteria are automatically selected based on the trade-off between the maximum likelihood of the model and the penalty on model complexity [18] , [19] . In addition, other criteria are developed based on, e.g., maximizing a Laplace approximation to the posterior distribution of the model evidence [20] , and the l 2 norm of residual error [21] .. One important assumption in estimating information-theoretic criteria scores for a selected model is that the data samples are independent and identically distributed (i.i.d.). However, the fMRI data samples have both spatial and temporal correlation. In the spatial domain, the localization of brain function causes the brain activation pattern to be spatially smooth and clustered [22] , [23] . The dynamic cerebral perfusion and the point spread function of the imaging system also introduce sample dependence on adjacent voxels in the image. In the temporal domain, the hemodynamic response function introduces dependence on fMRI time sequences. The fMRI sample dependence is localized and can be measured by entropy rate when the spatial or temporal fMRI data are modeled as a random process. Using the theoretical upper bound of entropy rate for Gaussian random process as the bench mark, a sampling scheme is proposed in [19] , to remove localized sample dependence in real-valued fMRI data and identify a subset of effectively i.i.d. samples to correctly calculate information-theoretic criteria for selecting model order.. In [14] , information-theoretic criteria for order selection on complex-valued model is developed and preliminary results are presented on complex-valued fMRI data. However, the i.i.d. sampling scheme in the complex domain is based on a limiting assumption that the real and imaginary parts of the fMRI samples are uncorrelated. A general form of entropy rate for complex random Gaussian process which takes into account the correlation between real and imaginary parts of complex-valued processes is given in [24] . We use the general entropy rate formula to calibrate the i.i.d. sampling scheme and hence to correctly calculate the information-theoretic criteria scores for order selection. We compare order selection results on real-valued (magnitude-only) and complex-valued fMRI dataset to observe that the additional information provided by the complex-valued data causes informationtheoretic criteria to select a model with higher order. We also study real-valued and complex-valued ICA estimation results to identify common components estimated by both methods, as well as additional components obtained by complex ICA only. The common and additional components suggest that a fully complex analysis may be more efficient in utilizing the fMRI data and hence provides a promising way of investigating brain function.. In the next section, we introduce the order selection with information-theoretic criteria and i.i.d. sampling on complex-valued fMRI data. We present experimental results of the order selection scheme using i.i.d. sampling, on simulated and actual fMRI data in Section III, and provide a discussion of the results in Section IV. In the Appendix, we give the entropy rate formula for a complex-valued Gaussian random process. In this paper, we study order selection for multivariate analysis on complex-valued fMRI data. By modeling the complex data as a finite-order MA process, we extend the i.i.d. sampling scheme proposed in [19] into complex-valued sample space, using entropy rate derived for the complex domain. It is shown both by simulated and actual data that the scheme improves the performance of order selection with information-theoretic criteria when the samples are correlated. The order selection scheme with i.i.d. sampling on complex-valued fMRI data is important for the performance of data-driven analysis approaches such as ICA. The developed method can be utilized in other signal processing scenarios where a lower dimensional informative subspace needs to be identified from high dimensional noisy observations.. We study the issue of model order selection in complex-valued fMRI data analysis in this work. Compared with magnitude-only fMRI data analysis, the results from order selection and ICA on complex-valued fMRI data suggest that a more comprehensive approach, adding the phase information, provides additional benefits. The fully complex analysis of fMRI data is a promising direction where several issues are of interest, such as study of order selection criteria other than information-theoretic criteria for complex fMRI data analysis, comparison of complex ICA algorithms for brain source estimation, preprocessing strategies for complex-valued fMRI data, and investigation of the phase signal variation on estimation of brain function. In Section III, we show a brain activation pattern consistently estimated by complex ICA on selected model order but not obtained in the real-valued analysis on the same dataset. The component shows spatially localized activation in part of the parietal lobe, suggesting that, part of it is involved in the performed finger tapping task. Although preliminary, the result motivates the application of a fully complex data-driven method on fMRI data, either as an alternative to conventional analysis, or, as a novel tool to investigate brain function in more complicated cognitive tasks.

B I O O B I I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B I O O O O O O O O O O O B I O O B I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B I O B I O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O B I O O O O O O O O O O O O O O B I O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Generic Fibrational Induction Abstract. This paper provides an induction rule that can be used to prove properties of data structures whose types are inductive, i.e., are carriers of initial algebras of functors. Our results are semantic in nature and are inspired by Hermida and Jacobs' elegant algebraic formulation of induction for polynomial data types. Our contribution is to derive, under slightly different assumptions, a sound induction rule that is generic over all inductive types, polynomial or not. Our induction rule is generic over the kinds of properties to be proved as well: like Hermida and Jacobs, we work in a general fibrational setting and so can accommodate very general notions of properties on inductive types rather than just those of a particular syntactic form. We establish the soundness of our generic induction rule by reducing induction to iteration. We then show how our generic induction rule can be instantiated to give induction rules for the data types of rose trees, finite hereditary sets, and hyperfunctions. The first of these lies outside the scope of Hermida and Jacobs' work because it is not polynomial, and as far as we are aware, no induction rules have been known to exist for the second and third in a general fibrational framework. Our instantiation for hyperfunctions underscores the value of working in the general fibrational setting since this data type cannot be interpreted as a set. Iteration operators provide a uniform way to express common and naturally occurring patterns of recursion over inductive data types. Expressing recursion via iteration operators makes code easier to read, write, and understand; facilitates code reuse; guarantees properties of programs such as totality and termination; and supports optimising program transformations such as fold fusion and short cut fusion. Categorically, iteration operators arise from initial algebra semantics of data types, in which data types are regarded as carriers of initial algebras of functors. Lambek's Lemma ensures that the carrier of the initial algebra of F is its least fixed point µF , and initiality ensures that, given any F -algebra h : F A → A, there is a unique F -algebra homomorphism, denoted f old h, from the initial algebra in : F (µF ) → µF to that algebra. For each functor F , the map fold : (F A → A) → µF → A is the iteration operator for the data type µF . Initial algebra semantics thus provides a well-developed theory of iteration which is.... • ...principled, in that it is derived solely from the initial algebra semantics of data types. This is important because it helps ensure that programs have rigorous mathematical foundations that can be used to ascertain their meaning and correctness.. • ...expressive, and so is applicable to all inductive types -i.e., to every type which is the carrier of an initial algebra of a functor -rather than just to syntactically defined classes of data types such as polynomial data types.. • ...correct, and so is valid in any model -set-theoretic, domain-theoretic, realisability, etc. -in which data types are interpreted as carriers of initial algebras.. Because induction and iteration are closely linked -induction is often used to prove properties of functions defined by iteration, and the correctness of induction rules is often established by reducing it to that of iteration -we may reasonably expect that initial algebra semantics can be used to derive a principled, expressive, and correct theory of induction for data types as well. In most treatments of induction, given a functor F together with a property P to be proved about data of type µF , the premises of the induction rule for µF constitute an F -algebra with carrier Σx : µF. P x. The conclusion of the rule is obtained by supplying such an F -algebra as input to the iteration operator for µF . This yields a function from µF to Σx : µF. P x from which a function of type ∀x : µF. P x can be obtained. It has not, however, been possible to characterise F -algebras with carrier Σx : µF. P x without additional assumptions on F . Induction rules are thus typically derived under the assumption that the functors involved have a certain structure, e.g., that they are polynomial. Moreover, taking the carriers of the algebras to be Σ-types assumes that properties are represented as type-valued functions. So while induction rules derived as described above are both principled and correct, their expressiveness is limited along two dimensions: with respect to the data types for which they can be derived and the nature of the properties they can verify. A more expressive, yet still principled and correct, approach to induction is given by Hermida and Jacobs [10] . They show how to lift each functor F on a base category of types to a functorF on a category of properties over those types, and take the premises of the induction rule for the type µF to be anF -algebra. Hermida and Jacobs work in a fibrational setting and the notion of property they consider is, accordingly, very general. Indeed, they accommodate any notion of property that can be suitably fibred over the category of types, and so overcome one of the two limitations mentioned above. On the other hand, their approach gives sound induction rules only for polynomial data types, so the limitation on the class of data types treated remains in their work.. This paper shows how to remove the restriction on the class of data types treated. Our main result is a derivation of a sound generic induction rule that can be instantiated to every inductive type, regardless of whether it is polynomial or not. We think this is important because it provides a counterpart for induction to the existence of an iteration operator for every inductive type. We take Hermida and Jacobs' approach as our point of departure and show that, under slightly different assumptions on the fibration involved, we can lift any functor on the base category of a fibration to a functor on the total category of the fibration. The lifting we define forms the basis of our generic induction rule.. The derivation of a generic, sound induction rule covering all inductive types is clearly an important theoretical result, but it also has practical consequences:. • We show in Example 2 how our generic induction rule can be instantiated to the families fibration over Set (the fibration most often implicitly used by type theorists and those constructing inductive proofs with theorem provers) to derive the induction rule for rose trees that one would intuitively expect. The data type of rose trees lies outside the scope of Hermida and Jacobs' results because it is not polynomial. On the other hand, an induction rule for rose trees is available in the proof assistant Coq, although it is neither the one we intuitively expect nor expressive enough to prove properties that ought to be amenable to inductive proof. Indeed, if we define rose trees in Coq by. Node : list rose -> rose then Coq generates the following induction rule rose_ind : forall P : rose -> Prop, (forall l : list rose, P (Node l)) -> forall r : rose, P r. But to prove a property of a rose tree Node l, we must prove that property assuming only that l is a list of rose trees, and without recourse to any induction hypothesis. There is, of course, a presentation of rose trees by mutual recursion as well, but this doesn't give the expected induction rule in Coq either. Intuitively, what we expect is an induction rule whose premise is forall [r_0, ..., r_n] : list rose, P(r_0) -> ... -> P(r_n) -> P(Node [r_0, ..., r_n]). The rule we derive for rose trees is indeed the expected one, which suggests that our derivation may enable automatic generation of more useful induction rules in Coq, rather than requiring the user to hand code them as is currently necessary.. • We further show in Example 3 how our generic induction rule can be instantiated, again to the families fibration over Set, to derive a rule for the data type of finite hereditary sets. This data type is defined in terms of quotients and so lies outside most current theories of induction.. • Finally, we show in Example 7 how our generic induction rule can be instantiated to the subobject fibration over ωCP O ⊥ to derive a rule for the data type of hyperfunctions. Because this data type cannot be interpreted as a set, a fibration other than the families fibration over Set is required; in this case, use of the subobject fibration allows us to derive an induction rule for admissible subsets of hyperfunctions. The ability to treat the data type of hyperfunctions thus underscores the importance of developing our results in the general fibrational framework. Moreover, the functor underlying the data type of hyperfunctions is not strictly positive [7] , so the ability to treat this data type also underscores the advantage of being able to handle a very general class of functors going beyond simply polynomial functors. As far as we know, induction rules for finite hereditary sets and hyperfunctions have not previously existed in the general fibrational framework.. Although our theory of induction is applicable to all inductive functors -i.e., to all functors having initial algebras, including those giving rise to nested types [15] , GADTs [21] , indexed containers [1] , dependent types [19] , and inductive recursive types [6] -our examples show that working in the general fibrational setting is beneficial even if we restrict our attention to strictly positive data types. We do, however, offer some preliminary thoughts in Section 5 on the potentially delicate issue of instantiating our general theory with fibrations appropriate for deriving induction rules for specific classes of higher-order functors of interest. It is also worth noting that the specialisations of our generic induction rule to polynomial functors in the families fibration over Set coincide exactly with the induction rules of Hermida and Jacobs. But the structure we require of fibrations generally is slightly different from that required by Hermida and Jacobs, so while our theory is in essence a generalisation of theirs, the two are, strictly speaking, incomparable. The structure we require of our fibrations is, nevertheless, certainly present in all standard fibrational models of type theory (see Section 4). Like Hermida and Jacobs, we prove our generic induction rule correct by reducing induction to iteration. A more detailed discussion of when our induction rules coincide with those of Hermida and Jacobs is given in Section 4.. We take a purely categorical approach to induction in this paper, and derive our generic induction rule from only the initial algebra semantics of data types. As a result, our work is inherently extensional. Although translating our constructions into intensional settings may therefore require additional effort, we expect the guidance offered by the categorical viewpoint to support the derivation of induction rules for functors that are not treatable at present. Since we do not use any form of impredicativity in our constructions, and instead use only the weaker assumption that initial algebras exist, this guidance will be widely applicable.. The remainder of this paper is structured as follows. To make our results as accessible as possible, we illustrate them in Section 2 with a categorical derivation of the familiar induction rule for the natural numbers. In Section 3 we derive an induction rule for the special case of the families fibration over Set. We also show how this rule can be instantiated to derive the one from Section 2, and the ones for rose trees and finite hereditary sets mentioned above. Then, in Section 4 we present our generic fibrational induction rule, establish a number of results about it, and illustrate it with the aforementioned application to hyperfunctions. The approach taken in this section is completely different from the corresponding one in the conference version of the paper [9] , and allows us to improve upon and extend our previous results. Section 5 concludes, discusses possible instantiations of our generic induction rule for higher-order functors, and offers some additional directions for future research.. When convenient, we identify isomorphic objects of a category and write = rather than ≃. We write 1 for the canonical singleton set and denote its single element by · . In Sections 2 and 3 we assume that types are interpreted as objects in Set, so that 1 also denotes the unit type in those sections. We write id for identity morphisms in a category and Id for the identity functor on a category. We have given a sound induction rule that can be used to prove properties of data structures of inductive types. Like Hermida and Jacobs, we give a fibrational account of induction, but we derive, under slightly different assumptions on fibrations, a generic induction rule that can be instantiated to any inductive type rather than just to polynomial ones. This rule is based on initial algebra semantics of data types, and is parameterised over both the data types and the properties involved. It is also principled, expressive, and correct.. Our derivation yields the same induction rules as Hermida and Jacobs' when specialised to polynomial functors in the families fibration over Set and in other fibrations, but it also gives induction rules for non-polynomial data types such as rose trees, and for data types such as finite hereditary sets and hyperfunctions, for which no fibrational induction rules have previously been known to exist. There are several directions for future work. The most immediate is to instantiate our theory to give induction rules for more sophisticated data types, such as nested types. These are exemplified by the data type of perfect trees given in Haskell-like syntax as follows:. data PTree a : Set where PLeaf : a → PTree a PNode : PTree (a, a) → PTree a. Nested types arise as least fixed points of rank-2 functors; for example, the type of perfect trees is µH for the functor H given by HF = λX. X + F (X × X). An appropriate fibration for induction rules for nested types thus takes B to be the category of functors on Set, E to be the category of functors from Set to P, and U to be postcomposition with the forgetful functor from Section 3. A liftingĤ of H is given byĤ P X (inl a) = 1 and H P X (inr n) = P (X × X) n. Taking the premise to be anĤ-algebra gives the following induction rule for perfect trees:. indP T ree : ∀ (P : Set → P).. (U P = PTree) → (∀(X : Set)(x : X). P (PLeaf x)) → (∀(X : Set)(t : PTree (X × X). P (X × X) t → P (PNode t))) → ∀(X : Set)(t : PTree X). P X t. This rule can be used to show, for example, that P T ree is a functor. Extending the above instantiation for the codomain fibration to so-called "truly nested types" [15] and fibrations is current work. We expect to be able to instantiate our theory for truly nested types, GADTs, indexed containers, dependent types, and inductive recursive types, but initial investigations show care is needed. We must ascertain which fibrations can model predicates on such types, since the codomain fibration may not give useful induction rules, as well as how to translate the rules to which these fibrations give rise to an intensional setting.. Matthes [15] gives induction rules for nested types (including truly nested ones) in an intensional type theory. He handles only rank-2 functors that underlie nested types (while we handle any functor of any rank with an initial algebra), but his insights may help guide choices of fibrations for truly nested types. These may in turn inform choices for GADTs, indexed containers, and dependent types.. Induction rules can automatically be generated in many type theories. Within the Calculus of Constructions [4] an induction rule for a data type can be generated solely from the inductive structure of that type. Such generation is also a key idea in the Coq proof assistant [5] . As far as we know, generation can currently be done only for syntactic classes of functors rather than for all inductive functors with initial algebras. In some type theories induction schemes are added as axioms rather than generated. For example, attempts to generate induction schemes based on Church encodings in the Calculus of Constructions proved unsuccessful and so initiality was added to the system, thus giving the Calculus of Inductive Constructions. Whereas Matthes' work is based on concepts such as impredicativity and induction recursion rather than initial algebras, ours reduces induction to initiality, and may therefore help lay the groundwork for extending implementations of induction to more sophisticated data types.

O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O B O O O O O


On

expressiveness

of

the

chain

graph

interpretations

In

this

article

we

study

the

expressiveness

of

the

different

chain

graph

interpretations.

Chain

graphs

is

a

class

of

probabilistic

graphical

models

that

can

contain

two

types

of

edges,

representing

different

types

of

relationships

between

the

variables

in

question.

Chain

graphs

is

also

a

superclass

of

directed

acyclic

graphs,

i.e.

Bayesian

networks,

and

can

thereby

represent

systems

more

accurately

than

this

less

expressive

class

of

models.

Today

there

do

however

exist

several

different

ways

of

interpreting

chain

graphs

and

what

conditional

independences

they

encode,

giving

rise

to

different

so

called

chain

graph

interpretations.

Previous

research

has

approximated

the

number

of

representable

independence

models

for

the

Lauritzen-Wermuth-Frydenberg

and

the

multivariate

regression

chain

graph

interpretations

using

an

MCMC

based

approach.

In

this

article

we

use

a

similar

approach

to

approximate

the

number

of

models

representable

by

the

latest

chain

graph

interpretation

in

research,

the

Andersson-MadiganPerlman

interpretation.

Moreover

we

summarize

and

compare

the

different

chain

graph

interpretations

with

each

other.

Our

results

confirm

previous

results

that

directed

acyclic

graphs

only

can

represent

a

small

fraction

of

the

models

representable

by

chain

graphs,

even

for

a

low

number

of

nodes.

The

results

also

show

that

the

Andersson-Madigan-Perlman

and

multivariate

regression

interpretations

can

represent

about

the

same

amount

of

models

and

twice

the

amount

of

models

compared

to

the

Lauritzen-Wermuth-Frydenberg

interpretation.

However,

at

the

same

time

almost

all

models

representable

by

the

latter

interpretation

can

only

be

represented

by

that

interpretation

while

the

former

two

have

a

large

intersection

in

terms

of

representable

models.

Chain

graphs

(CGs)

are

probabilistic

graphical

models

(PGMs)

that

extend

the

formalism

of

directed

acyclic

graphs

(DAGs)

i.e.

Bayesian

networks,

with

an

additional,

non-directed,

edge.

The

directed

edge

works

similarly

as

in

DAGs

when

it

comes

to

representing

independences

while

the

non-directed

edge

can

be

interpreted

in

different

ways,

giving

rise

to

different

so

called

CG

interpretations.

This

means

that

CGs

can

represent

every

independence

model

representable

by

any

DAG

while

the

opposite

does

not

hold.

The

most

researched

CG

interpretations

are

the

Lauritzen-Wermuth-Frydenberg

(LWF)

interpretation

the

Andersson-Madigan-Perlman

(AMP)

interpretation

and

the

multivariate

regression

(MVR)

interpretation

Each

interpretation

has

its

own

way

of

determining

conditional

independences

in

a

CG

and

it

can

be

noted

that

no

interpretation

subsumes

another

in

terms

of

representable

independence

models

The

question

of

how

much

more

expressive

CGs

are

compared

to

DAGs,

i.e.

how

many

more

independence

models

that

can

be

represented

by

CGs

compared

to

those

representable

by

DAGs,

has

recently

been

studied

for

CGs

of

the

LWF

interpretation

(LWF

CGs)

and

the

MVR

interpretation

(MVR

CGs)

The

results

from

these

studies

show

that

the

ratio

of

the

number

of

DAG

models

(independence

models

representable

by

DAGs)

compared

to

the

number

of

CG

models

(independence

models

representable

by

CGs)

decreases

exponentially

as

the

number

of

nodes

in

the

graphs

increases..

In

this

article

we

carry

out

a

similar

study

for

CGs

of

the

AMP

interpretation

(AMP

CGs)

and

conclude

the

research

of

expressiveness

of

independence

models

for

the

three

CG

interpretations.

This

allows

us

to

summarize

the

method

for

approximating

the

number

of

CG

models

as

well

as

the

results

we

have

seen

for

the

different

CG

interpretations

for

up

to

nodes.

We

also

compare

the

three

CG

interpretations

with

each

other,

as

well

as

to

DAGs,

in

terms

of

representable

independence

models

and

number

of

graphs

per

Markov

equivalence

class.

Finally

we

also

compare

the

three

CG

interpretations

to

their

non-directed

subclasses

which

are

undirected

graphs

(UGs)

i.e.

Markov

networks,

for

the

LWF

and

AMP

CGs

respectively

bidirected

graphs

(BDs)

i.e.

covariance

graphs,

for

MVR

CGs..

Measuring

the

number

of

representable

independence

models

should

in

principle

be

easy

to

do.

We

only

have

to

enumerate

every

possible

CG

and

check

whether

it

is

the

unique

representative

of

the

Markov

equivalence

class,

i.e.,

CG

model,

for

the

CG

interpretation

we

study.

The

problem

here

is

however

that

since

both

the

number

of

CGs

and

CG

models

grows

superexponentially

as

the

number

of

nodes

in

the

graphs

increases,

it

is

infeasible

to

enumerate

all

of

them

for

more

than

five

nodes.

Hence

we

instead

use

a

Markov

chain

Monte

Carlo

(MCMC)

approach

that

allows

us

approximate

the

ratio

of

CG

models

to

DAG

models

for

each

interpretation.

Then,

since

we

know

the

number

of

DAG

models

from

previous

research

we

can

calculate

the

number

of

CG

models

for

the

different

CG

interpretations..

The

MCMC

approach

consists

of

creating

a

Markov

chain

whose

states

are

all

possible

CG

models

for

a

given

number

of

nodes

and

whose

stationary

distribution

is

the

uniform

distribution

over

these

models.

What

CG

models

are

possible

depends

on

what

CG

interpretation

we

study.

By

transitioning

through

this

Markov

chain,

CG

models

can

then

be

sampled

from

approximately

the

uniform

distribution

over

all

CG

models

of

the

CG

interpretation

in

question.

For

each

of

these

CG

models

it

is

then

possible

to

be

check

whether

the

represented

independence

model

also

can

be

represented

as

a

DAG,

and

hence

to

calculate

the

ratio

of

CG

models

to

DAG

models.

The

CG

model

samples

also

allows

us

to

calculate

other

ratios,

such

as

whether

the

independence

model

can

be

represented

by

some

other

CG

interpretation.

Moreover

we

make

the

CG

models

publicly

available

online

and

it

is

our

intent

that

they

can

be

used

for

further

studies

in

the

field.

This

can

for

example

be

when

evaluating

CG

learning

algorithms

to

get

more

accurate

evaluations

than

what

is

achieved

today

when

randomly

generated

CGs

are

used

instead

of

randomly

generated

CG

models

The

rest

of

the

article

is

organised

as

follows.

In

the

next

section

we

discuss

the

theory

of

the

general

MCMC

sampling

algorithm

used

for

all

the

CG

interpretations.

This

is

followed

by

Section

where

we

define

how

this

general

algorithm

is

implemented

for

the

AMP

CG

interpretation.

The

implementations

for

the

LWF

and

MVR

interpretations

can

be

found

in

their

appropriate

articles

In

Section

we

then

present

the

results

we

can

calculate

from

using

the

sampling

algorithm

and

in

Section

we

have

a

short

conclusion.

The

article

also

has

an

Appendix

containing

the

proofs

for

the

theorems

presented

in

Section

as

well

as

a

notation

section

necessary

for

these

proofs.

Note

that

the

article

itself

contains

no

notation

section

since

the

terms

used

are

common

in

the

probabilistic

graphical

model

community.

However,

if

some

term

is

unfamiliar

to

the

reader

all

terms

used

are

properly

defined

in

the

Appendix.

In

this

article

we

have

presented

an

approach

for

sampling

AMP

CG

models

from

the

approximately

uniform

distribution

of

AMP

CG

models

for

up

to

nodes.

This

has

allowed

us

to

finalize

the

research

of

expressibility,

in

terms

of

the

number

of

representable

independence

models,

for

the

different

CG

interpretations

that

exist

in

research

today.

We

have

presented

relevant

ratios,

such

as

the

ratio

of

independence

models

representable

by

CGs

that

also

can

be

represented

by

DAGs,

for

the

different

CG

interpretations

for

up

to

nodes.

Moreover,

we

have

been

able

compare

the

different

CG

interpretations

with

each

other

as

well

as

calculate

how

large

their

intersections

approximately

are

in

terms

of

representable

independence

models.

Finally

we

have

also

been

able

to

determine

how

the

relevant

ratios

and

intersections

changes

as

the

number

of

nodes

in

the

models

increases,

i.e.

whether

they

follow

exponential

equations,

linear

equations

or

remain

constant..

The

results

presented

confirm

previous

results

that

DAGs

only

can

represent

a

fraction

of

the

independence

models

representable

by

CGs

with

more

than

nodes

and

that

the

ratio

decreases

exponentially

as

the

number

of

nodes

increases.

New

results

are

however

that

AMP

CGs

and

MVR

CGs

can

represent

approximately

the

same

number

of

independence

models

while

LWF

CGs

can

represent

approximately

less

models

than

the

other

two

CG

interpreta-

Moreover,

with

the

presented

sampling

method

for

AMP

CG

models

sampling

methods

for

CG

models

of

all

CG

interpretations

are

now

available

This

opens

up

for

MCMC

based

structure

learning

algorithms

for

the

different

CG

interpretations

that

traverses

the

space

of

CG

models

instead

of

the

space

of

CGs.

More

importantly

it

also

opens

up

for

further

studies

on

what

independence

models

and

systems

the

different

CG

interpretations

can

represent,

which

is

an

important

question

where

the

answer

is

still

unclear.

Using

the

MCMC

approach

described

in

Section

we

were

able

to

sample

CG

models

uniformly

for

each

CG

interpretation

for

up

to

nodes.

For

each

number

of

nodes

and

CG

interpretation

models

were

sampled

with

transitions

between

each

sample.

The

implementation

was

carried

out

in

C++,

run

on

an

Intel

Core

processor

and

it

took

approximately

one

week

to

complete

the

sampling

of

all

sample

sets

per

CG

interpretation.

Moreover

we

also

enumerated

all

CG

models

for

up

to

five

nodes,

for

each

interpretation,

to

allow

a

comparison

between

the

approximated

and

exact

values.

The

sampled

graphs

and

code

are

available

for

public

access

at

http

From

the

sampled

graphs

we

could

then

calculate

the

ratio

of

CG

models

whose

independence

models

could

be

represented

by

DAGs,

UGs

and

BGs

as

well

as

some

other

CG

interpretation

for

the

LWF,

AMP

and

MVR

CG

interpretation.

The

results

are

shown

in

Tables

and

and

are

discussed

in

Sections

and

Before

we

look

into

these

results

we

will

however

shortly

discuss

the

validity

of

the

approximated

uniform

distributions.

We

have

in

our

work

not

been

able

to

calculate

the

variance

in

the

results

due

to

the

sampling

process

since

this

would

take

several

months

in

calculation

time.

However,

there

are

several

indicators

suggesting

that

the

models

are

sampled

from

almost

uniform

distributions.

First,

that

the

calculated

ratios,

as

well

as

the

number

of

the

different

types

of

edges,

have

clear

smooth

trends

as

the

number

of

nodes

increases.

Secondly

we

have

also

seen,

although

no

formal

variance

have

been

calculated,

that

different

runs

of

the

MCMC

sampling

approach

gives

no

significant

difference

in

the

results

for

up

to

nodes.

For

more

than

nodes

we

could

however

see

that

this

was

no

longer

true

and

the

ratios

could

differ

significantly

for

a

certain

number

of

nodes

when

the

algorithm

was

run

multiple

times.

Hence

this

indicates

that

the

MCMC

sampling

approach

does

not

sample

from

the

uniform

distribution

if

it

is

run

with

more

than

nodes

and

the

described

parameters,

i.e.

the

number

of

sampled

models

and

transitions

between

each

sample.

O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Image segmentation algorithm by piecewise smooth approximation We propose a novel image segmentation algorithm using piecewise smooth (PS) approximation to image. The proposed algorithm is inspired by four well-known active contour models, i.e., Chan and Vese' piecewise constant (PC)/smooth models, the region-scalable fitting model, and the local image fitting model. The four models share the same algorithm structure to find a PC/smooth approximation to the original image; the main difference is how to define the energy functional to be minimized and the PC/smooth function. In this article, pursuing the same idea we introduce different energy functional and PS function to search for the optimal PS approximation of the original image. The initial function with our model can be chosen as a constant function, which implies that the proposed algorithm is robust to initialization or even free of manual initialization. Experiments show that the proposed algorithm is very appropriate for a wider range of images, including images with intensity inhomogeneity and infrared ship images with low contrast and complex background. Image segmentation is a popular problem in image processing and computer vision, which has been studied extensively in past decades. The segmentation goal is to separate image domain into a collection of distinct regions, upon which other high-level tasks such as objects recognition and tracking can be further performed. Due to the presence of noise, complex background, low intensity contrast with weak edges, and intensity inhomogeneity [1] , image segmentation is still a difficult problem in practical applications, especially for traditional segmentation methods [2] [3] [4] [5] [6] [7] [8] . Traditional methods, like Canny edge detection [7] , are simple and fast, but they always need further edge linking operation to produce continuous object boundaries [9] . To address these issues, more recent methods including implicit active contours [9] [10] [11] [12] [13] [14] [15] [16] [17] have been developed for image segmentation.. Implicit active contours are active contour models implemented via the level set method [18] . One of the remarkable advantages of active contour models is that it can provide smooth and closed contours, which is generally impossible in traditional segmentation methods. Existing implicit models can roughly be categorized into two classes: edge-based models [10, 11, 13] and region-based models [12, [14] [15] [16] . Edge-based models use local edge information (image gradient) to perform contour extraction, which are usually to noise and weak edges. Region-based models utilize the global and/or local image statistics inside and outside the active contour (evolving curve) to find a partition of image domain. They generally have better performance in the presence of weak or discontinuous boundaries and less sensitive to initialization.. A major category of the region-based level set methods is proposed to minimize the well-known Mumford and Shah (MS) functional [19] . Due to the difficulty of directly minimizing the MS functional, different approximation methods have been proposed to allow more efficient energy minimization. For example, the piecewise constant (PC) models [12, 20] approximate image domain by a set of homogenous regions, but which is not true for images with intensity inhomogeneity. In [21, 22] , more advanced piecewise smooth (PS) models have been proposed to improve the PC model performance in terms of intensity inhomogeneity. However, due to rather complicated algorithms, these models are usually computationally expensive. As indicated in [23] , the models using global image statistics usually have difficulty to segment many real-world images with intensity inhomogeneity.. To address this issue, recently localized region-based models [14, 16, [23] [24] [25] have been proposed. For example, Li et al. [14] proposed a region-scalable fitting (RSF) model (originally termed as local binary fitting (LBF) model [26] ), which draws upon spatially varying local region information and thus is able to deal with intensity inhomogeneity. The RSF model has better performance than PC and PS models in segmentation of accuracy and computational efficiency. However, since there are totally four convolutions to be computed at each iteration in the implementation, the RSF model is still computationally expensive. For more efficient segmentation, Zhang et al. [16] proposed a novel active contour model driven by local image fitting (LIF) energy. The LIF energy is constructed by the local image information, which can be viewed as a constraint of the differences between the fitting image and the original image. The complexity analysis and experimental results show that this model is more efficient than the LBF model, while yielding similar results.. These localized methods [14, 16, 25, 26] are, however, to some extent sensitive to contour initialization. Since the segmentation results typically depend on the selection of initial contours, these methods need user intervention to define the initial contours professionally. This means that they may be fraught with the problems of how and where to define the initial contours. The multiple-seed initialization used by Vese and Chan [21] also cannot always produce better results than the single-seed initialization [27] . Therefore, it is still a great challenge to find an efficient way to tackle the initialization problem.. In this article, inspired by the above-mentioned four models [Chan-Vese (CV), PS, RSF, and LIF models], we propose a novel image segmentation algorithm without initial contours using PS approximation. The four models share the same algorithm structure to find a PC/smooth approximation to the original image; the main difference is how to define the energy functional to be minimized and the PC/smooth function. Pursuing the same idea, in this study, we introduce different energy functional and PS functions to search for the optimal PS approximation of the original image. In the proposed algorithm, the initial function can be chosen as a constant function, which implies that the proposed algorithm allows for robustness to initialization or even free of manual initialization. Experiments show that the proposed algorithm works well for images with intensity inhomogeneity. Moreover, it is very appropriate for infrared images with low contrast and complex background.. The remainder of this article is organized as follows. In "Related studies" section, we briefly review several classical region-based models and indicate their limitations. The proposed algorithm is introduced in "The LIF model results in a local fitted image" section. The proposed model presents experimental results using a set of synthetic and real images. This article is summarized in "Experimental results" section. In this article, we propose a novel image segmentation algorithm by PS approximation. The proposed model, which pursues the basic ideas behind four well-known region-based models, introduces different energy functional and PS functions to find the optimal PS approximation of the original image. The initial function can be chosen as a constant function, which completely eliminates the need of initial contours. The proposed model is very appropriate for a wider range of images, including images with intensity inhomogeneity and infrared images with low contrast and complex background.

B I O O B I I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Communication in distributed tracking systems: an ontology-based approach to improve cooperation an ontology-based approach to improve cooperation Abstract: Current Computer Vision systems are expected to allow for the management of data acquired by physically distributed cameras. This is especially the case for modern surveillance systems, which require communication between components and a combination of their outputs in order to obtain a complete view of the scene. Information fusion techniques have been successfully applied in this area, but several problems remain unsolved. One of them is the increasing need for coordination and cooperation between independent and heterogeneous cameras. A solution to achieve an understanding between them is to use a common and well-defined message content vocabulary. In this research work, we present a formal ontology aimed at the symbolic representation of visual data, mainly detected tracks corresponding to real-world moving objects. Such an ontological representation provides support for spontaneous communication and component interoperability, increases system scalability and facilitates the development of high-level fusion procedures. The ontology is used by the agents of Cooperative Surveillance MultiAgent System, our multi-agent framework for multi-camera surveillance systems. In recent years, the interest in Computer Vision systems has reached new heights. In the simplest case, vision systems have a single sensor, which provides a sequence of frames to the image processing algorithms. These algorithms, mainly based on statistical prediction and inference methods, aim to automatically detect and trace the entities in the observation area, and to recognize and predict the actions that they are performing to act consequently. In the most complex case, systems encompass several distributed sensors, which additionally requires to fuse information acquired at different locations of the sensor network. Among Computer Vision tasks, tracking -the estimation of the number of objects in a scene, together with their instantaneous locations, kinematic states and any other characteristics required -is one of the most studied, and a first step before performing more intricate video analysis procedures (Yilmaz et al., 2006) .. Surveillance is a typical application domain where accurate tracking is required. Recently, the decreasing price of video camera hardware and the development of network technologies have given rise to the development of thirdgeneration surveillance systems (Regazzoni et al., 2001; Valera & Velastin, 2005) . This term designates systems that resemble the nature of the human intelligent process of surveillance (which activates certain cognitive abilities) and satisfy the requirements of modern applications (large number of cameras, geographical spread of resources, many monitoring points, etc.) Two difficulties can be identified in such distributed systems. Firstly, it is necessary to implement suitable procedures to integrate data generated at each location, in order to obtain a high-level interpretation of the whole scene. Secondly, the scalability of the system must be guaranteed, which can be difficult when new heterogeneous cameras are incorporated to build a large and scattered circuit.. In previous researches, we presented the Cooperative Surveillance Multi-Agent System (CS-MAS), an agent-based model and platform to support the development of third-generation surveillance systems (Patricio et al., 2007; Castanedo et al., 2010) . CS-MAS proposes the formation of smart camera coalitions; i.e., groups of sensors able to carry out complex processing tasks and cooperate with their neighbours by means of sophisticated interaction protocols. The combination of information acquired by geographically distributed cameras in CS-MAS improves tracking results in surveillance applications, since multiple fields of view are considered. Additionally, distribution increases system robustness and fault tolerance, since the same information may be captured and replicated at different points of the network.. Nevertheless, CS-MAS does not completely solve two difficulties. First, it is necessary to implement suitable procedures to represent data generated at each location to obtain a high-level and global interpretation of the scene. Second, the scalability of the system must be guaranteed, which can be difficult when new heterogeneous cameras are incorporated to build a large and scattered circuit.. To overcome these problems, we proposed a semantic model to represent the visual information shared in distributed artificial vision systems (Go´mez-Romero et al., 2009c ). In the current paper, we extend that preliminar work to provide a detailed description of the ontology to represent the tracking information managed by agents and interchanged in CS-MAS. We also explain how the ontological representation is integrated within the CS-MAS architecture and how concepts and relations are used to express and communicate agents' beliefs. As a major contribution, we show that the ontology behaves as an agreed vocabulary that allows tracking data to be represented in a symbolic, common and understandable way. Thus, information exchange is decoupled from information processing, which facilitates the incorporation of extended, heterogeneous and=or third-party components into the vision system. The advantages of this approach can be exploited in scenarios requiring complex agent interactions, such as camera handover. Interestingly enough, further functionalities to reason with high-level abstractions can be implemented on top of the ontological representation.. Ontologies are a state-of-the-art knowledge representation and reasoning formalism. They have proved to be valid in several scenarios that require interoperation between heterogeneous entities, e.g. the Semantic Web (Horrocks, 2008) . Ontologies have strong underpinnings in Description Logics (DLs) (Baader et al., 2003 (Baader et al., , 2008 ; in fact, ontology languages are usually (equivalent to) a decidable DL, as in the case of the standard OWL (Horrocks & Patel-Schneider, 2004 ) and its successor OWL 2 (Hitzler et al., 2008 ) -used in this work. Reasoning with ontologies is an automatic procedure that infers new axioms that have not been explicitly included in the knowledge base but are logical consequences of the represented axioms. An advantage of ontologies over classical multi-agent content languages, such as FIPA Semantic Language, is that the latter are undecidable in their general form -i.e., it is not guaranteed that all the inferences are computable in a finite time, whereas the former are decidable and are supported by current tools (APIs, reasoners, etc.) Hence, ontologies have been proposed to be the knowledge representation of agent systems (Hendler, 2001; Schiemann & Schreiber, 2006; Erdur & Seylan, 2008) .. The rest of this paper is organized as follows: next, we describe the context of our research and review related work on the use of ontologies in (distributed) Computer Vision systems. In section 3, we explain the role of the ontological representation inside the CS-MAS framework. In section 4, we describe the formulation of the ontology, whereas in section 5, we illustrate its use along with CS-MAS in a surveillance scenario, including instance creation, message-passing and support for high-level information fusion. Finally, the paper concludes with a discussion of our proposal and plans for future research work. In this paper, we have proposed an ontology to represent tracking data in Computer Vision systems. The ontology is used in the CS-MAS framework to represent agents' beliefs and to communicate them by means of FIPA messages with contents described semantically. We have presented the structure of the ontology -which is publicly available -its use within the CS-MAS framework, and an example of communication of tracking data.. Endorsing data with semantics has several advantages. Communication is easily achieved among agents, since the contents of messages are expressed in the same well-defined language. Systems are more flexible, extensible and independent of the implementation technologies. The ontology also facilitates the development of extended functionalities for the vision system built on top of it, as well as the publication of tracking data.. We plan to continue this research work in various directions. First, we are applying the framework to problems more complex than the test case presented in this paper. This will allow us to obtain more precise measures of the accuracy and the performance of the system. The use of the ontology in different domains may imply further refinements or simplifications, in order to achieve a proper trade-off between the information shared by the agents and the computational resources needed to manage it -which are usually high in the case of large ontologies.. Additionally, we are studying how to represent more information, and specifically, uncertain and=or imprecise tracking data and spatio-temporal relations (e.g., close, far, before, after). It must be taken into account that standard ontologies do not provide support for this kind of knowledge; therefore, extended formalisms, such as fuzzy and probabilistic, would be needed. Closely related is the problem of conflict resolution, which occurs when an agent receives information that is not coherent with its own perceptions. Regarding knowledge representation, the uncertainty resulting from merging incoherent or contradictory pieces of knowledge should also be properly represented (in the agents' belief bases) and transmitted (through FIPA messages) by using the ontology.. Other interesting contribution would be the implementation of software tools for visualizing and exporting the data obtained by tracking algorithms and annotated with the ontology. Such programs would be useful to review and test the accuracy of tracking processes with respect to a ground truth, both for expert and for non-expert users.. Moreover, as introduced in section 5.3, we are using the ontology as a basis for further highlevel processing of visual data. We are testing an extension of the tracking system that uses context-knowledge to infer additional scene information at different fusion levels, including situation assessment and feedback. This context layer is built on top of the tracking data ontology, which is the first step in the evolution from numeric to symbolic information. An expected additional result in that regard is the exploitation of feedback procedures to enhance lowlevel tracking by reasoning from high-level scene interpretation. Data integration and interpretation problems in Computer Vision have been faced by applying data and information fusion techniques. Fusion techniques combine data from multiple sensors and related information to achieve more specific inferences than could be achieved by using a single, independent sensor (Hall & Llinas, 2009 ). Data fusion systems are usually organized by following the guidelines of the Joint Directors of Laboratories (JDL) model (Steinberg & Bowman, 2009) . JDL classifies fusion processes according to the abstraction and the refinement of the entities involved. The canonical JDL model establishes five operational levels in the transformation of input signals to decision-ready knowledge (Llinas et al., 2004; , namely: signal feature assessment (L0); entity assessment (L1); situation assessment (L2); impact assessment (L3) and process assessment (L4). Tracking is considered an L1 task, since it aims at estimating the properties of isolated objects from pre-processed sensor signals (L0), instead of the relations between them. High-level information fusion, corresponding to the levels L2 and L3 of the JDL model, aims at obtaining a description of the relations between the objects in the perceived scenario. These relations are usually expressed with interpretable symbolic terms (e.g., actions, intentions, threats), instead of the usual numerical measures (e.g., density functions, movement vectors) calculated in L1. L4 tasks are aimed at planning and performing procedures to improve the whole fusion process, from low-level data acquisition to high-level situation assessment.. Conceptual models to acquire, represent and exploit formal knowledge in fusion have been extensively proposed (Nowak, 2003) . Ontologies have gained popularity in the last few years and are being applied to solve fusion problems in Computer Vision systems. Recent researches can be classified according to the levels defined by the JDL model.. At image-data level (i.e. JDL level 0), one of the most important contributions is Core Ontology for MultiMedia (COMM), an OWL ontology to encode MPEG-7 data (Arndt et al., 2008) . COMM does not represent high-level entities of the scene, such as tracks, objects or events. Instead, it identifies the components of an MPEG-7 video sequence in order to link them to (Semantic) Web resources. Similarly, the Media Annotations Working Group of the W3C is working in an OWL-based language for adding metadata to Web images and videos (Lee et al., 2009) . These approaches do not specifically aim at representing scene data, but the structure of the acquired video sequence, focusing on the normalization of the several existing video formats.. More related to our approach are such proposals targeted at modelling video content at the object level (i.e. JDL L1). For example, Franc¸ois et al. (2005) have created a framework for video event representation and annotation. In this framework, Video Event Representation Language (VERL) defines the concepts to describe processes (entities, events, time, composition operations, etc.), and Video Event Markup Language (VEML) is an XML-based vocabulary to markup video sequences (scenes, samples, streams, etc.). VEML 2.0 has been expressed in OWL, but only partially because it imports VERL elements that need a more expressive language. Moreover, the limitation in the number of entities represented in VEML 2.0 reduces its usefulness, as it is discussed by Westermann and Jain (2007) , who present a framework that supports representation of uncertain knowledge. An approach that stands halfway between data and object level is due to Kokar and Wang (2002) . In this research work, the data managed by the tracking algorithm are represented symbolically in a similar fashion as we do, but they do not take into account the particularities of using the vocabulary for information transmission. Similarly, ) have presented a first approach to the development of a tracking data ontology, a work that is still in progress.. At a more abstract level (i.e. JDL L2 and L3), scene interpretation issues are being dealt with ontologies as well. In , the authors discuss the use of OWL ontologies and SWRL rules to define and recognize scene events in Ambient Intelligence domains. In Neumann and Mo¨ller (2008) , an ad-hoc proposal for scene interpretation based on DLs is presented. The paper shows how the reasoning features of the RACER reasoning engine provide functionalities that support scene recognition. The approach is hardly generalizable, but illustrates the expressivity of DLs for such tasks and the existence of appropriate tools. The problem of representing high-level semantics of situations with a computable formalism is also faced in Kokar et al. (2009) . The authors present an OWL ontology (Situation Theory Ontology, STO) that encodes Barwise's situation semantics. Both research works tackle the problem of transforming numeric data into symbolic objects, because scene interpretation must eventually take raw video data as an input. Our representation has been purposely designed to solve this problem and could be used in combination with these high-level approaches.. In this regard, in previous research works, we have developed a framework for contextual scene recognition in surveillance applications and tracking enhancement (Go´mez-Romero et al., 2009a) . More abstract objects (e.g. people, moving items) with special features or behaviours have been defined by relying on the ontology hereby presented (Go´mez-Romero et al., 2009b) , in addition to scene interpretation rules (Go´mez-Romero et al., 2009d) . Thus, the ontology presented in the current paper is complementary to these approaches. While they are especially focused on high-level interpretation in one-camera configurations, the ontology described in this work may be used to bridge the gap between real-world physical images and high-level symbolic interpretation, which is known as the grounding problem (Pinz et al., 2008) .

O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Rational selection of experimental readout and intervention sites for reducing uncertainties in computational model predictions Background: Understanding the dynamics of biological processes can substantially be supported by computational models in the form of nonlinear ordinary differential equations (ODE). Typically, this model class contains many unknown parameters, which are estimated from inadequate and noisy data. Depending on the ODE structure, predictions based on unmeasured states and associated parameters are highly uncertain, even undetermined. For given data, profile likelihood analysis has been proven to be one of the most practically relevant approaches for analyzing the identifiability of an ODE structure, and thus model predictions. In case of highly uncertain or non-identifiable parameters, rational experimental design based on various approaches has shown to significantly reduce parameter uncertainties with minimal amount of effort. Results: In this work we illustrate how to use profile likelihood samples for quantifying the individual contribution of parameter uncertainty to prediction uncertainty. For the uncertainty quantification we introduce the profile likelihood sensitivity (PLS) index. Additionally, for the case of several uncertain parameters, we introduce the PLS entropy to quantify individual contributions to the overall prediction uncertainty. We show how to use these two criteria as an experimental design objective for selecting new, informative readouts in combination with intervention site identification. The characteristics of the proposed multi-criterion objective are illustrated with an in silico example. We further illustrate how an existing practically non-identifiable model for the chlorophyll fluorescence induction in a photosynthetic organism, D. salina, can be rendered identifiable by additional experiments with new readouts. Conclusions: Having data and profile likelihood samples at hand, the here proposed uncertainty quantification based on prediction samples from the profile likelihood provides a simple way for determining individual contributions of parameter uncertainties to uncertainties in model predictions. The uncertainty quantification of specific model predictions allows identifying regions, where model predictions have to be considered with care. Such uncertain regions can be used for a rational experimental design to render initially highly uncertain model predictions into certainty. Finally, our uncertainty quantification directly accounts for parameter interdependencies and parameter sensitivities of the specific prediction. In this work we illustrate how to analyze and quantify uncertainty propagation in ODE models based on profile likelihood samples. We introduce the profile likelihood sensitivity index, which reflects the individual contribution of an uncertain model parameter to a model prediction. In the case of several parameters, parameter interdependencies are -by definition of the profile likelihood -account for, and the sum of profile likelihood sensitivity indices can be used to quantify the overall effect. However, individual parameter uncertainty contributions are not clear. Here we propose to use Shannon's entropy on the individual profile likelihood sensitivity indices as an additional measure. The PLS entropy describes the amount of uncertainty contributed by each uncertain parameter to the overall PLS index. In this way, PLS entropy can be used to look for homogenous uncertainty contribution. We further describe in a general way, how profile likelihood sensitivity index and entropy can be used to identify experimental regions, where one has to collect data in order to reduce prediction uncertainties. Such an approach is especially valuable in large biochemical networks, where intuitive analysis is hampered by the complexity of the system. Additionally, PLS index and entropy provide information on prediction domains, where one has to consider model-based predictions with care.. We applied the concept of PLS indices and entropies to an intuitive in silico example to illustrate how one can rationally select additional readout and/or intervention sites in order to reduce prediction uncertainties. Finally, using the chlorophyll fluorescence induction of D. salina as a true life case, we illustrate how an initially nonidentifiable model can potentially be rendered identifiable by selection additional readout signals. Advances in technology and biotechnology in particular allow us to look inside biological cells and observe dynamic processes occurring at the molecular level. Still, many of these processes can only partially be observed in experiments hampering the experimental exploration of interaction mechanisms. Here, a computational abstraction of the dynamic biochemical process in the form of an ordinary differential equation system (ODE) with unknown parameters can provide answers to the dynamics of unmeasured states that in turn give information on interaction mechanisms. Furthermore, model-based predictions and optimizations are possible. Such a modelbased approach relies on the adequacy of the model, i.e. properly identified structure and parameter values. Often, the amount and quality of the experimental data is insufficient for complete model identification resulting into badly constrained or even non-identiafiable parameters, and thus uncertain dynamic model predictions. Although model predictions on observed data can be extremely certain and useful even in the presence of unidentifiable parameters (e.g. [1, 2] ), model predictions on unobserved (internal) model states that are related to these unidentifiable parameters can be highly uncertain. If model based predictions on internal states are of interest, experimental design can be used to rationally design new experiments with optimized content of information with respect to a specific model prediction. Several excellent publications have appeared over the last years, which focus on identification of computational models for biochemical systems by applying a variety of methodological optimal experimental design approaches, e.g. [3] [4] [5] [6] [7] [8] .. Profile likelihood estimation has been proven to be a valuable tool for parameter identifiability analysis [9] . Parameter identifiability analysis investigates whether a model parameter can be uniquely determined for the given data and input-output setting. For non-identifiable parameters, there exists an uncountable set of parameters, which yield the same model input-output behavior. As a result, predictions on internal states -states that are not directly observed in experiments -become highly uncertain [9] . An identifiability analysis can hint at a necessary re-design of an experimental input-output setup, a model re-parameterization or reduction to resolve nonidentifiabilities [4] .. In the following we briefly describe ODE modeling of biochemical processes, parameter estimation in ODE models and parameter identifiability analysis based on the profile likelihood. We then show how to use profile likelihood samples for quantifying individual contributions of parameter uncertainties to uncertainties in the model predictions by introducing the profile likelihood sensitivity index and profile likelihood sensitivity entropy. We further show, how to use this uncertainty quantification for experimental design by formulating the respective multi-criterion objective. The uncertainty quantification in combination with experimental design is illustrated with (i) an intuitive in silico example and (ii) a dynamic chlorophyll fluorescence induction model of the photosynthetic organism D. salina.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O B I O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O B I I O O O O O O O


In defense of the Desiderata A 1998 paper that delineated desirable characteristics, or desiderata for controlled medical terminologies attempted to summarize emerging consensus regarding structural issues of such terminologies. Among the Desiderata was a call for terminologies to be ''concept oriented.'' Since then, research has trended toward the extension of terminologies into ontologies. A paper by Smith, entitled ''From Concepts to Clinical Reality: An Essay on the Benchmarking of Biomedical Terminologies'' urges a realist approach that seeks terminologies composed of universals, rather than concepts. The current paper addresses issues raised by Smith and attempts to extend the Desiderata, not away from concepts, but towards recognition that concepts and universals must both be embraced and can coexist peaceably in controlled terminologies. To that end, additional Desiderata are defined that deal with the purpose, rather than the structure, of controlled medical terminologies. The original Desiderata paper discussed the entities represented by controlled terminologies without reference to ontologies, but it nevertheless reflected ontological principles. While it referred to the terminologic entities as ''concepts,'' it was describing desired characteristics of universals as well. As long as we consider that the purpose of terminologies is to support the recording and use of actual data, rather than primarily as a pure knowledge base of what is known in biomedicine, I believe that concepts and universals can coexist and commingle in controlled terminologies, to the advantage of those who seek to improve patient care through symbolic representation of patient information.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O