Finding Y-relevant part of X by use of PCR and PLSR model reduction methods The paper is considering the following question: Using principal component regression (PCR) or partial least squares regression (PLSR), how much data can be removed from X while retaining the original ability to predict Y? Two model reduction methods using similarity transformations are discussed, one giving projections of original loadings onto the column space of the …tted response matrixŶ (essentially the orthogonal signal correction (OSC) methods), and one giving projections of original scores onto the column space of the coe¢ cient matrixB (essentially the net analyte signal (NAS) methods). The loading projection method gives model residuals that are orthogonal to Y andŶ, which is valuable in certain applications. The score projection method, on the other hand, gives model residuals that are orthogonal toB, which is essential in other applications. It is shown that the reduced matrix X S Y from the score projection method is a subset of the reduced matrix X L Y from the loading projection method. It therefore has the smallest Frobenius norm, and thus the smallest total column variance, assuming centered data. Principal component regression and partial least squares regression (PCR and PLSR) are well known methods for solution of ill-posed multivariate regression problems. Both methods make use of factorizations of the regressor and response data matrices into X = P A i=1 t i p T i + E =X + E and Y = P A i=1 t i q T i + F =Ŷ + F, where the number of components A with score vectors t i and loading vectors p i and q i is determined through either cross-validation or test set validation, and where E and F are unmodeled residuals. The number of components in such latent variables (LV) models are often higher than strictly necessary, and methods for model reduction are therefore of interest. One reason for this is that interpretations of score and loading plots are easier with fewer components, as discussed in references given below, and in an industrial data example in Subsection 4. The present paper will, however, primarily focus on a di¤erent aspect, as illustrated in As a measure of the size of X Y we may use the Frobenius norm, de…ned in Section 3 below. This will also be a measure of the total column variance of X Y , assuming centered data. One method for model reduction is to identify and remove the Y-orthogonal part of X, which is the aim of the preprocessing orthogonal signal correction (OSC) methods [1, 2] , e.g. the OPLS algorithm [3] . The Y-orthogonal part of X can also be found by a post-processing similarity transformation of the original PCA/PCR or PLS factorization [4] . The starting point in reference [4] was the non-orthogonalized PLS factorization [5] , where in the single response case and as illustrated in Fig. 2 , all score vectors except the …rst one are orthogonal to the …tted response vectorŷ. Hence, the similarity transformation only has to split t 1 into one component t ST 1 in the direction ofŷ and one component orthogonal toŷ, while the score vectors in T 2:A (columns 2 to A of T) should be left as they are. It was also shown in Reference [4] that the results of this within a second similarity transformation are identical with the results from a slightly modi…ed version of the OPLS algorithm (OPLS with non-orthogonalized PLS). For the orthogonalized PLS factorization [5] , the situation is di¤erent. As illustrated in Fig. 3 , all the orthogonal score vectors must here be split into components in the direction of and orthogonal toŷ, but that can also be done with a similarity transformation (see Section 2). As shown in Section 2 the post-processing similarity transformation method illustrated in Figures 2 and 3 can be extended to cover also multi-response cases, with a response matrix Y and a …tted matrixŶ. The common e¤ect of all these similarity transformations is that the original loadings in the space spanned by the score vectors are projected onto the column space ofŶ. We will therefore refer to these methods as loading projection methods (although an alternative reference could have been score vector projections).. An alternative reduction method is obviously to project the original scores onto the column space of the coe¢ cient matrixB, and by doing so we can identify and removeB-orthogonal parts of X T . Such a projection was suggested already in Reference [6] , and it has been used in de…nitions of net analytic signal (NAS) [7, 8, 9 ]. An example related to this is the 2PLS algorithm presented in Reference [10] and intended for process monitoring applications, where the projection subplane includesb, and a more general treatment is given in Reference [11] . As illustrated in Fig. 4 , all loading vectors must here be split into components in the direction of and orthogonal toB, which as shown in Section 2 can also be done with a similarity transformation. We will refer to this type of reduction methods as score projection methods (although an alternative reference could here have been loading vector projections). Figure 4 . Loading vectors in relation toB for a general LV factorization of X. Here, P 2:A stands for columns 2 to A of the loading matrix P.. The fact that model reduction can be obtained through either loading or score projections onto reduced subspaces (removing Y-andŶ-orthogonal parts from X orB-orthogonal parts from X T ), raises interesting questions:. Will the two approaches give identical results? As shown in Section 2, the answer is no.. Which method will give the smallest Y-relevant part of X, in some reasonable sense? As shown in Section 3, X S Y from the score projection method is a subset of X L Y from the loading projection method, and it therefore has the smallest Frobenius norm, and thus the smallest total row and column variance. In that respect it is a better method for this purpose than use of OSC methods.. Does the reduction result in a standalone reduced LV model? As shown in Section 3, the answer is yes for the score projection method, and no for the loading projection method.. 2 Model reduction by similarity transformations In order to …nd the smallest part X S Y of X that can be used for explanation of Y, one should remove all information in X T orthogonal toB using score projections (essentially as in the NAS methods). The result The score projection method removes all Y-orthogonal information from the modeled part of X, just as the loading projection method does (this is the main objective of the OSC/OPLS methods). But in addition it removes some other information that is not necessary for prediction of Y.. The reduced score projection model is all that is needed for …ndingB and thus for prediction of a new response y new from new regressor data x new , and it may therefore be used as a standalone model.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Local Hoare reasoning about DOM The W3C Document Object Model (DOM) specifies an XML update library. DOM is written in English, and is therefore not compositional and not complete. We provide a first step towards a compositional specification of DOM. Unlike DOM, we are able to work with a minimal set of commands and obtain a complete reasoning for straight-line code. Our work transfers O'Hearn, Reynolds and Yang's local Hoare reasoning for analysing heaps to XML, viewing XML as an in-place memory store as does DOM. In particular, we apply recent work by Calcagno, Gardner and Zarfaty on local Hoare reasoning about simple tree update to this real-world DOM application. Our reasoning not only formally specifies a significant subset of DOM Core Level 1, but can also be used to verify, for example, invariant properties of simple Javascript programs. The Document Object Model (DOM) [12] specifies an XML update library, and is maintained by the World Wide Web Consortium (W3C). Its purpose is to be: a platform-and language-neutral interface that will allow programs and scripts to dynamically access and update the content, structure and style of documents.. A DOM implementation exists in most popular high-level languages, and is used in many applications for accessing and updating XML. For example, consider a webpage with a button labelled 'today's weather'; click on the button and Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. embedded Javascript (using an implementation of DOM) puts 'today's weather' in the tree. Using Context Logic, we have developed local Hoare reasoning about Minimal DOM. Our reasoning is compositional and complete for straight-line code. This means that we can focus on a minimal set of DOM commands and prove invariant properties about simple programs. We made the deliberate choice to work with the DOM tree structure and text nodes, rather than the full DOM structure which also consists of Attributes, DocumentFragments, etc. We took the view that it was important to understand the reasoning about the fundamental tree structure first, especially since DOM treats these other structures as nodes with similar properties. We will extend our reasoning to DOM Core Level 1 in future. We conjecture that there will be little additional conceptual reasoning to this extension. We have however already found holes in the DOM specification: for example, DOM does not specify how the normalize method of the Element class interacts with pointers in the store.. A future goal is to develop a prototype verification tool for reasoning first about Minimal DOM, and then about DOM Core Level 1. Calcagno, Dinsdale-Young and Gardner have recently shown that model-checking and validity are decidable for quantifier-free Context Logic applied to simple trees. We believe this result will extend to Context Logic for the DOM tree structure presented here. With quantifiers however, Context Logic applied both to simple trees and to the DOM tree structure becomes undecidable, just as for Separation Logic and Ambient Logic. We will investigate tractable fragments of Context Logic, inspired by the verification tool Smallfoot based on a fragment of Separation Logic. In particular, the last example in Section 5.3, which verifies that an XML address book schema is an invariant of a simple program which moves a person to a new address, is particularly enticing. An ambitious challenge is to provide a 'one-click' tool that checks if, for example, embedded Javascript in a web page can ever violate the schema assertions on that web page.

B I I O B O O O O O B O O B O O B O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O B O B O O O O O O O B O O O O O O O O O O O O B I I O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B I O O O B I I O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I O O B O O O O O O O B I O O O O O O O O B O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Quality adaptive least squares trained filters for video compression artifacts removal using a no-reference block visibility metric Compression artifacts removal is a challenging problem because videos can be compressed at different qualities. In this paper, a least squares approach that is self-adaptive to the visual quality of the input sequence is proposed. For compression artifacts, the visual quality of an image is measured by a no-reference block visibility metric. According to the blockiness visibility of an input image, an appropriate set of filter coefficients that are trained beforehand is selected for optimally removing coding artifacts and reconstructing object details. The performance of the proposed algorithm is evaluated on a variety of sequences compressed at different qualities in comparison to several other deblocking techniques. The proposed method outperforms the others significantly both objectively and subjectively. Due to the bandwidth limit of the broadcasting channels and the capacity limit of the storage media, video materials are always compressed with various compression standards, such as MPEG-2 and MPEG-4. These block transform based codecs divide the image or video frame into non-overlapping blocks (usually with the size of 8 x 8 pixels), and apply discrete cosine transform (DCT) on them. The DCT coefficients of neighboring blocks are thus quantized independently. At high or medium compression rates, the coarse quantization will result in various noticeable coding artifacts, such as blocking, ringing and mosquito artifacts. Coding artifact reduction techniques are used to remove various artifacts. Among the coding artifacts, blockiness which appears as discontinuities along block boundaries is the most annoying.. Therefore, an in-loop deblocking filter is specified for H.264/AVC to reduce blocking artifacts using coding parameters inside the encoder. However, JPEG compressed images and MPEG-2 compressed videos will remain ubiquitous, which makes post-processing aimed at the elimination of coding artifacts still a critical and indispensable solution.. Most coding artifact reduction techniques based on post-processing, e.g. [1] [2] [3] [4] [5] [6] , are designed according to heuristic tuning and testing, which takes a lot of time and is not always effective.. Recently, classification-based least squares trained filters (TF), initially designed for image resolution upscaling [7] , have been proposed for optimally removing digital coding artifacts [8, 9] . The momentary filter coefficients, during artifact reduction, depend on the local content of the image, which can be classified into classes based on the image characteristics in the filter aperture. To obtain the optimized filter coefficients, a training process should be performed in advance. The training process employs the combination of original images and the degraded versions of those original images as the training material and uses the Least Squares (LS) criterion to get the optimal coefficients, which is computationally intensive due to the large number of classes. Fortunately, the training process only needs to be performed off-line and once. The method introduced in [8] produces promising results, when the quality of the test sequence is similar to that of the source sequences used during training. It is because a fixed level of compression is adopted for degrading the original images. In this paper, we propose to train the algorithm on a range of compression levels and to select the most suitable set of filter coefficients for the test sequence. To do that, a quality (or blockiness) metric is required to indicate the quality level of the test sequence.. The main contribution of the paper is the use of the no-reference blockiness visibility metric for the least squares trained filters which makes the algorithm capable of dealing with various qualities of input sequences.. In the following, we first briefly introduce the classification based least squares filters in Section 2. Section 3 analyzes the properties of the algorithm on coding artifact reduction, when the original images are degraded with different qualities and using different compression methods. The new trained filters that are adaptive to the visual quality of the input sequence, which is indicated by a block visibility metric, are proposed in Section 4. We evaluate the performance of the proposed algorithm in contrast to the original trained filters and other adaptive filters. Finally, we draw our conclusion in Section 5. In this paper, we develop the classification based least square trained filters for compression artifacts removal using the quality control mechanism. The quality metric we adopt shows robust behavior in blockiness visibility measurement. For an input sequence with unknown quality, the proposed algorithm can choose the most suitable LUT for filtering based on the block visibility metric. The inclusion of the quality control mechanism coupled with the effectiveness of the blockiness metric enables the least squares trained filters to be a decent choice for deblocking images and videos of various qualities.. In future work, a local quality/blockiness metric will be designed to further improve the local adaptability of the algorithm.

O O O O O O O O B I I O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Distances on Rhombus Tilings The rhombus tilings of a simply connected domain of the Euclidean plane are known to form a flip-connected space (a flip is the elementary operation on rhombus tilings which rotates 180• a hexagon made of three rhombi). Motivated by the study of a quasicrystal growth model, we are here interested in better understanding how "tight" rhombus tiling spaces are flip-connected. We introduce a lower bound (Hamming-distance) on the minimal number of flips to link two tilings (flip-distance), and we investigate whether it is sharp. The answer depends on the number n of different edge directions in the tiling: positive for n = 3 (dimer tilings) or n = 4 (octogonal tilings), but possibly negative for n = 5 (decagonal tilings) or greater values of n. A standard proof is provided for the n = 3 and n = 4 cases, while the complexity of the n = 5 case led to a computer-assisted proof (whose main result can however be easily checked by hand). The discovering in the early 1980's of non-periodic crystals, soon called quasicrystals, renewed the interest in tilings. Indeed, tilings show that a global property such as non-periodicity can sometimes be enforced by purely local (hence physically realistic) constraints, namely the way neighbor tiles can match. The most celebrated example is probably the Penrose tilings, which are decagonal rhombus tilings of the plane whose non-periodicity can be enforced just by specifying the way rhombi are allowed to fit around a vertex. Several books have since been written on quasicrystals, and even those focusing on physical properties have a chapter on tilings, with Penrose tilings in the first place (the interested reader can easily found numerous references).. Tilings are also widely studied in statistical mechanics, in particular dimer tilings. These are tilings of a bounded subset of a planar grid either by dominoes (square grid) or rhombi (triangular grid). The aim is to understand statistical properties of the set of tilings of a given domain, called a tiling space. The interested reader can refer, e.g., to the recent survey [12] .. Strongly related to tilings is the notion of flip (also called phason-flip or simply phason). A flip is the local operation which acts on rhombus tiling by rotating 180. • a hexagon made of three rhombi 1 . Flips may model transformations which occur in real (quasi)crystals [14] . The general aim, which inspires this paper, is to understand the flip dynamics on tiling spaces.. In particular, the flip dynamics is said to be ergodic if the more flips are performed on a tiling, the closer it is from a typical tiling (that is, a tiling chosen uniformly at random in the tiling space). Ergodicity thus provides, when it holds, an algorithmic method to numerically investigate statistical properties of tiling spaces. Ergodicity have been proven first for dimer tilings [15] and then for any rhombus tilings [11] (for simply connected domains).. However, ergodicity does not tell anything about the number of flips which shall be performed before being relatively close to a typical tiling. This is not only a drawback from a computational viewpoint (how far a simulation shall be run?), but can also lead to physical mispredictions: ergodicity can yield properties that have no chance to be experimentally observed because the flip dynamics is much too slow. Ergodicity results can be refined by studying the mixing time of the Markov process whose steps consist in performing a flip on a vertex chosen uniformly at random (when possible). A polynomial bound on the mixing time of dimer tilings have been obtained in [16] , relying on the analysis of a related Markov chain [13, 17] . Numerical experiments suggest that a similar bound holds for rhombus tilings whose rhombi are defined over four possible edge directions (octogonal tilings) [8] .. In this context, the motivation of this paper comes from the model of quasicrystal growth introduced in [2] . Starting from a typical tiling (assumed to be an entropically-stabilized quasicrystal at high temperature), flips are randomly performed, with the probability of each flip depending on its neighborhood (assumed to model the energy of the quasicrystal). This process can be seen as the minimization of the energy of a tiling by a stochastic gradient descent in the tiling space. The question is whether a minimal energy tiling is reached (a sort of partial ergodicity), and at which rate (mixing time). In other word, the question is whether the shortest paths (in terms of flips) of tiling spaces are enough "regular" to be easily found by the gradient descent. As an attempt to measure this regularity, this paper introduces a distance, called Hamming-distance, and compares it to the above flip-distance.. The main result obtained in this paper is that rhombus tiling spaces are regular when rhombi are defined over at most four edge directions (dimer or octogonal tilings), but loose this regularity for five directions (decagonal tilings) or more. Theorem 1 formally states this in Section 4. Before, Section 2 and 3 first introduce main definitions, especially flip-and Hammingdistances.. Let us also mention that flips can be defined on tilings of the space by rhomboedra 2 . Ergodicity does not always hold in such cases [7] , and our results can be extended to show that rhomboedra tiling spaces are not regular, except when there is only one rhombohedron up to isometry (what is already known), and maybe for a finite number of small domains.

O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B O O O O O B O B O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Optimization Based on Bacterial Chemotaxis Abstract-We present an optimization algorithm based on a model of bacterial chemotaxis. The original biological model is used to formulate a simple optimization algorithm, which is evaluated on a set of standard test problems. Based on this evaluation, several features are added to the basic algorithm using evolutionary concepts in order to obtain an improved optimization strategy, called the bacteria chemotaxis (BC) algorithm. This strategy is evaluated on a number of test functions for local and global optimization, compared with other optimization techniques, and applied to the problem of inverse airfoil design. The comparisons show that on average, BC performs similar to standard evolution strategies and worse than evolution strategies with enhanced convergence properties. I N THE FIELD of optimization, many researchers have been inspired by biological processes such as evolution [1] , [2] or the food-searching behavior of ants [3] to develop new optimization methods such as evolutionary algorithms or ant codes. These techniques have been found to perform better than the classical heuristic or gradient-based optimization methods, especially when faced with the problem of optimizing multimodal, nondifferentiable, or discontinuous functions. Examples of applying these biologically inspired strategies in the field of engineering range from aerodynamic design (e.g., [4] ) to job-shop scheduling problems [5] .. Another biologically inspired optimization method is the chemotaxis algorithm, pioneered by Bremermann [6] and his coworkers [7] , [8] , proposed by analogy to the way bacteria react to chemoattractants in concentration gradients. This algorithm were analyzed for chemotaxis in a three-dimensional (3-D) gradient by Bremermann [6] and employed for the training of neural networks [7] , [8] . A similar algorithm is the guided accelerated random search technique [9] , which was applied to optimize parameters in flight control systems [9] , [10] and to optimize perceptrons [11] . Whereas these algorithms contain the fundamental concepts of bacteria chemotaxis (BC), more recent biological findings have provided further details and models of the process. This paper presents a novel optimization strategy whose roots are found in an extended biological model of bacterial chemotaxis [12] , [13] . The basic model and the resulting optimization algorithm differ from the chemotaxis algorithm as follows.. 1) The bacteria from our basic model incorporate information from concentration values. 2) They do not keep on running in the same direction in case of increasing concentrations of the chemoattractants. 3) Beyond studying the biological model, the main part of our work is dedicated to the further development of the basic strategy to yield a powerful, robust optimization technique. Bacteria are single-cell organisms, one of the simplest form of life developed on earth. Despite their "simplicity," they acquire information about their environment, orient themselves in this environment, and use this information efficiently to survive. This reaction of the organism to its environment has been subject to intensive research in the past decades, see, e.g., [14] - [17] . It is also interesting for scientists in the field of optimization to study the bacterial behavior. Like bacteria, optimization algorithms may be viewed as entities that gather information about a function landscape and then use this information to move to an optimum. At the same time, the simplicity and robustness of the process of bacterial chemotaxis suggest a starting point for constructing an optimization algorithm.. Although it has been found that bacteria do share information among each other [18] , not much is known presently about the communication patterns. Generally, bacteria are considered individuals and social interaction is not used in the models. These models differ from the interaction models for the behavior of social insects (such as ants, bees, wasps, or termites) that are viewed as systems with collective intelligence resulting in powerful problem-solving capabilities [19] .. The term taxis refers to the locomotory response of a cell to its environment. In a taxis, a cell responds such that it changes both its direction and the duration of the next movement step. The tactic response requires some directional information from the environment that bacteria obtain by comparing an environmental property at two different time steps. If the tactic response is related to information about chemical concentrations (that may be either attractants or repellents), it is called chemotaxis.. For optimization purposes, we concentrated on studying microscopic models that consider the chemotaxis of a single bacterium instead of macroscopic models that analyze the movement of bacteria colonies.. Our studies are based on a microscopic model for bacterial chemotaxis proposed by Berg and Brown [12] and Dahlquist et al. [13] . Berg and Brown [12] analyze the chemotaxis toward amino acids of the bacterium Escherichia coli. The analysis provides experimental measurements of parameters in the model. Dahlquist et al. [13] study bacterial chemotaxis toward amino acids of the bacterium Salmonella typhimurium. They present a mathematical model and present experimental results that validate their model. In the original model, the 3-D motion of a bacterium is approximated and the model parameters depend on the particular bacterial environment and on the experimental setup. For preliminary studies, an optimization algorithm is formulated based on the biological model. This algorithm is used for optimizing two-dimensional (2-D) test functions and contains model parameters that are selected a priori.. After studying the searching behavior of bacteria with this basic model, we enhance it for optimization purposes [20] . The first modification regards the determination of model parameters that will be referred to as strategy parameters in the following. The search for appropriate strategy parameters is inspired by nature. From an evolutionary perspective, bacteria adapt their motility properties so that they have a better chance to survive in changing environments. Our approach finds its counterpart in this evolutionary idea: using an evolution strategy (ES), we optimize the strategy parameters by minimizing the number of iterations that the virtual bacterium needs from the start to the optimum. The second modification regards the extension of the 2-D to an -dimensional model.. Further improvements include the automatic modification of strategy parameters according to the properties of the optimization function, a possibility of escaping plateaus, and additional features to facilitate the search of global optima. These improvements yield a new optimization algorithm called the BC algorithm.. To assess its properties, BC is evaluated on standard test problems, its performance is compared against other optimization algorithms, and it is finally applied to the optimization of airfoil profile shapes with an inverse design technique. This paper is organized as follows. Section II describes the basic bacterial chemotaxis model in two dimensions and tests it on various standard optimization problems. How appropriate parameters for the bacteria algorithm are chosen is shown in Section III. Section IV presents the extension of the bacteria algorithm from two to dimensions and improvements for the -dimensional strategy, yielding the BC strategy, which is applied on test functions. Convergence properties of the BC strategy are compared with those of other optimization algorithms in Section V. In Section VI, the BC code is further extended for the search of global optima. The application of the BC strategy on the inverse airfoil design is shown in Section VII and conclusions are finally drawn in Section VIII.

O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


The Power of Amnesia: Learning Probabilistic Automata with Variable Memory Length We propose and analyze a distribution learning algorithm for variable memory length Markov processes. These processes can be described by a subclass of probabilistic finite automata which we name Probabilistic Suffix Automata (PSA). Though hardness results are known for learning distributions generated by general probabilistic automata, we prove that the algorithm we present can efficiently learn distributions generated by PSAs. In particular, we show that for any target PSA, the KL-divergence between the distribution generated by the target and the distribution generated by the hypothesis the learning algorithm outputs, can be made small with high confidence in polynomial time and sample complexity. The learning algorithm is motivated by applications in human-machine interaction. Here we present two applications of the algorithm. In the first one we apply the algorithm in order to construct a model of the English language, and use this model to correct corrupted text. In the second application we construct a simple stochastic model for E.coli DNA. Statistical modeling of complex sequences is a fundamental goal of machine learning due to its wide variety of natural applications. The most noticeable examples of such applications are statistical models in human communication such as natural language, handwriting and speech (Jelinek, 1985 , Nadas, 1984 , and statistical models of biological sequences such as DNA and proteins (Krogh, Mian & Haussler, 1993) .. These kinds of complex sequences clearly do not have any simple underlying statistical source since they are generated by natural sources. However, they typically exhibit the following statistical property, which we refer to as the short memory property. If we consider the (empirical) probability distribution on the next symbol given the preceding subsequence of some given length, then there exists a length L (the memory length) such that the conditional probability distribution does not change substantially if we condition it on preceding subsequences of length greater than L.. This observation lead Shannon, in his seminal paper (Shannon, 1951) , to suggest modeling such sequences by Markov chains of order L > 1, where the order is the memory length of the model. Alternatively, such sequences may be modeled by Hidden Markov Models (HMMs) which are more complex distribution generators and hence may capture additional properties of natural sequences. These statistical models define rich families of sequence distributions and moreover, they give efficient procedures both for generating sequences and for computing their probabilities. However, both models have severe drawbacks. The size of Markov chains grows exponentially with their order, and hence only very low order Markov chains can be considered in practical applications. Such low order Markov chains might be very poor approximators of the relevant sequences. In the case of HMMs, there are known hardness results concerning their learnability which we discuss in Section 1.1.. In this paper we propose a simple stochastic model and describe its learning algorithm. It has been observed that in many natural sequences, the memory length depends on the context and is not fixed. The model we suggest is hence a variant of order L Markov chains, in which the order, or equivalently, the memory, is variable. We describe this model using a subclass of Probabilistic Finite Automata (PFA), which we name Probabilistic Suffix Automata (PSA) .. Each state in a PSA is labeled by a string over an alphabet Σ. The transition function between the states is defined based on these string labels, so that a walk on the underlying graph of the automaton, related to a given sequence, always ends in a state labeled by a suffix of the sequence. The lengths of the strings labeling the states are bounded by some upper bound L, but different states may be labeled by strings of different length, and are viewed as having varying memory length. When a PSA generates a sequence, the probability distribution on the next symbol generated is completely defined given the previously generated subsequence of length at most L. Hence, as mentioned above, the probability distributions these automata generate can be equivalently generated by Markov chains of order L, but the description using a PSA may be much more succinct. Since the size of order L markov chains is exponential in L, their estimation requires data length and time exponential in L.. In our learning model we assume that the learning algorithm is given a sample (consisting either of several sample sequences or of a single sample sequence) generated by an unknown target PSA M of some bounded size. The algorithm is required to output a hypothesis machineM , which is not necessarily a PSA but which has the following properties.M can be used both to efficiently generate a distribution which is similar to the one generated by M , and given any sequence s, it can efficiently compute the probability assigned to s by this distribution.. Several measures of the quality of a hypothesis can be considered. Since we are mainly interested in models for statistical classification and pattern recognition, the most natural measure is the Kullback-Leibler (KL) divergence. Our results hold equally well for the variation (L 1 ) distance and other norms, which are upper bounded by the KL-divergence. Since the KL-divergence between Markov sources grows linearly with the length of the sequence, the appropriate measure is the KL-divergence per symbol. Therefore, we define an -good hypothesis to be an hypothesis which has at most KL-divergence per symbol to the target source.. In particular, the hypothesis our algorithm outputs, belongs to a class of probabilistic machines named Probabilistic Suffix Trees (PST). The learning algorithm grows such a suffix tree starting from a single root node, and adaptively adds nodes (strings) for which there is strong evidence in the sample that they significantly affect the prediction properties of the tree.. We show that every distribution generated by a PSA can equivalently be generated by a PST which is not much larger. The converse is not true in general. We can however characterize the family of PSTs for which the converse claim holds, and in general, it is always the case that for every PST there exists a not much larger PFA that generates an equivalent distribution. There are some contexts in which PSAs are preferable, and some in which PSTs are preferable, and therefore we use both representation in the paper. For example, PSAs are more efficient generators of distributions, and since they are probabilistic automata, their well defined state space and transition function can be exploited by dynamic programming algorithms which are used for solving many practical problems. In addition, there is a natural notion of the stationary distribution on the states of a PSA which PSTs lack. On the other hand, PSTs sometimes have more succinct representations than the equivalent PSAs, and there is a natural notion of growing them.. Stated formally, our main theoretical result is the following. If both a bound L, on the memory length of the target PSA, and a bound n, on the number of states the target PSA has, are known, then for every given 0 < < 1 and 0 < δ < 1, our learning algorithm outputs an -good hypothesis PST, with confidence 1 − δ, in time polynomial in L, n, |Σ|, 1 and 1 δ . Furthermore, such a hypothesis can be obtained from a single sample sequence if the sequence length is also polynomial in a parameter related to the rate in which the target machine converges to its stationary distribution. Despite an intractability result concerning the learnability of distributions generated by Probabilistic Finite Automata (Kearns, et al., 1994) , that is described in Section 1.1, our restricted model can be learned in a PAC-like sense efficiently. This has not been shown so far for any of the more popular sequence modeling algorithms.. We present two applications of the learning algorithm. In the first application we apply the algorithm in order to construct a model of the English language, and use this model to correct corrupted text. In the second application we construct a simple stochastic model for E.coli DNA. Combined with a learning algorithm for a different subclass of probabilistic automata (Ron, Singer & Tishby, 1995) , the algorithm presented here is part of a complete cursive handwriting recognition system . The most powerful (and perhaps most popular) model used in modeling natural sequences is the Hidden Markov Model (HMM). A detailed tutorial on the theory of HMMs as well as selected applications in speech recognition is given by Rabiner (Rabiner, 1989) . A commonly used procedure for learning an HMM from a given sample is a maximum likelihood parameter estimation procedure that is based on the Baum-Welch method (Baume, et al., 1970 , Baume, 1972 (which is a special case of the EM (Expectation-Maximization) algorithm (Dempster, Laired & Rubin, 1977) ). However, this algorithm is guaranteed to converge only to a local maximum, and thus we are not assured that the hypothesis it outputs can serve as a good approximation for the target distribution. One might hope that the problem can be overcome by improving the algorithm used or by finding a new approach. Unfortunately, there is strong evidence that the problem cannot be solved efficiently.. Abe and Warmuth (Abe & Warmuth, 1992 ) study the problem of training HMMs. The HMM training problem is the problem of approximating an arbitrary, unknown source distribution by distributions generated by HMMs. They prove that HMMs are not trainable in time polynomial in the alphabet size, unless RP = NP. Gillman and Sipser (Gillman & Sipser, 1994 ) study the problem of exactly inferring an (ergodic) HMM over a binary alphabet when the inference algorithm can query a probability oracle for the longterm probability of any binary string. They prove that inference is hard: any algorithm for inference must make exponentially many oracle calls. Their method is information theoretic and does not depend on separation assumptions for any complexity classes. Natural simpler alternatives, which are often used as well, are order L Markov chains (Shannon, 1951 , Good, 1969 , also known as n-gram models. As noted earlier, the size of an order L Markov chain is exponential in L and hence, if we want to capture more than very short term memory dependencies in the sequences, of substantial length in the sequences, then these models are clearly not practical.. Höffgen (Höffgen, 1993 ) studies families of distributions related to the ones studied in this paper, but his algorithms depend exponentially and not polynomially on the order, or memory length, of the distributions. Freund et. al. (Freund, et al., 1993) point out that their result for learning typical deterministic finite automata from random walks without membership queries, can be extended to learning typical PFAs. Unfortunately, there is strong evidence indicating that the problem of learning general PFAs is hard. Kearns et. al. (Kearns, et al., 1994) show that PFAs are not efficiently learnable under the assumption that there is no efficient algorithm for learning noisy parity functions in the PAC model.. The machines used as our hypothesis representation, namely Probabilistic Suffix Trees (PSTs), were introduced (in a slightly different form) in (Rissanen, 1983) and have been used for other tasks such as universal data compression (Rissanen, 1983 , Rissanen, 1986 , Weinberger, Lempel & Ziv, 1982 , Willems, Shtarkov & Tjalkens, 1993 .. Perhaps the strongest among these results (which has been brought to our attention after the completion of this work) and which is most tightly related to our result is (Willems, Shtarkov & Tjalkens, 1993) . This paper describes an efficient sequential procedure for universal data compression for PSTs by using a larger model class. This algorithm can be viewed as a distribution learning algorithm but the hypothesis it produces is not a PST or a PSA and hence cannot be used for many applications. Willems et. al. show that their algorithm can be modified to give the minimum description length PST. However, in case the source generating the examples is a PST, they are able to show that this PST convergence only in the limit of infinite sequence length to that source.. Vitter and Krishnan (Vitter & Krishnan, 1991 , Krishnan & Vitter, 1993 ) adapt a version of the Ziv-Lempel data compression algorithm (Ziv & Lempel, 1978) to get a page prefetching algorithm, where the sequence of page accesses is assumed to be generated by a PFA. They show that the page fault rate of their algorithm converges to the page fault rate of the best algorithm that has full knowledge of the source. This is true for almost all page access sequences (in the limit of the sequence length). Laird and Saul (Laird & Saul, 1994) describe a prediction algorithm which is similar in spirit to our algorithm and is based on the Markov tree or Directed Acyclic Word Graph approach which is used for data compression (Blumer, 1990) . They do not analyze the correctnes of the algorithm formally, but present several applications of the algorithm.

O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Multilevel Monte Carlo method for parabolic stochastic partial differential equations We analyze the convergence and complexity of multilevel Monte Carlo discretizations of a class of abstract stochastic, parabolic equations driven by square integrable martingales. We show under low regularity assumptions on the solution that the judicious combination of low order Galerkin discretizations in space and an Euler–Maruyama discretization in time yields mean square convergence of order one in space and of order 1/2 in time to the expected value of the mild solution. The complexity of the multilevel estimator is shown to scale log-linearly with respect to the corresponding work to generate a single path of the solution on the finest mesh, resp. of the corresponding deterministic parabolic problem on the finest mesh. Stochastic partial differential equations (SPDEs for short) are increasingly used as models in engineering and the sciences. We mention only the pricing of energy derivative contracts, porous media flows, filtering, and interest rate models. Accordingly, their efficient numerical solution has received more and more attention in recent years. Most of the numerical analysis of solution methods for stochastic partial differential equations has been devoted to identifying sufficient smoothness conditions on the data (among other parameters, covariance spectrum and smoothness of initial data) for certain discretization schemes to achieve a certain order of strong resp. weak or even pathwise convergence. See, e.g. the survey [15] and the references therein, and also [11] , [20] , [2] , [3] , [4] for recent results.. Except for results on pathwise convergence as, e.g., in [7] , [3] , [4] , motivated by applications in numerical stochastic optimal control, commonly, moments of the solution such as first and second moments of the random solution are of main interest in applications. In numerical solution schemes for stochastic partial differential equations, such moments are approximated by Monte Carlo simulation, i.e., by averaging possibly large ensembles of approximated (in physical space and time) solution paths. We thus distinguish three principal sources of discretization errors in such moment approximations: spatial discretization, e.g. by Finite Elements or Finite Differences, time stepping errors due to e.g. Euler-Maruyama or Milstein time stepping and, finally, the sampling error incurred by replacing the mathematical expectations with finite ensemble averages. The convergence rate 1/2 in mean square of Monte Carlo estimates (which is not improvable as can be seen in the proof of Lemma 4.1) combined with the high cost of approximating sample paths (which is due to the low spatial and temporal regularity of partial differential equations driven by noise) renders the approximate solution of stochastic partial differential equations costly.. This effect is, to a lesser extent, already present in the context of Itô stochastic (ordinary) differential equations. Recently, it was observed in [12] , [13] that substantial efficiency gains in numerical simulation can be achieved by the use of so-called multilevel path simulation techniques in connection with Monte Carlo sampling. However, in [14] the authors show that multilevel Monte Carlo does not converge for equations with superlinearly growing coefficients if an inappropriate building block is used.. Analogous multilevel ideas have been proven successfully in the context of partial differential equations with random input data. We mention only [6] for elliptic partial differential equations with random coefficients, and [23] for Finite Volume solvers of scalar hyperbolic conservation laws with random initial data.. The analysis of a multilevel Monte Carlo (MLMC) discretization technique for parabolic stochastic partial differential equations driven by square integrable martingales is the purpose of the present paper. The multilevel Monte Carlo approach uses hierarchic meshes for the space and time approximation. A combination of a low number of Monte Carlo samples on very fine grids and increasing sample sizes on coarser meshes guarantees an optimal balance between the computational effort for sampling on one hand and solving the corresponding partial differential equation on the other. Using low-order Euler-Maruyama time stepping and space discretizations of low regularity, we give a-priori estimates on the mentioned three error contributions. We bound the strong error in mean square in Theorem 4.3 for the singlelevel discretization, i.e., the difference of the mild and the approximate solution for a fixed discretization mesh. This bound is the basis of the multilevel Monte Carlo error bound in Theorem 4.5 which explicitly contains error bounds in terms of the discretization parameters time step, space step, and the Monte Carlo sample size. Importantly, in the error analysis close to minimal assumptions on the spectrum of the covariance operator Q of the driving noise and on the initial data X 0 are imposed. The resulting multilevel Monte Carlo error bound in Theorem 4.5 is used to minimize the number of Monte Carlo samples at each discretization level in order to balance the statistical sampling with the spatial and temporal discretization errors. In Theorem 4.6, the resulting sampling numbers and a corresponding estimate of the asymptotic total work are given. The work estimate is shown to be superior to the corresponding work estimate for the singlelevel Monte Carlo method.. In the concluding remarks, we state, based on [4] , multilevel Monte Carlo results for the Milstein scheme, in connection with a low order spatial Finite Element discretization. This discretization is shown to yield twice the (strong) convergence rate of the method based on the Euler-Maruyama time stepping scheme, however under stronger assumptions on the smoothness of the initial data and on the decay of the spectrum of the covariance operator. The approach towards the analysis of discretization schemes can be generalized to higher order discretizations. Since more sophisticated spatial and temporal discretizations exhibit higher convergence rates only under stronger regularity assumptions on the data, and since multilevel Monte Carlo variants of such discretizations are dominated by the sampling, the overall computational work for such higher order schemes is dominated once more by the work for sample path generation. In this paper we gave an a-priori error and complexity analysis of a Galerkin EulerMaruyama discretization combined with multilevel Monte Carlo sampling for the numerical estimation of expectations of solutions of a class of parabolic partial differential equations driven by square integrable martingales. We proved that the multilevel Monte Carlo EulerMaruyama approach lowers the computational complexity to calculate the expectation of the solution of a parabolic stochastic partial differential equation compared to a standard Monte Carlo method. The combination of different sample sizes on various subsequent discretization levels lowers the overall complexity to the complexity of one solve of the deterministic. for a given refinement level L. Expressed in degrees of freedom of the numerical method, we obtain a bound for the computational complexity of the singlelevel Monte Carlo. In other words, for d = 1 the work of multilevel Monte Carlo is dominated by the number of samples on the coarsest level. For higher space dimensions it is dominated by the effort to solve the corresponding partial differential equation. The low order of the convergence rate stems from the low regularity requirements that we assumed for the equation.. The results in the present paper could be extended in two directions. First, in [21] it is shown that under stronger assumptions on A, the requirement that the mild solution belongs to L 2 (Ω; V ) can be relaxed, while still maintaining the same order of convergence of the approximation. More precisely, if A is densely defined, self-adjoint, positive definite, and not necessarily bounded but with compact inverse, the Euler-Maruyama scheme still converges in mean square of order 1/2 in time and 1 in space for less regular choices of F and G. A second extension of the present analysis of the multilevel Monte Carlo method is to couple it with a Milstein type method as introduced in [4] . There, the authors give an approximation scheme for a more general but linear advection-diffusion problem. , where A is the diffusion and B the advection operator. Similar results can be achieved when a nonlinearity F : H → H is added and sufficient assumptions on the regularity of the coefficients and the solution are made, such that the stochastic partial differential equation reads. where we set M = W although the following also holds for any continuous martingale in M 2 b (U ). The increment in the Milstein scheme is then given by. where r(δt n A ) denotes the rational approximation of the semigroup generated by A that depends on the time approximation (with parameter δt n ) and space approximation level (with parameter ). A combination of the proof in [4] and [16] leads to the following bound, dependent on the regularity parameter α ∈ N sup t∈Θ n. where C 1 and C 2 are constants. For α = 1, which meets the prerequisites on the regularity of this paper, we would not get better convergence than for the Euler-Maruyama scheme introduced in this paper. However, for more regular data, i.e., α = 2 and. , we obtain in particular sup. Still we have the (severe) consistency constraint δt n h 2 . To compare this error bound to the result for the Euler-Maruyama scheme proven in Theorem 4.4, we employ Lemma 4.1, which results in a convergence rate of O(h 2 + δt n + N −1/2 ) for the singlelevel Monte Carlo method. To prove the result for the multilevel Monte Carlo approach, we need to change the linear interpolation of the solution given in Equation (4.4) to

B I I O O O B I I I O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O B I I I O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O


Untangling a Planar Graph A straight-line drawing δ of a planar graph G need not be plane, but can be made so by untangling it, that is, by moving some of the vertices of G. Let shift(G, δ) denote the minimum number of vertices that need to be moved to untangle δ. We show that shift(G, δ) is NP-hard to compute and to approximate. Our hardness results extend to a version of 1BendPointSetEmbeddability, a well-known graph-drawing problem.Further we define fix(G, δ) = n − shift(G, δ) to be the maximum number of vertices of a planar n-vertex graph G that can be fixed when untangling δ. We give an algorithm that fixes at least p ((log n) − 1)/ log log n vertices when untangling a drawing of an n-vertex graph G. If G is outerplanar, the same algorithm fixes at least p n/2 vertices. On the other hand we construct, for arbitrarily large n, an n-vertex planar graph G and a drawing δG of G with fix(G, δG) ≤ √ n − 2 + 1 and an n-vertex outerplanar graph H and a drawing δH of H with fix(H, δH) ≤ 2 √ n − 1 + 1. Thus our algorithm is asymptotically worst-case optimal for outerplanar graphs. A drawing of a graph G maps each vertex of G to a distinct point of the plane and each edge uv to an open Jordan curve connecting the images of u and v. A drawing of G is plane if no two distinct edges cross, that is, intersect. By the famous theorem of Wagner [Wag36] , Fáry [Fár48] , and Stein [Ste51] , any planar graph admits a plane straight-line drawing, that is, a drawing that maps edges to straight-line segments. Obviously not every straight-line drawing of a planar graph is plane. In this paper we are exclusively interested in such straight-line drawings. Thus by a drawing we will always mean a straight-line drawing. Since a (straight-line) drawing is completely defined by the position of the vertices, moving a vertex is a natural operation to modify such a drawing. If a drawing is to be made plane-or untangled -by successively moving vertices, it is desirable to move as few vertices as possible. The smaller the number of moves, the less likely it is that an observer gets confused, that is, the more likely the observer's mental map [MELS95] is preserved during a sequence of changes. A recreational version of the problem of minimizing the number of moves is given by Tantalo's popular on-line game Planarity [Tan07] , where the aim is to untangle a straight-line drawing as quickly as possible, again by vertex moves. Actually, in Tantalo's game an additional difficulty for the player is the fixed size of the screen; Liske's [Lis08] version of the game allows rescaling and hence is fully equivalent to untangling.. We define the vertex-shifting distance d between two drawings δ and δ ′ of a graph G = (V, E) to be the number of vertices of G whose images under δ and δ ′ differ:. Given our edit operation, d represents the edit distance for straight-line drawings of graphs (see Fig. 1 for an example). For a drawing δ of a planar graph G, we denote by shift(G, δ) the minimum number of vertices that need to be moved in order to untangle δ. In some sense shift(G, δ) measures the distance of δ from planarity. This suggests the following computational problem. In this paper, we have presented several new results on the problem of untangling a given drawing of a graph, a problem originally introduced by Watanabe [Wat98] for the special case of cycles.. On the computational side, we have proved that MinShiftedVertices is NP-hard and also hard to approximate; we also showed that our proof technique extends to another graph drawing problem, namely 1BendPointSetEmbeddability with given vertex-point correspondence. Related questions that remain open are the inapproximability of MaxFixedVertices and the hardness of MaxFixedVertices and MinShiftedVertices for special classes of graphs such as cycles. We have shown that all these problems lie in PSPA CE, but do they also lie in N P? Also, we are not aware of any result in the direction of parameterized complexity.. On the combinatorial side, Table 1 summarizes the best currently known worst-case bounds for untangling several important classes of planar graphs. It reveals that the gap for general planar graphs is probably the most interesting remaining open problem in the field.

B O O O O B I O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O


The business process modelling ontology In this paper we describe the Business Process Modelling Ontology (BPMO), which is part of an approach to modelling business processes at the semantic level, integrating knowledge about the organisational context, workflow activities and Semantic Web Services. We harness knowledge representation and reasoning techniques so that business process workflows can: be exposed and shared through semantic descriptions; refer to semantically annotated data and services; incorporate heterogeneous data though semantic mappings; and be queried using a reasoner or inference engine. In this paper we describe our approach and evaluate BPMO through a use case. Business organisations today need agility and flexibility to deal with highly dynamic environments, providing ever-changing services and products as well as in interacting with diverse customers and partners. A significant problem in the area of Business Process Management (BPM) lies in bridging between the organisational context, the diverse process workflow notations, and the executable services that fulfill process activities, by which business analysts would like to understand, maintain and adapt their business processes.. Currently, business analysts use process modelling notations such as BPMN [11] and EPC [14] to define business process models as part of tool suites for BPM. These notations are useful at the business level, but alone they provide no inference reasoning over business processes. For example, BPMN tools are rich in controlflow constructs, but the graphical elements contain only limited textual information with no formal semantics. Some EPC-based tools such as ARIS [14] on the other hand, provide integration of different views (e.g. organisation, data and control) and levels (e.g. requirements and implementation); however mediating between these views and levels is a very complex task due to the variety of underlying representations. In addition, it is very difficult to use these notations to automatically query the business context or draw relations between existing processes or services.. In this paper we present the Business Process Modelling Ontology 1 (BPMO), which plays a key role in solving the problem above, by enabling the semantic annotation of high-level business process models, which we assume can be fulfilled by Web services. This work is part of the SUPER project 2 , which in particular provides a set of integrated ontologies for facilitating Semantic Business Process Management [6] .. The rest of this paper is structured as follows. Section 2 describes the BPMO approach. Section 3 describes the main concepts of BPMO. Section 4 illustrates the use of BPMO and shows results through a use case. Section 5 presents our conclusions and related work.

O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


BoltzTraP. A code for calculating band-structure dependent quantities A program for calculating the semi-classic transport coefficients is described. It is based on a smoothed Fourier interpolation of the bands. From this analytical representation we calculate the derivatives necessary for the transport distributions. The method is compared to earlier calculations, which in principle should be exact within Boltzmann theory, and a very convincing agreement is found. Method developments, the existence of user friendly distributed codes and the ever increasing computer power are making the calculation of band-structures, for even relatively complex materials, more and more straight forward. As several properties can be calculated from the energy bands and their derivatives, the usefulness off a generally applicable, easily portable and documented code for analysis of the bands should be clear. We have implemented and tested a method for obtaining an analytical representation of the band-structure. We have applied it to the calculation of transport coefficients. The method has been compared with an earlier calculation [11] , which in principle should be exact within Boltzmann theory, and we found a very convincing agreement.. It should be pointed out that the present method also has several advantages. First of all, when an analytical expression of the bands is found, they can be interpolated onto a finer k-mesh. Secondly, because only the energies are needed the code is easily portable to any band-structure code and furthermore does not require the storage of potentially large wavefunction files on disk. Finally, second derivatives necessary for the Hall coefficient are straightforwardly calculated which is not straightforward from the wavefunction itself. Even higher derivatives, necessary, e.g., for the calculation of magneto-resistance, could easily be calculated with the present method, but it remains to be seen whether the accuracy is high enough.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O